Cognotik - GoogleProxy

⚙️ CrawlerTaskExecutionConfigData


{
  "search_query": "latest trends in AI orchestration",
  "max_results": 20
}

→

🔍 SeedItem Results

https://example.com/ai-orchestration

Top 10 AI Orchestration Trends for 2024

Explore how autonomous agents and proxy-based search are revolutionizing...

https://techdocs.io/agents

Building Scalable Crawler Agents

A deep dive into using GoogleProxy for dynamic seed generation in large-scale...

Runtime Configuration

These parameters are passed during task execution via CrawlerTaskExecutionConfigData.

Field	Type	Description
`search_query` *	String	The primary search term. Leading/trailing whitespace is trimmed automatically.

Output Data (SeedItem)

Field	Type	Description
`link`	String	The destination URL of the search result.
`title`	String	The page title as indexed by Google.
`additionalData`	Map	Contains `snippet`, `pagemap`, and `displayLink`.

Static Configuration

Environment variables used by the factory:

GOOGLE_SEARCH_PROXY_ENDPOINT="https://..."

Lifecycle & Error Handling

Validation: Throws IllegalArgumentException if search_query is blank.
Network: Uses java.net.http.HttpClient with a 30-second timeout.
Proxy: Requests are routed through an AWS Lambda proxy to avoid IP blocking and manage rate limits.
Resilience: Non-200 responses from the proxy trigger a RuntimeException to allow orchestration-level retries.

Kotlin Boilerplate


// Registering the GoogleProxy in a CrawlerAgentTask
val strategy = GoogleProxy().createStrategy(crawlerTask, currentUser)
val seeds = strategy.getSeedItems(
    taskConfig = CrawlerTaskExecutionConfigData(search_query = "Cognotik Documentation"),
    orchestrationConfig = currentOrchestration
)