Agent Types

File: BaseAgent.kt

All agents inherit from this class. It standardizes how inputs are converted into chat messages and how responses are returned.

Generic Types

• IInput type (e.g., List<String>, CodeRequest)
• RReturn type (e.g., String, ParsedResponse<T>)

Key Properties

• model: The ChatInterface (the LLM provider, e.g., OpenAI, Anthropic).
• temperature: Controls randomness (0.0 for deterministic, 1.0 for creative).
• prompt: The system prompt/instructions.
• name: Optional identifier for the agent.

Key Methods

• respond(input, messages): Core method that processes input and returns a result. Can be called with custom messages.
• answer(input): Convenience method that automatically generates chat messages from input and calls respond().
• chatMessages(input): Abstract method that subclasses must implement to convert input into chat messages.
• withModel(model): Abstract method that returns a new agent instance with a different model.
• response(messages, model): Protected method that sends messages to the model and returns the raw response.

File: ChatAgent.kt

The standard agent for conversational text generation. It takes a history of strings and returns a raw string response.

• InputList<String> (A list of user messages/questions)
• OutputString (The raw content of the LLM's response)

It constructs a chat history starting with the system prompt, followed by each string in the input list as a separate user message.

val agent = ChatAgent(
    prompt = "You are a helpful assistant.",
    model = myChatModel,
    temperature = 0.7
)
val response = agent.respond(listOf("Hello", "Tell me a joke"))

File: ParsedAgent.kt

Converts natural language input into a specific class instance (T).

• InputList<String> (Instructions)
• OutputParsedResponse<T> (Contains the raw text and the deserialized object obj)

Key Features

• Schema Generation: Uses TypeDescriber to inject YAML schemas into prompts.
• Validation: Runs ValidatedObject logic after deserialization.
• Retries: If JSON parsing fails, it can retry with adjusted parameters (deserializerRetries).
• Single vs. Two-Stage: singleStage = false (default) uses a dedicated parser agent to extract and format JSON from the response.
• Custom Parser Prompt: The parserPrompt parameter allows additional instructions to the parser agent.

File: ParsedResponse.kt

A wrapper holding both the raw text and the deserialized obj.

• text: The raw response from the LLM.
• obj: The deserialized object of type T.
• clazz: The class of the deserialized object.
• map(cls, fn): Transforms the object using a function while preserving the original text.

val response: ParsedResponse<User> = agent.respond(listOf("Extract user info"))
val transformed: ParsedResponse<UserDTO> = response.map(UserDTO::class.java) { user ->
    UserDTO(user.name.uppercase(), user.email)
}

File: ParsedImageAgent.kt

Similar to ParsedAgent, but accepts images as input. It performs Visual Question Answering (VQA) where the answer is a structured object.

• InputList<ImageAndText> (Text prompts paired with BufferedImage)
• OutputParsedResponse<T>

• Images are automatically encoded to Base64 PNG format and embedded in the request.
• Supports vision-capable models (e.g., GPT-4-Vision, Claude 3).
• Includes automatic retry logic with deserializerRetries.

File: ProxyAgent.kt

This is a "Magic" agent. It creates a dynamic Java Proxy for a given interface or class. When you call a method on the proxy, the arguments are serialized, sent to the LLM, and the LLM "executes" the logic, returning the result.

• Dynamic Proxy Creation: Uses Java reflection to intercept method calls.
• Automatic Serialization: Arguments are converted to JSON and sent to the LLM.
• Retry Logic: Automatically retries failed calls with adjusted temperature (maxRetries).
• Validation: If the return type implements ValidatedObject, validation is performed.
• Examples: You can provide example input/output pairs via addExample() to improve accuracy.
• Metrics: Track performance via the metrics property, which includes request counts per method.

interface SentimentAnalyzer {
    fun analyze(text: String): SentimentResult
}
val proxy = ProxyAgent(SentimentAnalyzer::class.java, model).create()
val result = proxy.analyze("I love this library!") // LLM determines the return value

File: CodeAgent.kt

An autonomous agent capable of writing, executing, and fixing code in a sandboxed runtime environment.

Input: CodeRequest

• messages: Chat history with role information.
• codePrefix: Code to prepend before execution (e.g., imports, setup).
• autoEvaluate: Whether to automatically execute and fix code.
• fixIterations: Number of times to attempt fixing failed code.
• fixRetries: Number of times to retry the entire coding process.

Output: CodeResult

• code: The final generated code.
• status: One of Coding, Correcting, Success, or Failure.
• result: Contains resultValue and resultOutput.
• renderedResponse: The LLM's text response alongside the code.

Key Components

• CodeRuntime: The environment where code runs (e.g., a Kotlin script engine, JavaScript engine).
• symbols: A map of objects injected into the script context (allows the agent to control your application).
• language: Automatically determined from the CodeRuntime (e.g., "kotlin", "javascript").
• codeInterceptor: A function that can transform generated code before execution (useful for logging, sanitization, or instrumentation).
• evalFormat: When true, instructs the LLM to structure code as parameterized functions with a final invocation.
• fallbackModel: An optional secondary model to use if the primary model fails to generate valid code.
• describer: A TypeDescriber that generates API documentation for the injected symbols.

Self-Correction Loop

If autoEvaluate is true, the agent executes the code. If it throws an exception, the agent feeds the error back to the LLM to generate a fix (up to fixIterations times). If the code still fails, it retries the entire process (up to fixRetries times).

val agent = CodeAgent(
    codeRuntime = KotlinScriptRuntime(),
    symbols = mapOf("api" to myApiClient),
    model = myChatModel,
    temperature = 0.1
)
val result = agent.respond(CodeAgent.CodeRequest(
    messages = listOf("Calculate the sum of 1 to 100" to Role.user),
    autoEvaluate = true,
    fixIterations = 3
))
println(result.code)
println(result.result.resultOutput)

File: ImageAndText.kt

A simple data class that pairs text with an optional image.

• text: Text content.
• image: Optional BufferedImage (null if not present).

File: ImageGenerationAgent.kt

Generates images from text descriptions.

• InputList<String> (User instructions)
• OutputImageAndText (The generated image and the refined prompt used)

Key Components

• textModel: A ChatInterface used to refine the user's request into an optimized image generation prompt.
• imageModel: The image generation model (e.g., DALL-E 3).
• imageClient: The ImageClientInterface that communicates with the image generation service.
• width / height: Dimensions of the generated image (default: 1024x1024).

Workflow

1. Refinement: Uses the text LLM to transform the user request into an optimized image generation prompt.
2. Length Validation: If the prompt exceeds the model's maxPrompt limit, it's automatically shortened.
3. Generation: Sends the refined prompt to the ImageClientInterface.
4. Decoding: Handles both URL-based and Base64-encoded image responses.

File: ImageProcessingAgent.kt

Handles Vision tasks. It can analyze images or (depending on the backend model) edit them.

• InputList<ImageAndText> (Text prompts paired with images)
• OutputImageAndText (Analyzed text and optionally a modified image)

• Encodes images to Base64 PNG format and sends them alongside text to a vision-capable model.
• Supports multimodal responses (text and/or image).
• Falls back to input image if the model doesn't return an image.