Frontmatter Schema

Overview

DocProcessor processes markdown files that contain YAML frontmatter blocks. The frontmatter specifies how the documentation relates to source files — either as specifications that drive code generation, as documentation that should be updated based on source files, or as transformation rules between files. The processor also supports fetching and caching URL-based related resources, allowing documentation to reference external web content as context.

Frontmatter Format

Frontmatter must be enclosed between --- delimiters at the start of the markdown file:

---
key: value
list_key:
  - item1
  - item2
---

# Document content starts here

Supported Keys

# Single file
specifies: ../src/utils/helper.kt

# Single glob pattern
specifies: ../src/**/*.kt

# Multiple patterns
specifies:
  - ../src/models/*.kt
  - ../src/utils/*.kt

# Single file
documents: ../src/main/kotlin/MyClass.kt

# Multiple source files
documents:
  - ../src/**/*.kt
  - ../src/**/*.java

# Single transform
transforms: src/(.+)\.java -> generated/$1.kt

# Multiple transforms
transforms:
  - src/models/(.+)\.java -> kotlin/models/$1.kt
  - src/utils/(.+)\.java -> kotlin/utils/$1.kt

generates:
  output: path/to/output/file
  inputs:
    - input/pattern/*.kt
    - another/input.kt

# Single generate spec
generates:
  output: ../generated/combined.kt
  inputs:
    - ../src/models/*.kt
    - ../src/utils/*.kt

# Multiple generate specs
generates:
  - output: ../generated/models.kt
    inputs:
      - ../src/models/**/*.kt
  - output: ../generated/utils.kt
    inputs:
      - ../src/utils/**/*.kt

# Single related file
related: ../shared/constants.kt

# Multiple related files and URLs
related:
  - ../shared/constants.kt
  - ../config/settings.yaml
  - ./helper-docs.md
  - https://example.com/api-spec

# Use default file modification task
task_type: FileModification
# Use a different task type
task_type: CodeReview

# Reference a JSON config file
task_config_json: ./config/my-task-config.json
# Config file in parent directory
task_config_json: ../shared/task-settings.json

# Always apply patches to existing files
overwrite: PatchExisting
# Skip processing if target exists
overwrite: SkipExisting
# Always fully overwrite
overwrite: OverwriteExisting

# Custom prompt for the AI
specifies: ../src/Main.kt
prompt: Refactor this file to use coroutines instead of callbacks

specifies: ../src/Generated.kt
template_file: ./templates/class-template.kt

specifies: ../src/Generated.kt
template_file: ./templates/class-template.kt
data_file: ./data/model-config.json

Complete Example

---
specifies:
  - ../src/api/*.kt
  - ../src/models/*.kt
documents:
  - ../src/core/Engine.kt
transforms:
  - src/legacy/(.+)\.java -> src/modern/$1.kt
generates:
  output: ../generated/api-index.md
  inputs:
    - ../src/api/**/*.kt
related:
  - ../config/api-config.yaml
  - ./api-conventions.md
  - https://example.com/api-spec
overwrite: PatchExisting
task_type: FileModification
task_config_json: ./config/api-task-config.json
prompt: Update the API layer to conform to the latest specification
---

# API Documentation

This document specifies the API layer implementation...

Processing Behavior

• Dependency Resolution: Tasks are sorted topologically so dependencies are processed before dependents. Cycles are detected and broken automatically.
• File Resolution: All paths in frontmatter are resolved relative to the markdown file's parent directory.
• Glob Expansion: Simple globs (*.kt) match files in the specified directory. Recursive globs (**/*.kt) match files in all subdirectories. Bracket patterns (file[0-9].txt) match character ranges. Question mark patterns (file?.txt) match single characters. For transforms, the source pattern is a regex (not a glob) that matches against file paths relative to the doc file's directory. Literal paths (without wildcards) are returned even if the target file doesn't exist, enabling file creation.
• Multiple Specifications: A single target file can be specified by multiple documentation files. All specifications are combined when processing.
• Overwrite Modes: The processor supports different overwrite strategies for handling existing files.

Overwrite Modes

File Modification Time Checking

For OverwriteToUpdate and PatchToUpdate modes, the processor compares the target file's last modified time against:

• The documentation file itself
• All related files specified in the frontmatter
• All source/input files that contribute to the target

If any of these are newer than the target, the target will be processed.

Task Description Generation

The processor automatically generates appropriate task descriptions based on the frontmatter type:

• For specifies/transforms: Updates target files based on documentation and specifications
• For documents: Updates documentation to reflect current source code state
• For generates: Generates output files based on documentation and input files
• Custom prompt: If a single spec has a prompt frontmatter key, that prompt is used directly as the task description
• Non-file task types: Processes the file according to the task type with documentation as context

URL Fetching and Caching

Related resources specified as URLs (http:// or https://) are automatically fetched and cached locally:

• Cache location: .doc-processor-cache/url-cache within the root directory
• Cache TTL: 1 hour (cached content older than 1 hour is re-fetched)
• HTML content is automatically simplified (scripts, styles, interactive elements removed)
• Non-HTML content is stored as-is
• Failed fetches log a warning and return null (the resource is skipped)
• Cache files use a SHA-256 hash prefix for uniqueness

Rebasing

Both DocSpec and ModificationTask support rebasing from one root directory to another. This is used when the IntelliJ action needs to adjust paths for a different working directory. URL-based related resources are preserved as-is during rebasing.

Primary Source Resolution

When determining the primary source file for overwrite mode checks, the priority is:

1. First transform's source file
2. First spec's doc file
3. First document match's first supporting file (or doc file if no supporting files)
4. First generate match's first input file (or doc file if no input files)

Error Handling

• Invalid frontmatter YAML will cause parsing to fail
• Missing required fields in generates entries (like output) will result in incomplete specifications
• Invalid regex patterns in transforms will cause matching to fail for those rules
• Unknown task_type values will log a warning and fall back to FileModification
• Invalid task_config_json paths will be stored but may cause errors during task execution
• Files without frontmatter (not starting with ---) return null (silently skipped)
• Files with unclosed frontmatter (no closing ---) return null
• Files with frontmatter but no specifies, transforms, documents, or generates keys return null
• Non-existent files referenced in related are still returned (downstream code handles them)
• URL fetch failures log a warning and return null (the resource is skipped)
• Errors processing individual target files are caught and logged; other targets continue processing

Data Structures

The frontmatter is parsed into a DocSpec containing:

TransformSpec

GenerateSpec

ModificationTaskConfig

Represents the configuration for a single modification task:

ModificationTask

Represents a complete modification task ready for execution:

Additional Processing Classes

TransformMatch

Represents a matched transformation from source to destination:

GenerateMatch

Represents a matched generation specification:

DocumentMatch

Represents a documentation update specification:

Implementation Notes

Frontmatter Parsing

The frontmatter is parsed using a custom simple YAML parser (not SnakeYAML). The parser handles string values, list values, and map values (for generates).

• Lines are split on the first colon to extract key-value pairs
• Lines without colons are ignored
• If the value after the colon is empty, the parser looks for subsequent list items (lines starting with - )
• Empty keys (colon with no value and no subsequent list items) are not added to the result map
• Values are trimmed of whitespace

Transform Pattern Matching

Transform patterns use Java regex syntax. The source pattern is matched against file paths relative to the documentation file's directory. When a match is found:

1. The regex is applied to the relative file path
2. Capture groups are extracted from the match
3. Backreferences ($0, $1, etc.) in the destination pattern are replaced with the captured values
4. The destination path is resolved relative to the documentation file's directory

IntelliJ Integration

The DocProcessorAction provides an IntelliJ IDE action that:

1. Filters selected files to markdown files (.md or .markdown extensions)
2. Creates a DocProcessor instance with the configured fast and smart models
3. Calls getAll() to collect all modification tasks from the selected files
4. Shows a DocProcessorTaskDialog with a checklist of tasks for user selection
5. Executes the first selected task via SingleTaskApp in a browser session

The action is available through the DocProcessorActionGroup which provides a submenu with all overwrite mode options:

The dialog includes an "Auto-fix issues" checkbox and displays task details including target files and related files.