The runbook framework

Junior is a small runbook framework — an executor of any runbook. A runbook owns one task (collect → render → publish against its own schemas); a set of shared, schema-agnostic LLM harnesses drive the model for every runbook. The built-in runbooks review code diffs — the flagship, not the boundary — but nothing in the framework is git-shaped.

This page is the deep-dive on those two abstractions. For the high-level picture see Architecture; to add your own, see Adding runbooks & harnesses.

Why a framework

The data flowing between phases is git/MR-shaped — ReviewContext in, ReviewOutput out:

project_id, mr_iid, mr_title, source_branch, target_branch, labels,
full_diff, changed_files, comments, ...

That’s the schema of one domain (“review a code diff”). The moment a fork wants a runbook that reviews something else — a Jira ticket, a design doc, a release checklist — those fields are dead weight and the wrong shape. So the data shape flowing between phases is not global: each runbook brings its own. The stable part — collect → process → publish, driven by a shared LLM caller — is what the framework keeps fixed.

Runbook × Harness

Junior has exactly two independent extension points. Keeping them independent is the whole design. There is no third “platform” selector — platform (GitLab / GitHub / local) is folded into a runbook’s own collect/publish.

  RUNBOOK (domain)                    LLM HARNESS (shared)
  one class per domain                 one HARNESS per LLM driver
  ┌───────────────────────────┐        ┌───────────────────────────┐
  │ local_review              │        │ claudecode                │
  │ github_pr_review          │  uses  │ codex                     │
  │ gitlab_pr_review          │ ─────► │ pydantic                  │
  │ …a fork's runbook        │        │ deepagents                │
  └───────────────────────────┘        └───────────────────────────┘
  brings its own Context + Result      schema-agnostic: takes the
  schemas, collect/render/publish      output_schema as a parameter

• A Runbook is the unit a fork implements. It owns its Context and Result schemas and the domain logic (collect, render-for-LLM, publish). The platform variants (GitLab/GitHub/local) are runbooks in the same family, not a separate selector.
• A Harness is shared across all runbooks. It knows nothing about code, Jira, or diffs — it takes a system prompt, a user message, and a target output schema, calls the model, and returns a validated object of that schema plus token usage.

The key move: the output schema is a parameter of the harness, not a hard-coded type. That’s what lets one set of harnesses serve every domain. The code-review runbooks ask the harness for ReviewOutput; a fork’s JiraReview asks the same harness for TicketAssessment.

Interfaces (ABC, not Protocol)

Both abstractions live in src/junior/runbook/base.py as abstract base classes, deliberately, because this is a forkable framework:

• Runtime enforcement. Forget a method → TypeError: Can't instantiate abstract class … immediately. Protocol only checks under a type checker, which a forker may never run, surfacing mistakes as obscure AttributeErrors later.
• An obvious extension anchor. class JiraReview(Runbook[...]) says exactly what to subclass and what to implement.
• A home for shared defaults. system_prompt(), validate(), is_blocking(), is_empty(), and summary() all have base implementations a fork overrides only as needed.

The LLM harness

from abc import ABC, abstractmethod
from typing import ClassVar
from pydantic import BaseModel, Field, SerializeAsAny


class Usage(BaseModel):
    input_tokens: int = 0
    output_tokens: int = 0
    total_tokens: int = 0


class LLMResult(BaseModel):
    """The validated model output + runtime metadata the harness adds."""
    output: SerializeAsAny[BaseModel]      # an instance of the requested output_schema
    usage: Usage = Field(default_factory=Usage)
    errors: list[str] = Field(default_factory=list)   # partial-failure notes


class Harness(ABC):
    """A driver for one way of calling an LLM. Shared by all runbooks."""

    name: ClassVar[str]                    # matches the HarnessKind member
    file_access: ClassVar[bool] = False    # True if the harness reads repo files itself

    @abstractmethod
    def complete(
        self,
        *,
        system_prompt: str,
        user_message: str,
        output_schema: type[BaseModel],    # ← from runbook.result_model
        settings: "Settings",
    ) -> LLMResult:
        """Call the model and return an LLMResult whose .output is an
        instance of output_schema."""

The six harnesses (junior.harnesses.{claudecode,codex,pydantic,deepagents,pi,gemini}) each implement complete and expose a module-level HARNESS instance. The schema is passed in, never hard-coded, and each harness wires it through its own mechanism:

Harness	file_access	How output_schema is applied
claudecode	True	claude -p … --json-schema <schema>
codex	True	strict JSON schema via openai’s to_strict_json_schema
pydantic	False	a single structured call with output_type=output_schema
deepagents	False	a submit tool whose args_schema=output_schema
pi	True	schema embedded in the system prompt, reply validated with model_validate

[!NOTE] pydantic and deepagents make a single structured call for the requested schema — not the old per-prompt sub-agent fan-out. file_access lets a runbook skip inlining a full diff for harnesses that read files themselves.

Harnesses are resolved by the HarnessKind enum, whose value is the harness module path; registry.get_harness imports it and reads HARNESS:

class HarnessKind(_ModulePathEnum):
    PYDANTIC   = "junior.harnesses.pydantic"
    CODEX      = "junior.harnesses.codex"
    CLAUDECODE = "junior.harnesses.claudecode"
    DEEPAGENTS = "junior.harnesses.deepagents"
    PI         = "junior.harnesses.pi"

The runbook

from abc import ABC, abstractmethod
from typing import ClassVar, Generic, TypeVar
from pydantic import BaseModel

C = TypeVar("C", bound=BaseModel)    # this domain's context schema
R = TypeVar("R", bound=BaseModel)    # this domain's result schema (raw LLM output)


class Runbook(ABC, Generic[C, R]):
    """One review domain. A fork subclasses this and registers it by name."""

    name: ClassVar[str]
    context_model: ClassVar[type[BaseModel]]   # C — for JSON round-trip (--from-file)
    result_model: ClassVar[type[BaseModel]]    # R — the schema handed to the harness
    needs_git: ClassVar[bool] = False          # preflight requires a .git repo?

    # --- phase 1: collect ---
    @abstractmethod
    def collect(self, settings: "Settings") -> C:
        """Gather domain context. May dispatch internally (gitlab/github/local)."""

    # --- phase 2 inputs ---
    @abstractmethod
    def render(self, context: C, settings: "Settings", *, file_access: bool) -> str:
        """Turn context into the user message. file_access tells whether the harness
        reads files itself (so a full diff need not be inlined)."""

    SYSTEM_PROMPT: ClassVar[str] = ""        # one-line role; override per runbook

    def system_prompt(self, settings: "Settings") -> str:
        """SYSTEM_PROMPT role + the user's context.prompts. Override to append rules."""
        return merge_prompts(self.SYSTEM_PROMPT, list(settings.context.prompts))

    # --- phase 3: publish (only with --publish) ---
    @abstractmethod
    def publish(
        self, settings: "Settings", result: R, usage: Usage,
        *, errors: list[str],
    ) -> None:
        """Custom publish — runs ONLY with --publish (post / pretty render / script)."""

    def render_output(self, result: R) -> str:                    # no --publish output
        return result.model_dump_json(indent=2)                   # default: raw JSON

    # --- validation + policy hooks ---
    def validate(self, settings, *, publish_enabled) -> list[str]: return []
    def is_blocking(self, result: R) -> bool: return False   # exit 1 on CI?
    def is_empty(self, context: C) -> bool: return False      # skip the LLM?
    def summary(self, result: R) -> dict: return {}           # final "done" line
    def output_destination(self, settings, *, publish_enabled) -> str: ...  # done-log sink

The runner (generic)

run_runbook (src/junior/runbook/runner.py) ties one runbook to one harness. It is domain-agnostic and never mentions diffs or MRs. Collection is the caller’s concern — the runner takes an already-collected context — so junior run --from-file ctx.json can bypass collect():

def run_runbook(runbook, harness, context, settings, *, publish_enabled) -> LLMResult:
    result = harness.complete(
        system_prompt=runbook.system_prompt(settings),   # SYSTEM_PROMPT role + --prompts
        user_message=runbook.render(context, settings, file_access=harness.file_access),
        output_schema=runbook.result_model,         # ← R schema
        settings=settings,
    )
    if publish_enabled:                              # custom publish only with --publish;
        runbook.publish(                            # otherwise the CLI writes render_output()
            settings, result.output, result.usage, errors=result.errors,
        )
    return result

junior dry-run -o ctx.json serializes context (tagged with the runbook name so --from-file knows which runbook produced it). The exit code is 1 if runbook.is_blocking(result.output) else 0.

LLM settings

Harness selection, model, keys, and limits live in one group, settings.llm (LLMSettings) — renamed from the old settings.review:

class LLMSettings(BaseSettings):
    harness: HarnessKind = HarnessKind.CLAUDECODE      # which LLM driver
    model: str = ""                                    # "provider:model" or ""
    # API keys, max_file_size, max_tokens_per_agent …

[!NOTE] llm.harness (env HARNESS, flag --harness) is canonical. The old llm.backend / BACKEND / --backend is kept as a deprecated alias.

The system prompt is the runbook’s own: its one-line SYSTEM_PROMPT role merged with the user’s context.prompts (--prompt / --prompt-file) via the shared prompt_loader.merge_prompts(). Entries accept inline text or file://... URIs.

The built-in code-review family

The built-in code-review runbooks all review a git diff with the same ReviewContext in and ReviewOutput out, so they share a base class CodeReviewRunbook (src/junior/runbooks/code_review/base.py). It holds everything common — render, system_prompt, result_model, is_blocking, is_empty, summary, and a template-method publish that (only with --publish) assembles a ReviewResult and delegates to _post_to_platform. Without --publish, all of them emit the raw ReviewOutput as JSON via render_output. The variants differ only in collect and _post_to_platform:

Runbook	collect	_post_to_platform (--publish)	validate requires
local_review	local git diff	renders pretty Markdown locally	—
github_pr_review	GitHub PR via API	PR review comments	GITHUB_TOKEN, repo, PR number
gitlab_pr_review	GitLab MR via API	MR note + inline threads	GITLAB_TOKEN, CI_PROJECT_ID, MR iid
bitbucket_pr_review	Bitbucket DC PR via API	PR comment + inline comments	BITBUCKET_URL / TOKEN / PROJECT / REPO / PR_ID

Where the shared pieces come from:

Framework slot	Implementation
context_model	ReviewContext (junior.runbooks.code_review.models)
result_model	ReviewOutput (junior.runbooks.code_review.models)
render()	build_user_message() (code_review/render.py)
system_prompt()	SYSTEM_PROMPT role + user prompts, then build_review_prompt() adds BASE_RULES (code_review/instructions.py)
collect()	junior.collect.{local,github,gitlab,bitbucket}.collect(settings)
publish() (--publish)	_post_to_platform: local → junior.publish.local Markdown; github/gitlab/bitbucket → post_review(...)
render_output() (no --publish)	the raw ReviewOutput as JSON (default hook)
is_blocking()	any critical comment or recommendation == request_changes

Each variant imports the collect/publish helper it needs directly — there is no central resolved_collector / resolved_publisher dispatch. pre_formatted (the --publish-file shortcut) is handled by publish_prepared. At publish time the ReviewOutput is wrapped into a ReviewResult — a thin subclass of the shared LLMResult envelope that composes the LLM output with the usage token counts and errors (no flat duplication), adding only the domain pre_formatted.

Result schemas of the built-ins

The output schema (result_model) is just a parameter — serialized to the harness with model_json_schema() (the exact JSON below is what goes out as --json-schema / output_type) and validated on the way back — so the built-ins return completely different shapes. The four code-review runbooks share ReviewOutput (junior.runbooks.code_review.models); only summary is required:

{
  "$defs": {
    "Recommendation": {
      "enum": ["approve", "request_changes", "comment"],
      "title": "Recommendation", "type": "string"
    },
    "ReviewCategory": {
      "description": "Category of a review finding.",
      "enum": ["logic", "security", "bug", "naming", "optimization", "dry_violation", "kiss_violation"],
      "title": "ReviewCategory", "type": "string"
    },
    "Severity": {
      "enum": ["low", "medium", "high", "critical"],
      "title": "Severity", "type": "string"
    },
    "ReviewComment": {
      "description": "A single review comment from the AI agent.",
      "type": "object",
      "required": ["category", "severity", "message"],
      "properties": {
        "category": {"$ref": "#/$defs/ReviewCategory"},
        "severity": {"$ref": "#/$defs/Severity"},
        "message": {"title": "Message", "type": "string"},
        "file_path": {"anyOf": [{"type": "string"}, {"type": "null"}], "default": null, "title": "File Path"},
        "line_number": {"anyOf": [{"type": "integer"}, {"type": "null"}], "default": null, "title": "Line Number"},
        "suggestion": {"anyOf": [{"type": "string"}, {"type": "null"}], "default": null, "title": "Suggestion"}
      },
      "title": "ReviewComment"
    }
  },
  "type": "object",
  "required": ["summary"],
  "properties": {
    "summary": {"title": "Summary", "type": "string"},
    "recommendation": {"$ref": "#/$defs/Recommendation", "default": "comment"},
    "comments": {"items": {"$ref": "#/$defs/ReviewComment"}, "title": "Comments", "type": "array"}
  },
  "title": "ReviewOutput"
}

is_blocking() reads the result directly: any severity == "critical" or recommendation == "request_changes". An instance the model might return:

{
  "summary": "One critical SQL-injection regression; otherwise clean.",
  "recommendation": "request_changes",
  "comments": [
    {"category": "security", "severity": "critical",
     "message": "find_user() interpolates user_id into the SQL string",
     "file_path": "db.py", "line_number": 42,
     "suggestion": "Use a parameterized query."}
  ]
}

The built-in weather_advice returns something entirely unrelated — same framework, different result_model (junior.runbooks.weather.runbook):

{
  "$defs": {
    "OutfitItem": {
      "type": "object",
      "required": ["item", "reason"],
      "properties": {
        "item": {"title": "Item", "type": "string"},
        "reason": {"title": "Reason", "type": "string"}
      },
      "title": "OutfitItem"
    }
  },
  "type": "object",
  "required": ["summary"],
  "properties": {
    "summary": {"description": "one-line headline of the conditions and the plan", "title": "Summary", "type": "string"},
    "outfit": {"description": "concrete items to wear, each with a one-line reason", "items": {"$ref": "#/$defs/OutfitItem"}, "title": "Outfit", "type": "array"},
    "risks": {"description": "things to watch for (rain, UV, cold snap, wind, ice); empty if none", "items": {"type": "string"}, "title": "Risks", "type": "array"},
    "tips": {"description": "optional extras (umbrella, sunscreen, swap shoes later)", "items": {"type": "string"}, "title": "Tips", "type": "array"}
  },
  "title": "WeatherAdviceOutput"
}

Adding a runbook (Jira example)

class JiraTicket(BaseModel):                 # any shape — no git fields
    key: str
    summary: str
    description: str
    acceptance_criteria: list[str]
    comments: list[str]

class TicketAssessment(BaseModel):           # the schema the LLM must return
    verdict: Literal["ready", "needs_work"]
    gaps: list[str]
    questions: list[str]

@register_runbook
class JiraReview(Runbook[JiraTicket, TicketAssessment]):
    name = "jira_review"
    context_model = JiraTicket
    result_model = TicketAssessment
    SYSTEM_PROMPT = "You are a backlog-refinement reviewer. ..."   # one-line role

    def collect(self, settings):  ...                 # hit Jira API → JiraTicket
    def render(self, ticket, settings, *, file_access):  ...   # ticket → text
    def publish(self, settings, result, usage, *, errors):  ...   # only with --publish

JiraReview gets all five LLM harnesses, the whole prompt/config mechanism, JSON round-trip, and the CLI for free. It writes zero subprocess/LLM-calling code.

Selecting & registering a runbook

Runbook selection is explicit — there is no token auto-detection and no implicit default. Pick one with --runbook NAME, config runbook:, or env RUNBOOK. Harnesses stay on --harness:

junior run --runbook local_review                  # review the local diff
junior run --runbook gitlab_pr_review --publish    # review an MR, post results
junior run --runbook jira_review --harness codex   # a fork's runbook

The registry (src/junior/runbook/registry.py) merges runbooks from four sources:

1. Built-in — every subpackage of junior.runbooks is auto-discovered via a pkgutil scan (no hardcoded list); importing it runs its @register_runbook.

2. External plugin — a pip-installed package declaring a junior.runbooks entry point:

   [project.entry-points."junior.runbooks"]
   jira_review = "junior_jira.runbook:JiraReview"

3. Direct path — --runbook "pkg.module:ClassName" loads a Runbook subclass directly, an escape hatch for quick experiments.

4. Repo-local (local_runbooks) — load_local_runbooks() loads <project>/.junior/runbooks/*.py (@register_runbook classes) and YAML manifests driving a ScriptRunbook (any language). Executes repo code; set local_runbooks: false to skip it. See Adding runbooks.

Module layout

src/junior/
  runbook/                 ← the framework (domain-agnostic)
    base.py                 ← Runbook + Harness ABCs, LLMResult, Usage
    runner.py               ← run_runbook()
    registry.py             ← get_runbook (built-in + entry-point + path + repo-local), get_harness
  harnesses/                ← shared LLM drivers (each exposes HARNESS)
    claudecode.py  codex.py  pydantic.py  deepagents.py  pi.py
  runbooks/
    code_review/            ← the built-in runbook family
      base.py               ← CodeReviewRunbook (shared render/prompt/publish)
      local.py  github.py  gitlab.py  bitbucket.py   ← per-platform collect + _post_to_platform
      models.py             ← ReviewContext, ReviewOutput, ReviewResult, …
      render.py             ← build_user_message()
      instructions.py       ← build_review_prompt() + BASE_RULES
    weather/                ← example non-code-review runbook (live weather → outfit)
    script/                 ← ScriptRunbook: YAML manifest (prompt + optional schema/collect/publish)
  collect/  publish/        ← platform helpers imported by the runbooks

Settings, validation, exit codes

• Settings split into settings.context (the task/inputs), settings.llm (harness
  • role prompt + keys + limits), settings.output (output file, platform tokens, publish flag), plus top-level settings.runbook and settings.log_level.
• Validation is split: generic checks (context files, LLM API key) live in Settings.preflight; runbook-specific checks (publishing needs a token) live in Runbook.validate.
• Exit code comes from Runbook.is_blocking — for code review, a critical comment or a request_changes recommendation fails CI (exit 1).