Local Agent

tools and plugins

How to inject tools and explicit plugin instances into a local Agent

tools and plugins

tools

Local Agents can receive an explicit tool set:

const agent = new Agent({
  id: "repo-helper",
  workspace: new Workspace({ path: "/path/to/project" }),
  tools: {
    my_tool: myTool,
  },
});

These tools are available during session execution.

shell

Use @downcity/shell when the Agent should own shell execution tools:

import { Agent, Workspace } from "@downcity/agent";
import { Shell } from "@downcity/shell";
import { MacOsSeatbeltSandbox } from "@downcity/sandbox-macos";

const agent = new Agent({
  id: "repo-helper",
  workspace: new Workspace({
    path: "/path/to/project",
    shell: new Shell({ sandbox: new MacOsSeatbeltSandbox() }),
  }),
});

Workspace creates WorkspaceTools from five file/search tools by default, and Shell contributes two command tools. Agent registers all of them in agent.tools:

  • shell_exec: one-shot non-interactive commands
  • shell_session: long-running or stdin-driven interactive PTY sessions
  • grep: structured project-content search powered directly by ripgrep
  • find: project file discovery with POSIX glob patterns
  • read: paginated text reads and image or PDF attachments
  • write: file creation or explicit atomic full-file replacement
  • edit: atomic exact-text edits within one file

File and search tools can only access the Workspace root and remain available without Shell. Generic unrestricted belongs only to Shell; File/Search, Plugin, and custom tools do not share that switch. Outside-project access requires Shell unrestricted approval, an explicit user attachment, or a dedicated host tool. grep and find respect ignore files, return at most 200 results by default, and accept max_results up to 2,000.

When read opens a PNG, JPEG, GIF, WebP, BMP, or PDF, output contains only the normal file result and a User File Part pointing to the local file is injected into the next model step. The Tool Result contains no base64. Other binary files return metadata only.

read returns the file SHA-256. When another process may change the file concurrently, pass it as edit.expected_sha256 to prevent a stale edit from replacing newer content:

const current_result = await agent.tools.read.execute({
  file_path: "src/index.ts",
});
const current = current_result.output;

await agent.tools.edit.execute({
  file_path: "src/index.ts",
  expected_sha256: current.sha256,
  edits: [
    {
      old_text: "const enabled = false;",
      new_text: "const enabled = true;",
    },
  ],
});

Unrestricted sandbox requests belong to the Session that started the tool call:

const interactions = await session.interactions();
await session.respond({
  interaction_id: interactions[0].request.interaction_id,
  response: { kind: "approval", decision: "approved" },
});

A live UI reads the complete request from a pending Interaction Part while the related Tool is waiting-user, then calls session.respond(...) after the user decides.

Explicitly register the standalone tool when the model needs to ask questions:

import { Agent, Workspace } from "@downcity/agent";
import { AskQuestionsTool } from "@downcity/agent/tools";

const agent = new Agent({
  id: "repo-helper",
  workspace: new Workspace({ path: "/path/to/project" }),
  tools: {
    ask_question: AskQuestionsTool,
  },
});

The model invokes ask_question as a standard Tool Call, not as a provider-specific message type. It supports text, single_select, and multi_select; every question must explicitly provide its type. Once the user responds, the answers become the Tool Result and the model continues in the next Step of the same Turn. See Asynchronous User Interactions for the complete host flow.

plugins

Local Agents can also receive explicit plugin instances:

import { Agent, Workspace } from "@downcity/agent";
import { SkillPlugin } from "@downcity/plugins/skill";

const agent = new Agent({
  id: "repo-helper",
  workspace: new Workspace({ path: "/path/to/project" }),
  plugins: [new SkillPlugin()],
});

Add WebPlugin only when a search, document, or browser provider is configured. See the Web Plugin guide.

Registered plugin actions can be called through agent.plugins.run_action(...), and plugin system(context, execution_context) text is injected into session prompts.

PluginContext does not expose a generic path bundle. A plugin uses context.workspace_path as the project boundary and owns its domain storage paths inside the plugin. Operations on Agent Sessions go through context.sessions, not the Agent's Session directories.

context.sessions directly references the Agent's AgentSessions; it is not an adapter. Plugins use context.sessions.runtime(session_id) for the synchronous runtime port. SDK callers still restore an existing public Session with await agent.sessions.get(session_id).

Each Agent has its own plugin registry and its own plugin_read / plugin_call tool instances. If one process creates and caches multiple Agents, a session's plugin_call only reaches the registry for the Agent that owns that session.

plugin action metadata

When actions exist, PluginRegistry provides two tools and Agent registers them in its final tool collection:

  • plugin_read: reads registered plugin actions, input schemas, and examples
  • plugin_call: executes a plugin action

When the model is unsure how to call an action, it should call plugin_read({ plugin, action }) first, then call plugin_call(...). plugin_call validates payloads against the action input_schema before execution.

WorkspaceTools, Plugin Tools, and AgentOptions.tools cannot reuse a name. Agent throws on a conflict instead of silently overriding an existing tool.

Custom plugins can declare action metadata with create_plugin / create_action:

import { Agent, Workspace, create_action, create_plugin } from "@downcity/agent";
import { z } from "zod";

const demo_plugin = create_plugin({
  name: "demo",
  title: "Demo",
  description: "Demo actions",
  actions: {
    echo: create_action({
      description: "Echo text",
      input_schema: {
        zod: z.object({
          text: z.string(),
        }),
        json_schema: {
          type: "object",
          required: ["text"],
          properties: {
            text: { type: "string" },
          },
        },
      },
      examples: [
        {
          title: "Echo text",
          payload: { text: "hello" },
        },
      ],
      execute: async ({ input, execution_context }) => ({
        success: true,
        data: {
          text: input.text,
          session_id: execution_context?.session_id,
        },
        message: "echoed",
      }),
    }),
  },
});

const agent = new Agent({
  id: "repo-helper",
  workspace: new Workspace({ path: "/path/to/project" }),
  plugins: [demo_plugin],
});

The zod schema handles runtime validation. json_schema and examples are returned by plugin_read for the model or UI. execution_context is present when the action runs through a session tool and can be absent for direct CLI, HTTP, or scheduler calls. Use execution_context.workspace_env for the effective env committed at the current session step; context.workspace_env is the current Workspace configuration snapshot. Existing class extends BasePlugin plugins still work, and can gradually move individual actions to create_action(...).

Tools and Plugin Actions share one Session result path: put the ordinary result in output, and return content that belongs in the conversation as messages: [{ role, parts }]. User Parts become available in the next Step; Assistant Parts are written to the current Assistant Message. Session does not download or rewrite files. An Action must save a file first and return a File Part pointing to that local path.

ChatPlugin

Use ChatPlugin when you want the SDK runtime to own long-lived chat channels:

import { Agent, Workspace } from "@downcity/agent";
import { ChatPlugin, TelegramChannel } from "@downcity/plugins/chat";

const agent = new Agent({
  id: "repo-helper",
  workspace: new Workspace({ path: "/path/to/project" }),
  plugins: [
    new ChatPlugin({
      channels: [
        new TelegramChannel({
          env: {
            TELEGRAM_BOT_TOKEN: process.env.TELEGRAM_BOT_TOKEN,
          },
        }),
      ],
    }),
  ],
});

Each channel object owns its own env and credential parsing, so ChatPlugin only manages lifecycle, queue, and actions.

ImagePlugin

Use ImagePlugin when you want an Agent to generate images during a conversation:

import { Agent, Workspace } from "@downcity/agent";
import { ImagePlugin } from "@downcity/plugins/image";

const agent = new Agent({
  id: "creative-agent",
  workspace: new Workspace({ path: "/path/to/project" }),
  model,
  plugins: [
    new ImagePlugin({
      default_model: "image-model-id",
      list_models: async () => {
        const catalog = await city.ai.catalog();
        return catalog.forModality("image");
      },
      image_create: (input) => city.ai.image_create(input),
      image_result: (input) => city.ai.image_result(input),
    }),
  ],
});

When the default image model depends on application state, pass a function:

new ImagePlugin({
  default_model: async ({ input }) => {
    return input.aspect_ratio === "16:9" ? "image-wide-model-id" : "image-model-id";
  },
  image_create: (input) => city.ai.image_create(input),
  image_result: (input) => city.ai.image_result(input),
});

After registration, the Agent automatically gets the built-in plugin_read / plugin_call tools. The model can inspect action metadata first:

await plugin_read({
  plugin: "image",
  action: "image_create",
});

Image creation consumes provider quota. The Agent should ask the user to explicitly confirm the exact image creation or edit request before calling image_create.

After confirmation, it can use the job-style actions directly:

const job = await plugin_call({
  plugin: "image",
  action: "image_create",
  payload: {
    prompt: "A cinematic illustration of a rainy city corner at night",
    aspect_ratio: "16:9",
  },
});

await plugin_call({
  plugin: "image",
  action: "image_result",
  payload: {
    job_id: job.data.job_id,
  },
});

For reference images or image edits, use content:

const job = await plugin_call({
  plugin: "image",
  action: "image_create",
  payload: {
    content: [
      { type: "text", text: "Change this image to a white studio background" },
      { type: "image", url: "./input.png" },
    ],
  },
});

prompt and content are the two public input formats exposed to the Agent. Use prompt for text-only image generation. Use content whenever the request includes reference images, image edits, or multiple context parts. If both are present, content wins and prompt is not forwarded downstream.

content[].url can be an online URL, a local absolute path, or a path relative to the Agent project root. Local images are read by ImagePlugin and converted into the data URL format required by the City image job, so the model does not need to pass base64. The Agent should not pass messages or data_url.

default_model is the plugin-level default image model. It can be a string, or a sync/async function. When the Agent calls image_create without a model, ImagePlugin injects the string value or function return value before calling the downstream image_create(input). If the payload explicitly includes model, that payload value wins. Without default_model, call models first when the Agent needs to inspect available image model IDs.

image_result reads the current job state once by default. If it returns queued or running, keep the job_id and call image_result again later. For short jobs, pass until_done: true to wait for a terminal state in one tool call, with optional max_wait_ms / poll_interval_ms. If it returns succeeded, ImagePlugin downloads remote File Parts into .downcity/image/results/<job_id>/, returns the complete job result in data, and copies the localized File Parts into the current Assistant Message.

ImagePlugin is provider-agnostic. It does not know about City, OpenAI, or DeepSeek directly. Pass image_create and image_result job functions, and the plugin exposes the two-step job actions to the Agent; by default the product, Agent, or caller decides when to poll again based on poll_after_ms, while until_done is only a simple plugin-level wait helper.

“Resolved provider input” means this: the Agent only passes simple prompt or content; ImagePlugin reads local files into data URLs and converts content into ImagePluginResolvedInput.messages before calling image_create(input). This is the internal boundary from ImagePlugin to City / provider adapters, not a third Agent-facing payload format.

The image_result callback may return a remote URL or a local path. For a remote URL, ImagePlugin makes the local Workspace path the File Part's primary url and preserves the original online address in providerMetadata.downcity.source_url. If local storage fails, the remote URL remains usable and providerMetadata.downcity.localization_error plus the action message report the failure. Session persists the resulting File Part, while direct callers can read the same localized result from data.

plugin HTTP

If Downcity exposes the Agent HTTP gateway, registered plugins with plugin.http.server.register(...) are also mounted onto the same HTTP app automatically.

So:

  • plugin existence comes from the plugins array you pass into Agent
  • plugin HTTP exposure happens when Downcity publishes the Agent HTTP gateway

See also

Agent Constructor Options

Detailed explanation of Workspace env, WorkspaceTools, and local Agent id, instruction, model, tools, and plugins

Shell & Sandbox

Configure shell execution, sandboxing, and approval flows for the local Agent

Table of Contents

tools and pluginstoolsshellpluginsplugin action metadataChatPluginImagePluginplugin HTTPSee also