Metadata and storage

Session SQLite, canonical messages, and Composer context policies

A persisted Session belongs to the Agent storage scope provided by City. Without City, the default is in-memory storage. Use the Session API instead of treating these files as an application database.

The SDK provides the Agent single-database v13 storage path. Canonical codecs, metadata/preview, and interaction projections do not open databases or depend on a partition layout. Historical format schemas and conversion entry points belong to the source repository's scripts/ offline tools; old directories are inspected only to detect required migration.

SessionHistoryMeta.v is metadata projection protocol version 2, independent of database schema 13. Composer reads checkpoints by Session, namespace, and checkpoint protocol version. A version mismatch returns no checkpoint so summaries can be rebuilt from canonical history; initialization does not drop or recreate tables.

~/.downcity/agents/<agent_id>/sessions.db
~/.downcity/agents/<agent_id>/session-files/<session_id>/attachments/

An Agent shares one sessions.db across active and archived Sessions:

  • sessions: authoritative identity, top-level type and kind, creation facts in relations, Workspace binding, title, preview, counts, and revision.
  • session_configuration: model binding, approval settings, execution profile, and base system snapshot.
  • messages and message_parts: canonical content scoped to its Session record.

Each Session has one permanent UUIDv4, exposed as id on Session objects, summaries and info. Storage and cross-object references use session_id. New Sessions, forks, and main-conversation replacements receive new IDs. Archiving and restoring resources for the same Workspace ID preserve its ID. Forked Messages identify their source with origin.session_id and source Message/Turn IDs. The Agent database selects its current main conversation through a persistent pointer.

Session.updated_at starts at the Session creation time and advances monotonically only when a User Message is received (using its created_at) or an Agent Message enters its terminal state after completion, failure, or cancellation (using its ending updated_at). Streaming content, title or configuration changes, history truncation, and archiving preserve this activity time. Message updated_at and revision continue to track content changes; the Session record revision tracks committed directory changes independently of activity time.

Custom Session stores implement create_message(candidate, options) with a serializable candidate that omits sequence. Storage assigns the final sequence and returns the committed Message. The optional assert_receivable check runs before dispatch; a dispatched transaction settles from its actual commit result.

A storage request that has not been sent waits up to 30 seconds for an exiting Worker to finish before using a replacement. Waiting requests remain subject to storage capacity limits; a wait timeout reports session_storage_queue_timeout. Once sent, the request has a separate 60-second response timeout.

If a storage Worker fails during a write, the runtime waits for the failed writer to terminate and checks the original operation receipt. A committed receipt returns its original result. Otherwise the error carries the original operation_id with an unknown or not-committed outcome; the runtime does not replay the write automatically.

The API reconstructs SessionMessage aggregates when reading. The table layout is not exposed to the UI or model providers. Canonical history is never deleted or rewritten by context compaction.

sessions.workspace_id records ownership determined at creation. It cannot be replaced, cleared, or added later. Metadata writes reject ownership mismatches, and configure rejects Workspace fields. Restoration accepts only resources with the same ID; history remains readable without those resources. Agent ownership, type consistency, and archive state remain checks; they do not create separate identities for the same UUID.

Host execution profiles may supply a creation_key. Storage maps that business key to a newly allocated UUID in one transaction. Reusing the key selects the same Session; an archived or deleted selection is rejected. Use a new business key for a new attempt. The key is not the Session ID.

Read-only browsing and archived records

A local Agent exposes agent.sessions.read_main() to read the current main conversation without creating a Session or restoring an executor. It returns null when the pointer is empty. Call agent.sessions.self() explicitly to ensure a main conversation exists; concurrent calls select the same persisted Session.

agent.sessions.read(session_id, type) reads active history. agent.sessions.read_archived(session_id) reads archived history, and remove_archived(session_id) permanently deletes that exact Session. Views contain session_id, archived, metadata, messages(), configuration(), and interactions(). A view rejects further reads if its expected archive state changes. The permanent ID never redirects to a new main conversation.

agent.sessions.read_metadata(addresses) accepts up to 100 { session_id, type } references. Results follow input order and contain session_id, archived, and metadata, or null for a missing record or type mismatch. It loads no message bodies and does not restore executors. An empty batch does not access storage.

AgentSessionBrowser provides independent read-only access. It does not create a missing database, load models, connect a Workspace Shell, or migrate data. For an existing Agent root with no database or legacy partitions, a first list returns an empty page and main() returns null. Exact reads require existing storage. Existing empty databases, incompatible formats, permission failures, and corruption remain explicit errors.

import { AgentSessionBrowser } from "@downcity/agent";

const browser = new AgentSessionBrowser({ agent_id, storage_root_path });
try {
  const current_main = await browser.main();
  const page = await browser.list({ type: "chat", limit: 30 }, true);
  const entry = page.items[0];
  if (entry) {
    const archived = await browser.get_archived({ session_id: entry.id });
    const history = await archived.messages({ limit: 50 });
  }
} finally {
  await browser.dispose();
}

browser.get_record(session_id) reads the same permanent identity in either archive state. get(session_id, type) checks active state; get_archived({ session_id, type? }) checks archived state. The Agent remains the owner and authorization boundary. A UUID alone grants no access.

Removal events carry the removed session_id. Match that ID when invalidating caches; a delayed event for an archived main conversation does not identify its replacement. Lists use live keyset cursors: pass next_cursor with unchanged filters, deduplicate by summary id, and restart from the first page after session_list_cursor_invalid.

Await dispose() to release a Browser's storage lease. An independent reader can outlive an executor. In-memory Browsers must acquire a lease from the same owner; a second :memory: database cannot read the original history.

Attachment ownership and deletion

Attachments belong to session-files/<session_id>/attachments. Archiving preserves them. Permanent deletion records attachment cleanup and retires the ID in the same transaction as structured deletion. Opening a Session never sweeps files.

Node.js hosts can call prepare_agent_session_storage({ agent_id, storage_root_path }) under exclusive root ownership to validate an existing target-format database and prepare access coordination. It does not create a business database or convert history.

cleanup_agent_session_attachments(input) is an explicit offline SDK maintenance operation under exclusive root ownership. It also requires the Agent's file lease exclusively; active readers return session_attachments_in_use. It visits only committed cleanup records, preserves failures for retry, and accepts only framework-owned files or a validated ownership manifest. Never remove coordination files to bypass readers.

Resume a subtree archive

sessions.archive({ id, type, operation_id, root_session_id }) freezes a postorder plan before archiving descendants. operation_id identifies this operation; root_session_id, when supplied, must equal id. The result includes operation_id, root_session_id, plan_fingerprint, and the original root archive time.

After partial completion or a lost acknowledgment, reuse the original Session and operation IDs:

await agent.sessions.archive({
  id: original_session_id,
  type: "work",
  operation_id: original_operation_id,
  root_session_id: original_session_id,
});

A retry reads the fixed plan and cannot archive a replacement main conversation. Archiving the current main clears its pointer without creating another Session. The next explicit self() allocates a new UUID.

Saved Tool results use res_2_ references containing the source permanent Session ID, Message ID and sequence, Part ID, and content digest. Lookup verifies the original result after archiving. Forks can read matching canonical results copied into their own history. Result references grant no cross-Agent permission.

Composer and derived context

SessionComposer returns base system instructions, dynamic system context, messages, and tools. The Session assembles the base snapshot with the current dynamic context before sending it to the model. The caller passes canonical history and a namespaced derived store into every call; a Composer never opens Session storage by itself.

The default DefaultSessionComposer stores cumulative summaries and boundaries in its Session and Composer namespace in composer_checkpoints. It groups human Messages, complete Agent Steps, and independent input inputs into indivisible summary units. Parts from one Step, including intervening host records and its tool-call/result group, stay together.

Each Agent Message persists stable_part_sequence, an immutable committed-prefix watermark advanced by complete Step or input commits. Even an open Message may contribute units within that prefix. An unfinished Tool, Action, or model-text unit blocks further advancement; the Composer stops at an open Message's watermark and retains the newest request unit in original form. Summary boundaries can fall inside a Message, but cannot split a complete Step.

compose() returns the latest <session-context-summary> system block in context_system_blocks and builds a view of canonical Parts after the boundary. Summary input uses disclosed tool results and attachment references without media binary content. No summary is added when no checkpoint exists.

snapshot() / syncshot() only persist or refresh base blocks. Recovery reads base instructions from session_configuration.system_snapshot, or generates them from current Agent configuration if no persisted snapshot exists. Summaries come from Composer checkpoints, and the history view comes from canonical Parts. Model requests and session.system() previews always use the base snapshot followed by the current dynamic context.

Summaries and any other projected context are rebuildable derived data, not Messages. When a provider reports a context limit, or real usage crosses the pressure threshold, Session asks the Composer to advance its derived state through advance_context(). There is no public session.compact(), Active file, Segment file, or second message log.

Recovery and transactions

When a host calls agent.resume_work(), trusted background turns that still need closure restore a Session. Recovery does not replay models or commands. Turns with a completion record and no recoverable Messages skip execution-object assembly. If the completion journal was written before a streaming canonical Message closed, recovery still closes that Message and preserves the original result. Retries of pending completion deliveries run independently.

Background recovery failures identify the Agent, type, permanent Session ID, Turn ID, and recovery phase. Storage failures also include the command name and known numeric SQLite error code. Busy, locked, corrupt, constraint, and I/O failures report session_storage_busy, session_storage_locked, session_storage_corrupt, session_storage_constraint_failed, and session_storage_io_failed, respectively. Unknown failures still report session_storage_transaction_failed. Diagnostics exclude SQL parameters and message content, and failed writes are not automatically replayed. A per-Session failure does not stop later Sessions from being scanned; directory query failures still propagate to the host.

Database connections wait up to 250ms for transient lock contention starting with the first schema check, including WAL recovery locks when opening read-only connections. Contention beyond that limit still reports an error.

The Message envelope and its Parts commit in one SQLite transaction; Mutations publish only after commit. On startup, SessionMessages closes the non-terminal Agent Messages and running Actions selected by Storage; Storage does not interpret domain terminal states. Attachments remain under attachments/, while Parts store attachment references.

Model streaming follows one boundary: events drive live rendering, memory assembles the stream, and SQLite stores only stable semantic checkpoints. Ordinary chunks do not write SQLite; complete model steps, Tool transitions, Interactions, Actions/Errors, and Message closure do.

Agent Session storage requires schema v13. Reading or creating a Session does not migrate existing data; an incompatible Agent root reports session_migration_required. Use the offline migration tool for explicit conversion, backup and verification.

Custom local hosts can import AGENT_SESSION_SCHEMA_VERSION and assert_agent_session_schema(database, agent_id) from @downcity/agent to share the SDK's storage version and Agent ownership checks. database is a Node.js SQLite connection owned by the host. Validation only reads data; it neither initializes nor migrates storage. Version errors include the Agent, database path, actual version and required version so you can identify which data needs conversion.

Custom Composers use derived.read_checkpoint(schema_version) and derived.insert_checkpoint(checkpoint). The derived capability is scoped to the current Session and Composer namespace; it accepts structured data and does not expose SQL callbacks.

Human inputs are User Messages. Runtime inputs persist as Agent input Parts with per-input request metadata, and Interactions are nested in their owning Tool Part. Input commits preserve the current writer; Message boundaries are not used to signal runtime input reception.