LLM context compression and compaction¶
Use a11.sdk.llm_context from a library, an ordinary action, or a host integration.
Context policy is optional. Concrete interact_with_* actions and the router
read x-a11-llm-context-config; an absent header disables their context layer.
See portable Interaction views, direct calls and Flow, external harnesses, and measurement for complete configurations.
Content and history¶
compress_content returns {text, report}. The original argument remains
unchanged. json removes insignificant JSON whitespace while preserving number
lexemes, duplicate keys, and string tokens. lines and json-keys emit
versioned JSON envelopes that expand_content can reverse byte for byte.
auto chooses the smallest candidate under the selected counter, including
encoding overhead. A candidate must save at least min_gain tokens.
count_context uses a host-supplied tokenizer or a byte-based estimate labeled
byte_estimate. It does not estimate billing, request framing, image costs,
provider tool-schema overhead, or hidden server history.
compact_history returns a checkpoint proposal, covered source IDs, a digest,
a retained boundary, and a report. System/developer messages and recent entries
are retained. The selected prefix ends at a complete tool-call/result group.
A registered callback or caller-supplied checkpoint provides summary text;
there is no default summarization model. A failed budget check raises an A11
status and produces no replacement history.
The provider proxy rejects history compaction; it has no API to install the
proposal in the conversation owner. Use a host installer or native coordinator
so later requests reuse the installed prefix.
a11.sdk.llm_context.content
¶
Compress complete content through the native deterministic transforms.
compress_content
¶
compress_content(text: str, config: LlmContextConfig | dict | None = None, *, counter: Callable[[str], int] | None = None) -> dict
Select a smaller model-facing view of complete text.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Original content, retained by the caller. |
required |
config
|
LlmContextConfig | dict | None
|
Immutable policy or policy dictionary. Omit to request the default deterministic transforms. An explicit disabled policy returns original content with counts. |
None
|
counter
|
Callable[[str], int] | None
|
Optional host tokenizer used for candidate selection. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
A dictionary with text and a versioned report describing the format, |
dict
|
guarantee, source digest, counts, and transform time. Shadow mode |
dict
|
returns original text and reports candidate counts separately. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Policy, content, tokenizer output, or limits fail validation. |
count_context
¶
Count text with a tokenizer or a labeled byte estimate.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
UTF-8 content whose model-facing size is measured. |
required |
counter
|
Callable[[str], int] | None
|
Host tokenizer returning a nonnegative integer. Omit to use the native byte estimate. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Token count, UTF-8 byte count, and counting source. |
Raises:
| Type | Description |
|---|---|
StatusException
|
The supplied counter returns an invalid count. |
expand_content
¶
Recover the exact original from a reversible context envelope.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
An a11.lines/v1 or a11.json-keys/v1 JSON envelope. |
required |
max_bytes
|
int
|
Limit for the encoded input and expanded output. |
8 * 1024 * 1024
|
Returns:
| Type | Description |
|---|---|
str
|
The original UTF-8 text, including its whitespace and numeric tokens. |
Raises:
| Type | Description |
|---|---|
StatusException
|
The envelope is malformed or exceeds the byte limit. |
a11.sdk.llm_context.history
¶
Propose checkpoints while the caller owns session history.
compact_history
async
¶
compact_history(history: list[dict], config: LlmContextConfig | dict, *, summarizer: Callable[[list[dict]], str | Awaitable[str]] | None = None, counter: Callable[[str], int] | None = None, checkpoint: str | None = None, summary_factory: Callable[[str], dict] | None = None) -> dict
Summarize a complete prefix, retaining instructions and recent turns.
A summarizer accepts prefix messages and returns checkpoint text, optionally asynchronously. A supplied checkpoint replaces that call. The returned source digest and covered IDs let a host validate a replacement proposal. Shadow mode returns the original history alongside the proposed history.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
history
|
list[dict]
|
Source message dictionaries with optional stable id values. |
required |
config
|
LlmContextConfig | dict
|
Policy with byte/token budgets and retained suffix size. |
required |
summarizer
|
Callable[[list[dict]], str | Awaitable[str]] | None
|
Host-selected prefix-to-text callback. |
None
|
counter
|
Callable[[str], int] | None
|
Optional tokenizer for budget checks. |
None
|
checkpoint
|
str | None
|
Caller-supplied summary text, replacing the callback. |
None
|
summary_factory
|
Callable[[str], dict] | None
|
Optional formatter for the checkpoint message in a provider's wire shape. It receives labeled checkpoint text. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
History/proposed_history, checkpoint, source digest, covered IDs, |
dict
|
retained boundary, and a versioned report. An under-budget or disabled |
dict
|
invocation returns unchanged history and no checkpoint. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Content/budgets are invalid, no summarizer is bound, or protected entries and the checkpoint exceed the budget. |
compact_interactions
async
¶
compact_interactions(interactions: list[Interaction], config: LlmContextConfig | dict, *, summarizer: Callable[[list[dict]], str | Awaitable[str]] | None = None, counter: Callable[[str], int] | None = None, checkpoint: str | None = None, source_revision: int | str = 0, reporter: Callable[[dict], None | Awaitable[None]] | None = None) -> dict
Build an Interaction checkpoint proposal through the shared engine.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
interactions
|
list[Interaction]
|
Ordered canonical sources, including typed fragments. |
required |
config
|
LlmContextConfig | dict
|
Policy and limits. This call selects summary compaction. |
required |
summarizer
|
Callable[[list[dict]], str | Awaitable[str]] | None
|
Host-selected synchronous or asynchronous prefix callback. |
None
|
counter
|
Callable[[str], int] | None
|
Optional tokenizer for budget checks. |
None
|
checkpoint
|
str | None
|
Existing checkpoint text, replacing a summarizer call. |
None
|
source_revision
|
int | str
|
Revision of the canonical source snapshot. |
0
|
reporter
|
Callable[[dict], None | Awaitable[None]] | None
|
Optional synchronous or asynchronous metadata sink. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
A serialized Interaction-view proposal. Accept it through its history |
dict
|
owner before calling use_interaction_view. Canonical sources remain |
dict
|
separate from generated checkpoints and projected model content. |
Interaction views and installation¶
project_interactions returns a proposal with existing typed Interaction
sources and a separate normalized projection. compact_interactions selects
summary compaction. accept_interaction_view rechecks the snapshot and requires
an explicit owner acknowledgement. use_interaction_view materializes only
accepted projections, excluding the covered prefix. Accepted projections replay
without loading source-producer SDKs or repeating summarization.
The plain JSON contract is a11.llm-context.interaction-view/v1. It uses existing
Interaction/Chunk serialization and introduces no new runtime value or wire tag.
entries[].source_id=null identifies generated checkpoint content; retained
entries name canonical sources. projection_digest covers entries and
instructions. view_id binds that projection to its source digest, source
revision, coverage, boundary, and policy. The installation records installed,
owner, view_id, source_digest, source_revision, and the committed
revision. Digests provide integrity; owner access remains host-authorized.
Projected Interaction metadata contains the producer a11.llm_context and a
bounded a11.llm-context.view proof. A backend validates the full prefix and
its content/instruction digest before request formatting. sources retains
original action inputs/outputs, typed chunks, usage, and backend metadata;
model-facing normalized parts contain the portable text/image/tool subset.
Providers declare diagnostic-only replay_metadata_keys beside their existing
register_interaction_normalizer call. Unknown metadata remains opaque. This
capability declaration adds no provider-specific compaction algorithm.
a11.sdk.llm_context.interactions
¶
Project, accept, and replay model-facing views of canonical interactions.
project_interactions
async
¶
project_interactions(interactions: list[Interaction], config: LlmContextConfig | dict, *, source_revision: int | str = 0, summarizer: Callable[[list[dict]], str | Awaitable[str]] | None = None, checkpoint: str | None = None, counter: Callable[[str], int] | None = None, reporter: Callable[[dict], None | Awaitable[None]] | None = None) -> dict
Build an inspectable, unaccepted view before choosing a provider.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
interactions
|
list[Interaction]
|
Ordered canonical Interaction records. Typed fragments remain in serialized sources independently of generated summaries. |
required |
config
|
LlmContextConfig | dict
|
Opt-in compression and summary policy. Native compaction belongs to a harness owner and is unavailable in a local view. |
required |
source_revision
|
int | str
|
Explicit revision of the host's canonical snapshot. |
0
|
summarizer
|
Callable[[list[dict]], str | Awaitable[str]] | None
|
Host-selected sync/async prefix-to-text callback. |
None
|
checkpoint
|
str | None
|
Existing checkpoint text, replacing a summarizer call. |
None
|
counter
|
Callable[[str], int] | None
|
Optional tokenizer; otherwise counts are labeled estimates. |
None
|
reporter
|
Callable[[dict], None | Awaitable[None]] | None
|
Optional sync/async sink for transform and compaction reports. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
A versioned JSON proposal with canonical sources, normalized projection, |
dict
|
instructions, covered IDs, boundary, policy, and integrity digests. |
dict
|
No history owner is modified. Shadow proposals cannot be accepted. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Identity, limits, ordering, or budget validation fails. |
encode_interactions
¶
Serialize existing Interaction values as ordinary typed A11 chunks.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
interactions
|
list[Interaction]
|
Canonical records, including original typed fragments. |
required |
Returns:
| Type | Description |
|---|---|
list[dict]
|
JSON-compatible Chunk documents using the existing Interaction wire tag. |
list[dict]
|
Serialization leaves source records and native chunk metadata unchanged. |
decode_interactions
¶
Read Interaction values from their existing typed Chunk representation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunks
|
list[dict]
|
JSON-compatible Chunk documents from encode_interactions. |
required |
Returns:
| Type | Description |
|---|---|
list[Interaction]
|
Independent source records with their native typed data intact. |
Raises:
| Type | Description |
|---|---|
StatusException
|
A chunk does not decode as an Interaction. |
interaction_view_sources
¶
Decode the canonical sources retained independently of a projection.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
view
|
dict
|
Proposal or accepted view with its complete source archive. |
required |
Returns:
| Type | Description |
|---|---|
list[Interaction]
|
Independent Interaction records with their original typed fragments. |
use_interaction_view
¶
Materialize an accepted portable view as ordinary A11 Interactions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
view
|
dict
|
Owner-accepted JSON view with validated provenance. |
required |
Returns:
| Type | Description |
|---|---|
list[Interaction]
|
A new Interaction chain with the covered prefix excluded. Existing |
list[Interaction]
|
NormalizedMessage content uses an SDK producer normalizer; participating |
list[Interaction]
|
backends retain responsibility for their wire formatting. Instructions |
list[Interaction]
|
attach once at the start. Original typed fragments remain in sources. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Installation is absent or opaque content cannot be replayed through portable normalization without losing data. |
validate_interaction_view
¶
Check a serialized proposal or accepted view without installing it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
view
|
dict
|
Versioned JSON Interaction view, including canonical sources. |
required |
Raises:
| Type | Description |
|---|---|
StatusException
|
Digests, source coverage, or the projection disagree. |
a11.sdk.llm_context.interaction_owner
¶
Install Interaction views through revisioned host ownership.
accept_interaction_view
async
¶
accept_interaction_view(proposal: dict, read_history: Callable[[], dict | Awaitable[dict]], install_view: Callable[..., dict | Awaitable[dict]]) -> dict
Accept a proposal through an explicit history-owner installation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
proposal
|
dict
|
Unaccepted JSON view returned by project_interactions. |
required |
read_history
|
Callable[[], dict | Awaitable[dict]]
|
Host callback returning interactions and their revision. |
required |
install_view
|
Callable[..., dict | Awaitable[dict]]
|
Atomic owner callback receiving the proposal plus expected_revision and source_digest. It installs a distinct view while retaining canonical history. Its acknowledgement contains installed, owner, view_id, source_digest, source_revision, revision. |
required |
Returns:
| Type | Description |
|---|---|
dict
|
An accepted JSON view after the owner acknowledges installation. |
dict
|
No implicit retry follows a failed or uncertain acknowledgement. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Sources/revisions changed, shadow mode is selected, or the owner cannot acknowledge this exact installation. |
persist_interaction_view
async
¶
persist_interaction_view(proposal: dict, store: RetainedContentStore, scope: dict, *, claim_token: str | None = None) -> dict
Install a declared source snapshot through an authorized retention owner.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
proposal
|
dict
|
Unaccepted Interaction view with its declared source revision. |
required |
store
|
RetainedContentStore
|
Existing RetainedContentStore owned by the receiving host. |
required |
scope
|
dict
|
Host-authorized owner/conversation/branch, expected_revision, and stable operation_id. The owner revision serializes updates. |
required |
claim_token
|
str | None
|
Optional durable operation token fencing summary workers. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
An accepted view after the owner's durable compare-and-swap commit. |
dict
|
Canonical source chunks and the projection share the saved manifest. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Scope/revision conflicts or the commit is uncertain. |
load_interaction_view
async
¶
load_interaction_view(store: RetainedContentStore, scope: dict) -> dict | None
Read an accepted view from a host-authorized retention scope.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
store
|
RetainedContentStore
|
Receiving host's RetainedContentStore. |
required |
scope
|
dict
|
Authorized owner/conversation/branch identity. |
required |
Returns:
| Type | Description |
|---|---|
dict | None
|
An accepted JSON view attested by the owner's stored commit, or None. |
dict | None
|
Retrieval uses the owner's ordinary manifest and revision contract. |
compact_host_interactions
async
¶
compact_host_interactions(read_history: Callable[[], dict | Awaitable[dict]], install_view: Callable[..., dict | Awaitable[dict]], config: LlmContextConfig, **options: Any) -> dict
Build and install a view while the host retains its inference client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
read_history
|
Callable[[], dict | Awaitable[dict]]
|
Host snapshot callback with interactions and revision. |
required |
install_view
|
Callable[..., dict | Awaitable[dict]]
|
Atomic owner installer using accept_interaction_view's acknowledgement contract; it retains the canonical sources. |
required |
config
|
LlmContextConfig
|
Explicit projection/compaction policy. |
required |
**options
|
Any
|
Summarizer, checkpoint, counter, or reporter options accepted by project_interactions. Source revision comes from the snapshot. |
{}
|
Returns:
| Type | Description |
|---|---|
dict
|
Accepted view, or an uninstalled proposal for disabled/shadow policy. |
Policy and host dependencies¶
LlmContextConfig is immutable and rejects unknown fields. The header is bounded
to 8192 bytes and cached by its raw bytes. Equal inherited headers reuse the
same decoded policy. LlmContextServices binds host tokenizers, summarizers,
retention, and recording independently of caller policy. Its ceilings apply at
each receiving action boundary.
| Setting | Default | Meaning |
|---|---|---|
enabled |
true in an explicit config |
Master switch; an absent action header resolves to disabled. |
boundary |
provider |
Select provider-message or backend-agnostic interaction projection. Accepted Interaction views select their own projection boundary. |
input_content, tool_results |
off |
Select editable request text and model-facing tool results independently. |
strategy |
auto |
auto, json, lines, or json-keys. |
max_bytes |
8 MiB | Bound content admitted to transforms. |
min_gain |
16 | Minimum estimated or counted token reduction. |
compaction |
off |
summary selects history checkpointing; native coordinates supported harness-owned sessions. |
summarizer |
empty | Host-bound callback name or registered summarizer action name. |
max_tokens, reserve_tokens |
24000, 2048 | History budget and space reserved for continuation. |
retain_recent |
4 | Minimum recent message count to retain, extended to preserve tool groups. |
retention |
false |
Persist original and projected request views through configured host storage. |
shell.mode |
off |
off, observe, or compress at completed-result boundaries. |
measurement.mode |
off |
off, observe, or shadow; shadow sends the original content. |
measurement.run_id, measurement.arm |
empty | Link observations to a host-selected experiment. |
An enabled config alone selects no request, tool, or history transforms.
Standalone compress_content(text) explicitly requests compression and uses
its default policy. Utility actions use their effective action header.
a11.sdk.llm_context.config
¶
Resolve immutable context policy from ordinary action headers.
LlmContextConfig
¶
Bases: BaseModel
Select model-facing transforms independently of generation options.
Input text, tool results, history summaries, retention, and shell results have separate opt-ins. Counts use max_tokens minus reserve_tokens as the history budget. Header absence resolves to disabled; an explicitly created configuration has enabled=True and each automatic transform initially off. Policy fields are immutable, bounded, and reject unknown names.
LlmContextScope
¶
Bases: BaseModel
Carry owner and revision identity across action/session boundaries.
LlmContextHeaders
¶
Bases: StrEnum
Action headers carrying caller policy and explicit owner scope.
LlmContextServices
dataclass
¶
LlmContextServices(max_bytes: int = 8 * 1024 * 1024, max_tokens: int = 2000000, summarizers: Mapping[str, Callable] = dict(), authorize_scope: Callable | None = None, resolve_scope: Callable | None = None, retention: Any = None, recorder: Any = None, token_counter: Callable[[str], int] | None = None, revision: str = '1')
Carry host dependencies bound independently on each receiving registry.
Attributes:
| Name | Type | Description |
|---|---|---|
max_bytes |
int
|
Host ceiling for admitted content. |
max_tokens |
int
|
Host ceiling for requested history budgets. |
summarizers |
Mapping[str, Callable]
|
Names mapped to sync/async prefix-to-text callbacks. |
authorize_scope |
Callable | None
|
Sync/async predicate receiving an Action or incoming HTTP request and its explicit scope. |
resolve_scope |
Callable | None
|
Sync/async HTTP request-to-scope resolver. The host authenticates the request before returning a scope; authorize_scope then authorizes that request and scope independently. |
retention |
Any
|
Optional RetainedContentStore owned by this host. |
recorder |
Any
|
Optional metadata RunRecorder or compatible record sink. |
token_counter |
Callable[[str], int] | None
|
Optional tokenizer returning a nonnegative count. |
revision |
str
|
Host-selected service configuration revision. Change it when changing tokenizer/summarizer behavior to invalidate saved views. |
decode_config
¶
decode_config(raw: bytes | str | None) -> LlmContextConfig
Decode a bounded policy header, reusing equal immutable values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
raw
|
bytes | str | None
|
UTF-8 JSON bytes/text or None for disabled policy. |
required |
Returns:
| Type | Description |
|---|---|
LlmContextConfig
|
An immutable validated policy. A bounded process cache and an optional |
LlmContextConfig
|
active-action fast path avoid repeated decoding of inherited headers. |
Raises:
| Type | Description |
|---|---|
StatusException
|
The header exceeds 8192 bytes or fails validation. |
bind_services
¶
bind_services(registry: ActionRegistry, services: LlmContextServices) -> None
Bind host dependencies to a registry and its declared SDK handlers.
Registered Python handlers retain their host binding when the native registry is copied for a session. A receiving process registers its own services; caller headers carry no Python dependencies.
configure_registry
¶
configure_registry(registry: ActionRegistry, config: LlmContextConfig, *, names: tuple[str, ...] | list[str] = ('interact_with_llm',), services: LlmContextServices | None = None) -> None
Publish policy defaults using existing header schemas and handlers.
Explicit per-call headers override these defaults. Each receiving registry binds its own host dependencies, including registries copied for sessions.
Programmatic host integration¶
project_messages projects provider-shaped message dictionaries in a host's
request callback. compact_host_history reads an explicit revisioned snapshot,
prepares a checkpoint, rechecks its source, and calls the host's atomic
installer. These APIs use Python callbacks and require no command or proxy.
Other languages can invoke the same utilities as ordinary A11 actions.
a11.sdk.llm_context.runtime
¶
Apply context policy at SDK action boundaries and native Flow calls.
project_messages
async
¶
project_messages(messages: list[dict], config: LlmContextConfig, *, services: LlmContextServices | None = None, reporter: Callable[[dict], None | Awaitable[None]] | None = None, summarizer: Callable[[list[dict]], str | Awaitable[str]] | None = None, stateful: bool = False) -> list[dict]
Project provider messages in a host's programmatic request hook.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
messages
|
list[dict]
|
Provider-shaped JSON message dictionaries. Instructions, signed/unknown blocks, and original messages remain intact. |
required |
config
|
LlmContextConfig
|
Explicit LLM context policy. |
required |
services
|
LlmContextServices | None
|
Optional host tokenizer, summarizers, and metadata recorder. |
None
|
reporter
|
Callable[[dict], None | Awaitable[None]] | None
|
Optional sync/async report callback, overriding the recorder. |
None
|
summarizer
|
Callable[[list[dict]], str | Awaitable[str]] | None
|
Prefix-to-text callback for summary compaction. |
None
|
stateful
|
bool
|
Whether the provider owns history behind an opaque ID. |
False
|
Returns:
| Type | Description |
|---|---|
list[dict]
|
A projected message list; unchanged disabled paths reuse the input list. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Limits fail or compaction requires unavailable history replacement support. This hook installs no host session history. |
project_view
async
¶
project_view(messages: list[dict], config: LlmContextConfig, *, services: LlmContextServices | None = None, scope: dict | None = None, reporter: Callable[[dict], None | Awaitable[None]] | None = None, summarizer: Callable[[list[dict]], str | Awaitable[str]] | None = None, stateful: bool = False) -> dict
Project and optionally persist a host-authorized request view.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
messages
|
list[dict]
|
Original provider message dictionaries. |
required |
config
|
LlmContextConfig
|
Explicit LLM context policy. |
required |
services
|
LlmContextServices | None
|
Receiving-host storage, counting, and recording services. |
None
|
scope
|
dict | None
|
Authorized owner/conversation/branch and operation/revision. |
None
|
reporter
|
Callable[[dict], None | Awaitable[None]] | None
|
Optional sync/async observer overriding the service recorder. |
None
|
summarizer
|
Callable[[list[dict]], str | Awaitable[str]] | None
|
Optional sync/async summary callback. |
None
|
stateful
|
bool
|
Whether the provider owns an opaque history continuation. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Projected messages, explicit scope, and optional view descriptor. |
dict
|
Durable operation claims recover an accepted view after a lost reply. |
a11.sdk.llm_context.harness
¶
Install checkpoint proposals through a host's own history API.
compact_host_history
async
¶
compact_host_history(read_history: Callable[[], dict | Awaitable[dict]], replace_history: Callable[..., dict | Awaitable[dict]] | None, config: LlmContextConfig, *, summarizer: Callable[[list[dict]], str | Awaitable[str]] | None = None, counter: Callable[[str], int] | None = None, reporter: Callable[[dict], None | Awaitable[None]] | None = None) -> dict
Compact through host callbacks while inference stays with that host.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
read_history
|
Callable[[], dict | Awaitable[dict]]
|
Read a snapshot containing history and a revision string or integer. The mapped history uses the compact_history contract. |
required |
replace_history
|
Callable[..., dict | Awaitable[dict]] | None
|
Atomically install a proposal using expected_revision and source_digest keyword arguments. Return installed=True and the new revision only after acceptance. None reports an unavailable replacement capability. |
required |
config
|
LlmContextConfig
|
Policy selecting budgets, retained suffix, and shadow mode. |
required |
summarizer
|
Callable[[list[dict]], str | Awaitable[str]] | None
|
Host-selected sync/async checkpoint callback. |
None
|
counter
|
Callable[[str], int] | None
|
Optional tokenizer for history budget checks. |
None
|
reporter
|
Callable[[dict], None | Awaitable[None]] | None
|
Optional sync/async metadata observer for the operation and installation result. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
A versioned result with the proposal and installation acknowledgement. |
dict
|
Disabled, under-budget, and shadow calls leave history unchanged. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Snapshots are invalid, history changes during summary generation, or replacement support/acknowledgement is unavailable. A failed or indeterminate installation is never retried implicitly. |
Ordinary actions¶
register(registry, services=..., names=...) installs selectable ordinary
handlers. Each has a unary JSON request input and result output. Policy and
scope use the same headers as model actions. The usual action transports,
describer, status chunks, and Flow dispatch apply.
| Action | Request data |
|---|---|
llm_context_project_interactions, llm_context_compact_interactions |
interactions: typed Chunk JSON documents, optional source_revision, checkpoint |
llm_context_accept_interaction_view |
view, authorized scope with expected revision and stable operation ID |
llm_context_load_interaction_view |
Authorized scope |
llm_context_use_interaction_view |
Accepted view; result contains typed projected interactions |
llm_context_compress, llm_context_count, llm_context_expand |
text |
llm_context_compact |
history, optional checkpoint |
llm_context_shell |
Completed command result, optional source and bypass |
llm_context_retain |
text, authorized scope |
llm_context_retrieve |
reference, authorized scope, optional cursor and max_bytes |
llm_context_save_view |
view, source_digest, authorized scope with operation/revision |
llm_context_load_view |
Authorized scope |
llm_context_retrieve authorizes against the receiving host's authorize_scope
callback. A caller-supplied owner or conversation string grants no access.
a11.sdk.llm_context.actions
¶
Register context utilities as ordinary local, remote, and Flow actions.
register
¶
register(registry: ActionRegistry, *, services: LlmContextServices | None = None, names: list[str] | tuple[str, ...] | None = None) -> None
Install selectable LLM context handlers with host-owned dependencies.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
registry
|
ActionRegistry
|
Receiving host registry, shared by direct and Flow calls. |
required |
services
|
LlmContextServices | None
|
Optional tokenizer, summarizers, retention owner, authorizer, and recorder. SDK handler bindings survive native registry copies. |
None
|
names
|
list[str] | tuple[str, ...] | None
|
Selected utility names; omit to install all context utilities. |
None
|
Raises:
| Type | Description |
|---|---|
StatusException
|
A selected utility name is unknown. |
schema
¶
Describe one utility with a unary JSON request and result.
Retention, shell results, and HTTP interception¶
RetainedContentStore uses native SQLite chunk stores for bytes and a separate
SDK SQLite index for scopes, expiry, and view revisions. References name an
owner-local object. A remote host accesses that owner's utility action; a node
ID alone has no routing or authorization meaning.
a11.sdk.llm_context.retention
¶
Retain exact content in native stores with scoped SDK reference metadata.
RetainedContentStore
¶
RetainedContentStore(root: str | Path, *, owner: str, max_bytes: int = 64 * 1024 * 1024, ttl_seconds: float = 3600, max_records: int = 10000)
Bind a local durable owner to native chunk data and a revision index.
Remote callers use utility actions on this owner; node IDs are owner-local. Applications authorize scopes before calling these host-side methods.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
root
|
str | Path
|
Host-selected directory for native stores and the SDK index. |
required |
owner
|
str
|
Stable identity used by references and scoped lookups. |
required |
max_bytes
|
int
|
Quota across published and pending content. |
64 * 1024 * 1024
|
ttl_seconds
|
float
|
Lifetime of newly retained content and views. |
3600
|
max_records
|
int
|
Ceiling for content reservations and operation records. |
10000
|
Concurrent owner startup uses an OS file lock for native SQLite connection initialization. View/claim transactions use SQLite's cross-process locking. Hosts initialize the store outside latency-sensitive asynchronous tasks.
retain
async
¶
Publish a reference after storing its exact UTF-8 content.
retrieve
async
¶
Read an exact UTF-8 page under a host-authorized scope.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
reference
|
dict
|
Owner/reference descriptor returned by retain. |
required |
scope
|
dict
|
Authorized owner, conversation, and branch. |
required |
cursor
|
int
|
UTF-8 byte offset, initially zero. |
0
|
max_bytes
|
int
|
Maximum page size, up to 1 MiB. |
65536
|
Returns:
| Type | Description |
|---|---|
dict
|
Text, current/next cursor, source digest, and expiry. next_cursor |
dict
|
is None at the end. A multibyte character stays within one page. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Scope/reference is unavailable, expired, corrupt, or the requested page cannot start/end on UTF-8 boundaries. |
save_view
async
¶
save_view(scope: dict, view: dict, *, source_digest: str, expected_revision: int, operation_id: str, claim_token: str | None = None) -> dict
Commit a view with revision checks and durable operation fencing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
scope
|
dict
|
Host-authorized owner, conversation, and branch. |
required |
view
|
dict
|
JSON view document with optional retained source descriptor. |
required |
source_digest
|
str
|
Identity of the canonical input used for this view. |
required |
expected_revision
|
int
|
Current scope revision, or zero for a new scope. |
required |
operation_id
|
str
|
Stable identity for retry reconciliation. |
required |
claim_token
|
str | None
|
Token supplied when claiming summary work. A replaced or abandoned token fences late workers out of publication. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Owner, scope, committed revision, and owner-local reference. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Revision, source, operation, or claim conflicts. |
collect_garbage
async
¶
Clear expired and abandoned payloads, retaining revision heads.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
pending_grace_seconds
|
float
|
Age after which unpublished reservations are abandoned. The host accounts for live writers before collection. |
300
|
Returns:
| Type | Description |
|---|---|
dict
|
Counts of reclaimed payloads and expired operation records. |
load_view
async
¶
Read the current view and revision under a host-authorized scope.
inspect_operation
async
¶
Read durable operation state for receiving-host reconciliation.
claim_operation
async
¶
claim_operation(scope: dict, *, operation_id: str, source_digest: str, expected_revision: int, claim_token: str | None = None) -> dict | None
Claim summary work or recover its accepted view without rerunning it.
Pass a unique claim_token through save_view to fence late workers after reconciliation. Pending and abandoned operations require host decisions.
reconcile_operation
async
¶
reconcile_operation(scope: dict, operation_id: str, *, expected_token: str, disposition: str, evidence: str) -> dict
Record an explicit owner decision and fence the previous worker.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
scope
|
dict
|
Host-authorized scope for the operation. |
required |
operation_id
|
str
|
Durable operation identity to inspect or resolve. |
required |
expected_token
|
str
|
Claim observed by the authorized recovery caller. |
required |
disposition
|
str
|
abandon, retry, or inspect. Retry is an explicit owner decision after reconciling external effects, never an automatic rerun after a lost response. |
required |
evidence
|
str
|
Host audit note describing that reconciliation. |
required |
Returns:
| Type | Description |
|---|---|
dict
|
Current operation metadata, including its replacement claim token. |
fork_view
async
¶
Copy an accepted view into a new authorized branch.
a11.sdk.llm_context.shell
¶
Project completed command results without executing or rewriting commands.
intercept_shell_result
¶
intercept_shell_result(result: dict, config: LlmContextConfig, *, counter: Callable[[str], int] | None = None, bypass: bool = False, source: dict | None = None) -> dict
Return separate original and model views of a completed command.
The result contract uses stdout/stderr text or bytes and exit_code. Optional fields include interactive, complete, signal, and truncated. Unknown fields survive. Command hints in source are metadata. This function never runs a command.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
result
|
dict
|
Completed result with exit_code, captured streams, and optional line arrays. Truncation/interactive/incomplete states pass through. |
required |
config
|
LlmContextConfig
|
Policy selecting shell off, observe, or compress. |
required |
counter
|
Callable[[str], int] | None
|
Optional tokenizer used by the shared content utility. |
None
|
bypass
|
bool
|
Per-command passthrough selected by the host. |
False
|
source
|
dict | None
|
Optional source metadata retained separately from output. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Original result, independent model-facing view, field reports, and |
dict
|
source metadata. Compressed line arrays carry a JSON representation |
dict
|
envelope; original typed arrays remain in original. |
shell_hook
¶
shell_hook(event: dict, config: LlmContextConfig, *, counter: Callable[[str], int] | None = None) -> dict
Adapt the versioned completed-result hook envelope.
External harness recipes map their event keys into this explicit contract. Pre-execution events pass through with an unsupported capability report.
a11.sdk.llm_context.proxy
¶
Serve provider request projections over A11's native streaming HTTP relay.
create_proxy
¶
create_proxy(upstream: str, *, profile: str = 'chat', config: LlmContextConfig | None = None, services: LlmContextServices | None = None, bind_address: str = '127.0.0.1', port: int = 0, max_inflight: int = 32, timeout_seconds: float = 120, options: Http2Options | None = None) -> Http2Server
Start an embeddable provider proxy with host-selected policy.
Upstream is a fixed origin with an optional path prefix. Credentials pass through from each request; inbound context headers never configure the host. Responses, including SSE and provider errors, remain native byte streams. Stop the returned native server to release its listening socket.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
upstream
|
str
|
Fixed HTTP/HTTPS origin with an optional literal path prefix. |
required |
profile
|
str
|
chat, responses, or anthropic request format. |
'chat'
|
config
|
LlmContextConfig | None
|
Host-selected policy; omitted policy disables projections. |
None
|
services
|
LlmContextServices | None
|
Host dependencies and optional metadata recorder. |
None
|
bind_address
|
str
|
Listener address, initially loopback. |
'127.0.0.1'
|
port
|
int
|
Listener port; zero selects an available port. |
0
|
max_inflight
|
int
|
Maximum admitted concurrent exchanges. |
32
|
timeout_seconds
|
float
|
Deadline per relayed exchange. |
120
|
options
|
Http2Options | None
|
Native HTTP transport/TLS and buffering options. |
None
|
Returns:
| Type | Description |
|---|---|
Http2Server
|
The running native Http2Server. Responses remain streamed by the native |
Http2Server
|
relay, including non-success statuses and trailers. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Configuration, limits, or listener setup fail. |
project_request
async
¶
project_request(body: bytes, *, path: str, profile: str, config: LlmContextConfig, services: LlmContextServices | None = None, reporter: Callable[[dict], Awaitable[None]] | None = None, scope: dict | None = None) -> bytes
Project supported JSON messages, preserving unknown request fields.
The profile names a wire format, independently of the harness using it. History compaction requires a separate host owner or native coordinator. This request-local proxy only applies deterministic content transforms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
body
|
bytes
|
Complete JSON request bytes within the policy byte limit. |
required |
path
|
str
|
Incoming provider path, including any query. |
required |
profile
|
str
|
chat, responses, or anthropic wire shape. |
required |
config
|
LlmContextConfig
|
Host-selected projection and compaction policy. |
required |
services
|
LlmContextServices | None
|
Host tokenizer and named summarizers. |
None
|
reporter
|
Callable[[dict], Awaitable[None]] | None
|
Optional async metadata observer for each selected span. |
None
|
scope
|
dict | None
|
Host-authenticated and authorized scope for retained views. |
None
|
Returns:
| Type | Description |
|---|---|
bytes
|
Original bytes when no field changes, or a JSON request containing |
bytes
|
projected messages. Unknown request fields retain their JSON values. |
Raises:
| Type | Description |
|---|---|
StatusException
|
Parsing/limits fail or required history/storage capabilities are unavailable. |
The generic a11.net.http.relay_http forwards to an explicit URL through the
native connection pool. It streams response bytes and trailers, preserves HTTP
error responses, strips hop-by-hop/private A11 headers, and performs no retry.
Its deadline and cancellation affect the exchange's stream.
Measurement and CLI¶
a11.sdk.llm_context.measurement
¶
Record context operations and compare matched disabled/enabled runs.
RunRecorder
¶
Append metadata JSONL to a host-selected bounded file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | Path
|
Host-owned output path, independent of caller headers. |
required |
max_bytes
|
int
|
Sink ceiling. Records exceeding the ceiling increment dropped and leave the file unchanged. |
64 * 1024 * 1024
|
Call record with operation/usage metadata. Transform helpers emit no source content; callers supplying extra records control those records' contents. Each instance serializes its own appends. Separate processes select separate run files or provide a shared recording service.
load_runs
¶
Load operation records from a JSON array or JSONL stream.
summarize_run
¶
Aggregate transform records and known provider contributions.
Replacement usage revisions share an id. Reconciliation/context-only rows contribute no consumption. Known agent/turn aggregates cover their request details; unverified cumulative coverage and missing counters stay unknown. Repeated operation IDs contribute transform counts once.
compare_runs
¶
Compare matched run records with explicit missing coverage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
baseline
|
list[dict]
|
Records from the disabled policy or baseline arm. |
required |
treatment
|
list[dict]
|
Records from the enabled policy or treatment arm. |
required |
prices
|
dict | None
|
Optional per-million tariffs for input_tokens, cached_input_tokens, cache_write_tokens, and output_tokens. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
A versioned comparison with per-arm totals, reductions, and optionally |
dict
|
usage-derived cost. Quality stays unmeasured. Missing counters or |
dict
|
tariffs leave cost unknown; projected counts are separate evidence. |
evaluate_pair
async
¶
evaluate_pair(workload: dict, invoke: Callable[[str, dict, Callable[[dict], None]], dict | Awaitable[dict]], *, setup: dict, grade: Callable[[dict], object] | None = None, prices: dict | None = None) -> dict
Run isolated baseline/treatment callbacks with matched metadata.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
workload
|
dict
|
JSON-serializable immutable workload specification. |
required |
invoke
|
Callable[[str, dict, Callable[[dict], None]], dict | Awaitable[dict]]
|
Host callback receiving arm, independent workload copy, and a metadata sink. It creates an isolated session/workspace, enables only the treatment policy, and returns outcome data. All model, summary, retrieval, and subagent usage goes to the same sink with distinct usage IDs and explicit scopes. |
required |
setup
|
dict
|
Shared provider/model/harness versions, tokenizer identity, seed, and generation options. Record actual upstream settings. |
required |
grade
|
Callable[[dict], object] | None
|
Optional outcome grader returning a JSON value. Errors propagate. |
None
|
prices
|
dict | None
|
Optional known per-million disjoint token tariffs. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Matched records and comparison, measured wall time, and explicit quality |
dict
|
results. Provider-reported costs remain host evidence in outcome rows. |
dict
|
Calls run sequentially; side-effect isolation belongs to the host. |
a11 llm-context --help lists standalone operations. compress, count, and
expand read bounded UTF-8 stdin; compact, retrieve, and shell-hook read
JSON. shell-filter reads/writes raw bytes and preserves passthrough bytes.
Other operations emit JSON. Diagnostics use stderr.
--config uses the same strict policy validator. --record appends reports.
compact --summarizer module:callable loads an explicitly selected host callable;
policy headers never select import paths. compare --baseline FILE --treatment
FILE reads recorded JSON/JSONL. measure compares captured input with the same
utility disabled/enabled without executing a model or a command.
Harness adapters¶
The shared transforms accept provider messages, completed results, and explicit
checkpoint proposals. adapters maps those contracts onto public harness APIs.
The optional Claude Agent SDK import occurs when its hook factory is called.
a11.sdk.llm_context.adapters
¶
Map public harness callbacks onto shared LLM context utilities.
compact_codex_thread
async
¶
compact_codex_thread(request: Callable[[str, dict], Awaitable[dict]], next_event: Callable[[], Awaitable[dict]], thread_id: str, *, timeout: float = 120, reporter: Callable[[dict], None | Awaitable[None]] | None = None) -> dict
Await native Codex compaction before sending the next turn.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
Callable[[str, dict], Awaitable[dict]]
|
Existing app-server RPC transport returning decoded results. |
required |
next_event
|
Callable[[], Awaitable[dict]]
|
Dedicated subscription established before the request. It yields notifications while the host retains its ordinary reader. |
required |
thread_id
|
str
|
Explicit thread identity to compact. |
required |
timeout
|
float
|
Whole-operation deadline in seconds. |
120
|
reporter
|
Callable[[dict], None | Awaitable[None]] | None
|
Optional sync/async metadata sink. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Completion evidence from the matching compaction item and turn. |
Raises:
| Type | Description |
|---|---|
StatusException
|
The matching turn fails or evidence is incomplete. |
TimeoutError
|
Completion is indeterminate; reconcile thread state before issuing another compaction request. |
compact_claude_session
async
¶
compact_claude_session(client: Any, *, instructions: str = '', timeout: float = 120, reporter: Callable[[dict], None | Awaitable[None]] | None = None, on_message: Callable[[object], None | Awaitable[None]] | None = None) -> dict
Compact a connected Claude Agent SDK client through its public command.
The host serializes this operation with its receive_response reader. The client remains connected and resumes through query after this returns.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
Any
|
Connected ClaudeSDKClient with query and receive_response. |
required |
instructions
|
str
|
Optional native summary focus instructions. State which facts must survive independently of transcript/file access. |
''
|
timeout
|
float
|
Whole-operation deadline in seconds; timeout is indeterminate. |
120
|
reporter
|
Callable[[dict], None | Awaitable[None]] | None
|
Optional sync/async operation metadata sink. |
None
|
on_message
|
Callable[[object], None | Awaitable[None]] | None
|
Optional observer for every compaction message, including provider usage and compact_boundary evidence. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Native installation evidence after compact_boundary and a successful |
dict
|
ResultMessage. Explicit insufficient-history status returns |
dict
|
installed=False. No generated summary is injected into the session. |
Raises:
| Type | Description |
|---|---|
StatusException
|
The command fails or emits no compaction boundary. |
compact_acp_session
async
¶
compact_acp_session(request: Callable[[str, dict], Awaitable[dict]], session_id: str, commands: list[dict]) -> dict
Invoke Junie's advertised native compaction command through ACP.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
Callable[[str, dict], Awaitable[dict]]
|
Existing ACP transport. It routes session/update notifications through the host and resolves session/prompt on completion. |
required |
session_id
|
str
|
Explicit negotiated session identity. |
required |
commands
|
list[dict]
|
Latest advertised command list for this session. |
required |
Returns:
| Type | Description |
|---|---|
dict
|
The acknowledged session/prompt response. Missing /compress support |
dict
|
raises FAILED_PRECONDITION before any request is sent. |
project_acp_prompt
async
¶
project_acp_prompt(params: dict, config: LlmContextConfig) -> dict
Project editable text in an ACP session/prompt request.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
params
|
dict
|
Session prompt parameters containing prompt content blocks. |
required |
config
|
LlmContextConfig
|
Explicit text compression policy. ACP resource/image blocks and session identity retain their original values. |
required |
Returns:
| Type | Description |
|---|---|
dict
|
Parameters with projected user text. Filesystem and terminal RPC |
dict
|
responses stay under the host's canonical resource contract. |
codex_tool_hook
¶
codex_tool_hook(event: dict, config: LlmContextConfig) -> dict
Return a Codex PostToolUse replacement for a proven successful result.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event
|
dict
|
Public PostToolUse input. tool_response must contain an integer exit_code and captured stdout/stderr. Unknown result shapes pass. |
required |
config
|
LlmContextConfig
|
Explicit shell policy; absence of compression returns {}. |
required |
Returns:
| Type | Description |
|---|---|
dict
|
A continue/stopReason hook decision when the model view changes, or {}. |
dict
|
The original tool result remains with the host. Code-mode promises use |
dict
|
Codex's normal completion path. |
claude_code_hooks
¶
claude_code_hooks(config: LlmContextConfig) -> dict
Build optional Claude Agent SDK hooks for successful Bash presentation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
LlmContextConfig
|
Explicit shell policy. Errors and interrupted results pass. |
required |
Returns:
| Type | Description |
|---|---|
dict
|
ClaudeAgentOptions hooks using the installed optional claude-agent-sdk. |
dict
|
Only stdout/stderr presentation changes; other output fields survive. |
Passive response observations¶
ResponseUsageObserver consumes bounded JSON or SSE response events through the
native relay observer. Usage revisions share a request identity and use the
same provider normalization as SDK clients. Incomplete frames, limits, errors,
and non-success HTTP statuses produce partial coverage. Transport completion
alone supplies no missing usage counters. The relay disables a failing observer
and continues forwarding the original response.
a11.sdk.llm_context.observation
¶
Observe bounded JSON/SSE usage while relaying provider bytes unchanged.
ResponseUsageObserver
¶
ResponseUsageObserver(profile: str, request_id: str, record: Callable[[dict], None], *, metadata: dict | None = None, max_event_bytes: int = 1024 * 1024, started_at: float | None = None)
Consume passive relay events and publish shared usage snapshots.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
profile
|
str
|
chat, responses, or anthropic wire format. |
required |
request_id
|
str
|
Stable request identity for replacement usage revisions. |
required |
record
|
Callable[[dict], None]
|
Synchronous metadata sink. It runs on the relay worker. |
required |
metadata
|
dict | None
|
Host-selected run/arm/model labels. |
None
|
max_event_bytes
|
int
|
Maximum buffered JSON document or SSE event. |
1024 * 1024
|
started_at
|
float | None
|
Optional host monotonic request start, including projection time in the first-byte measurement. |
None
|