Skip to content

Data and serialization

The wire records A11 moves — chunks, fragments, node references, and wire messages — and the registry that turns Python objects into chunks and back.

Representation and value type

A chunk's metadata defines how to read its bytes. The media type names the representation, such as text/plain, application/json, image/png, or application/x-msgpack. A type media-type parameter names the application value when the representation does not describe it fully.

The seven JSON-native shapes (object, array, string, integer, number, boolean, and null) need no type parameter. Bare application/json and application/x-msgpack therefore decode to ordinary schemaless values. Model fields and action port schemas provide the expected type when one is declared.

Payload content does not select an application class. A requested object type is a best-effort decode target, and incompatible data returns a deserialization error.

Application values and encoded payloads

Use a11.to_chunk(value) and the serialization registry for Python values. Serializable A11 and SDK types carry stable wire tags so another language can select its corresponding type. JSON-native values remain plain data without a tag.

Use Chunk(data=..., metadata=...) for bytes already encoded in their final format. For example, PNG bytes belong in an image/png chunk and an HTTP body retains its received media type. Nodes write these through put_chunk() and read them through next_chunk(); no application-value codec is involved.

This boundary keeps serialization and transport separate: codecs map values to representations, while chunks carry any valid representation through stores, nodes, and wire streams.

Chunk

a11.data.types.Chunk

Chunk(metadata: Any | None = None, ref: str = '', data: str | bytes | bytearray | memoryview = b'')

A unit of node data with optional metadata and ref.

Create a chunk from optional metadata, a ref, and payload data.

approx_bytes property

approx_bytes: int

Approximate in-memory size of the chunk in bytes.

data property writable

data: bytes

Raw payload bytes of the chunk.

metadata property writable

metadata: ChunkMetadata | None

Optional metadata describing the chunk.

ref property writable

ref: str

Reference identifying the chunk's stored payload.

from_msgpack staticmethod

from_msgpack(data: Any) -> Chunk

Deserialize a value from MessagePack bytes.

model_construct classmethod

model_construct(**values: Any)

Construct a native value from trusted field values.

Native records retain C++ invariants. This validates input rather than creating an invalid object.

debug_string

debug_string() -> str

Return a human-readable debug string.

get_mimetype

get_mimetype() -> str

Return the chunk's MIME type, or empty if it has no metadata.

is_empty

is_empty() -> bool

Return whether the chunk has no payload data.

is_null

is_null() -> bool

Return whether the chunk is null (no metadata and no data).

materialize

materialize() -> None

Encode an attached local object into this chunk's payload bytes.

to_msgpack

to_msgpack() -> bytes

Serialize the value to MessagePack bytes.

validate

validate() -> None

Raise if the value fails structural validation.

a11.data.types.ChunkMetadata

ChunkMetadata(mimetype: str, timestamp: Any | None = None, attributes: Any = {})

Metadata describing a chunk of node data.

Create chunk metadata from a MIME type, timestamp, and attributes.

approx_bytes property

approx_bytes: int

Approximate in-memory size of the metadata in bytes.

attributes property writable

attributes: _ByteMapView

Byte-string attribute map attached to the chunk.

mimetype property writable

mimetype: str

MIME type describing the chunk payload.

timestamp property writable

timestamp: Time | None

Optional timestamp associated with the chunk.

from_msgpack staticmethod

from_msgpack(data: Any) -> ChunkMetadata

Deserialize a value from MessagePack bytes.

model_construct classmethod

model_construct(**values: Any)

Construct a native value from trusted field values.

Native records retain C++ invariants. This validates input rather than creating an invalid object.

debug_string

debug_string() -> str

Return a human-readable debug string.

get_attribute

get_attribute(key: str) -> bytes

Return the attribute bytes for a key, raising if it is absent.

set_attribute

set_attribute(key: str, bytes: Any) -> None

Set the attribute bytes for a key.

to_msgpack

to_msgpack() -> bytes

Serialize the value to MessagePack bytes.

validate

validate() -> None

Raise if the value fails structural validation.

Fragments and references

A NodeFragment associates one chunk with a node ID, sequence position, and continuation state. Sequence numbers restore the node's logical order even when transport messages arrive out of order.

A NodeRef identifies a range held by another node. Support depends on the backing store; the store reference lists the implementations that retain indexed references.

a11.data.types.NodeFragment

NodeFragment(data: Any, id: str = '', seq: SupportsInt | None = None, continued: bool = False)

A fragment of a logical node carrying a Chunk or NodeRef.

Create a node fragment from Chunk/NodeRef data and framing fields.

approx_bytes property

approx_bytes: int

Approximate in-memory size of the fragment in bytes.

continued property writable

continued: bool

Whether more fragments follow for this node.

data property writable

data: Chunk | NodeRef

Payload of the fragment as either a Chunk or a NodeRef.

id property writable

id: str

Identifier of the logical node this fragment belongs to.

seq property writable

seq: int | None

Optional sequence number of the fragment.

from_msgpack staticmethod

from_msgpack(data: Any) -> NodeFragment

Deserialize a value from MessagePack bytes.

model_construct classmethod

model_construct(**values: Any)

Construct a native value from trusted field values.

Native records retain C++ invariants. This validates input rather than creating an invalid object.

debug_string

debug_string() -> str

Return a human-readable debug string.

get_chunk

get_chunk() -> Chunk

Return the fragment's Chunk, raising if it holds a NodeRef.

get_node_ref

get_node_ref() -> NodeRef

Return the fragment's NodeRef, raising if it holds a Chunk.

to_msgpack

to_msgpack() -> bytes

Serialize the value to MessagePack bytes.

validate

validate() -> None

Raise if the value fails structural validation.

a11.data.types.NodeRef

NodeRef(id: str, offset: SupportsInt | None = 0, length: Any | None = None, storage: StorageIdentity | None = None)

Reference to a fragment range of another logical node.

Create a storage-qualified reference to a fragment range.

approx_bytes property

approx_bytes: int

Approximate in-memory size of the ref in bytes.

id property writable

id: str

Identifier of the referenced node.

length property writable

length: int | None

Optional number of referenced fragments.

offset property writable

offset: int

First referenced fragment sequence.

storage property writable

storage: StorageIdentity | None

Optional host-resolved storage namespace.

from_msgpack staticmethod

from_msgpack(data: Any) -> NodeRef

Deserialize a value from MessagePack bytes.

model_construct classmethod

model_construct(**values: Any)

Construct a native value from trusted field values.

Native records retain C++ invariants. This validates input rather than creating an invalid object.

debug_string

debug_string() -> str

Return a human-readable debug string.

to_msgpack

to_msgpack() -> bytes

Serialize the value to MessagePack bytes.

validate

validate() -> None

Raise if the value fails structural validation.

with_credentials

with_credentials(credentials: StorageCredentials, service: str | None = None) -> NodeRef

Return a qualified copy with selected access fields.

Messages

A WireMessage batches action control messages and node fragments. Control establishes or updates an action lifecycle; data continues on the mapped nodes independently. One transport message may contain work for several actions and nodes, and one action's stream may span many wire messages.

a11.data.types.Port

Port(name: str = '', id: str = '')

A named input or output port of an action.

Create a port from a name and node id.

approx_bytes property

approx_bytes: int

Approximate in-memory size of the port in bytes.

id property writable

id: str

Identifier of the node bound to the port.

name property writable

name: str

Name of the port.

from_msgpack staticmethod

from_msgpack(data: Any) -> Port

Deserialize a value from MessagePack bytes.

model_construct classmethod

model_construct(**values: Any)

Construct a native value from trusted field values.

Native records retain C++ invariants. This validates input rather than creating an invalid object.

debug_string

debug_string() -> str

Return a human-readable debug string.

to_msgpack

to_msgpack() -> bytes

Serialize the value to MessagePack bytes.

validate

validate() -> None

Raise if the value fails structural validation.

a11.data.types.ActionMessage

ActionMessage(id: str = '', name: str = '', inputs: Any = [], outputs: Any = [], headers: Mapping[str, bytes] | None = {})

A message invoking a named action with input and output ports.

Create an action message from id, name, ports, and headers.

approx_bytes property

approx_bytes: int

Approximate in-memory size of the message in bytes.

headers property writable

headers: _ByteMapView

Byte-string header map attached to the action.

id property writable

id: str

Identifier of the action invocation.

inputs property writable

inputs: _PortVectorView

Input ports of the action.

name property writable

name: str

Name of the action being invoked.

outputs property writable

outputs: _PortVectorView

Output ports of the action.

from_msgpack staticmethod

from_msgpack(data: Any) -> ActionMessage

Deserialize a value from MessagePack bytes.

model_construct classmethod

model_construct(**values: Any)

Construct a native value from trusted field values.

Native records retain C++ invariants. This validates input rather than creating an invalid object.

debug_string

debug_string() -> str

Return a human-readable debug string.

to_msgpack

to_msgpack() -> bytes

Serialize the value to MessagePack bytes.

validate

validate() -> None

Raise if the value fails structural validation.

a11.data.types.WireMessage

WireMessage(node_fragments: Any = [], actions: Any = [], headers: Mapping[str, bytes] | None = {})

A wire-format message bundling node fragments and actions.

Create a wire message from node fragments, actions, and headers.

actions property writable

actions: _ActionMessageVectorView

Action messages carried by the message.

approx_bytes property

approx_bytes: int

Approximate in-memory size of the message in bytes.

headers property writable

headers: _ByteMapView

Byte-string header map attached to the message.

node_fragments property writable

node_fragments: _NodeFragmentVectorView

Node fragments carried by the message.

from_json staticmethod

from_json(value: str) -> WireMessage

Deserialize a message from its JSON wire encoding.

from_msgpack staticmethod

from_msgpack(data: Any) -> WireMessage

Deserialize a value from MessagePack bytes.

model_construct classmethod

model_construct(**values: Any)

Construct a native value from trusted field values.

Native records retain C++ invariants. This validates input rather than creating an invalid object.

debug_string

debug_string() -> str

Return a human-readable debug string.

to_json

to_json() -> str

Serialize the message to its JSON wire encoding.

to_msgpack

to_msgpack() -> bytes

Serialize the value to MessagePack bytes.

validate

validate() -> None

Raise if the value fails structural validation.

Serialization

a11.data.serialization.SerializationRegistry

SerializationRegistry(*, register_defaults: bool = False)

A registry of serializers and deserializers indexed by type and MIME.

New registries are empty. Pass register_defaults=True or call register_defaults to install the built-in JSON and MessagePack codecs. The process-wide registry returned by get_global_serialization_registry already contains them.

set_type_tag

set_type_tag(obj_type: type, tag: str) -> None

Pin the wire tag used to identify obj_type in serialized data.

Overrides the default fully-qualified name, giving a type a short, stable identifier or disambiguating two like-named types.

register_serializer

register_serializer(obj_type: type, mimetype: str, serializer: SerializerFn, *, local_passthrough: bool = False) -> None

Register serializer(obj) for a type and exact media type.

local_passthrough permits snapshot-based local delivery when a matching deserializer is registered. The serializer must return bytes or text and round-trip the same value type.

register_deserializer

register_deserializer(obj_type: type, mimetype: str, deserializer: DeserializerFn, *, receives_chunk: bool | None = None) -> None

Register a data deserializer for a type and exact media type.

A deserializer may accept either data or data, obj_type. A callback whose first argument is named chunk (or is annotated as a Chunk) receives the complete chunk instead of chunk.data. receives_chunk can be used to select that behavior explicitly.

register

register(obj_type: type, mimetype: str, serializer: SerializerFn, deserializer: DeserializerFn, *, receives_chunk: bool | None = None, local_passthrough: bool = False) -> None

Register a codec pair.

local_passthrough opts in to passing snapshots through local nodes without codec calls. The serializer must return text or bytes, and the deserializer must reconstruct the same value type from those bytes.

register_defaults

register_defaults() -> None

Install the standard JSON and MessagePack registrations.

to_local_chunk

to_local_chunk(obj: Any, mimetype: str = '') -> Chunk

Attach a replay-safe snapshot when a codec permits local passage.

to_chunk_async async

to_chunk_async(obj: Any, mimetype: str = '') -> Chunk

Serialize with caller codecs on a worker thread.

Built-in codecs run inline. Caller codecs may block the event loop, regardless of payload size, so they run through asyncio.to_thread.

from_chunk_async async

from_chunk_async(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', obj_type: type | None = None) -> Any

from_chunk, off the event loop only when it might block it.

The mirror of to_chunk_async; see it for why the decision is made on the codec and not the payload.

to_chunk

to_chunk(obj: Any, mimetype: str = '') -> Chunk

Serialize obj into a chunk.

If mimetype is empty, the closest registered Python type wins and registration order chooses its preferred representation. An explicit MIME value selects a representation and can be exact or contain * wildcards; the object's own type decides the tag, so a type parameter in it is ignored.

from_chunk

from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str], obj_type: type[_T]) -> _T
from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', *, obj_type: type[_T]) -> _T
from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', obj_type: None = None) -> Any
from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', obj_type: type | None = None) -> Any

Deserialize chunk.

Selectors choose the representation: they are matched in order against the chunk's media type and may contain wildcards. If the chunk has no MIME metadata, a supplied exact selector acts as the representation.

What comes back is chosen separately. An explicit obj_type always wins and the registry makes a best effort to produce it, reporting the deserializer's own error when the data will not fit. Otherwise the chunk's type parameter names the type, and when it names nothing -- because it is absent, or because the module that would define it was never imported -- the payload decodes to whatever its format describes: a dict, a list, a scalar.

resolve_type

resolve_type(name: str) -> type | None

The Python type a wire tag names, or None if none is known.

The reverse of [_type_tag][a11.data.serialization. SerializationRegistry._type_tag]: given a11.sdk.AudioBuffer, the class it identifies. What a tag resolves to depends on what has been registered and imported, so a caller that gets None has learned that this process does not know the type -- not that it does not exist.