Data and serialization¶
The wire records A11 moves — chunks, fragments, node references, and wire messages — and the registry that turns Python objects into chunks and back.
Representation and value type¶
A chunk's metadata defines how to read its bytes. The media type names the
representation, such as text/plain, application/json, image/png, or
application/x-msgpack. A type media-type parameter names the application
value when the representation does not describe it fully.
The seven JSON-native shapes (object, array, string, integer, number,
boolean, and null) need no type parameter. Bare application/json and
application/x-msgpack therefore decode to ordinary schemaless values. Model
fields and action port schemas provide the expected type when one is declared.
Payload content does not select an application class. A requested object type is a best-effort decode target, and incompatible data returns a deserialization error.
Application values and encoded payloads¶
Use a11.to_chunk(value) and the serialization registry for Python values.
Serializable A11 and SDK types carry stable wire tags so another language can
select its corresponding type. JSON-native values remain plain data without a
tag.
Use Chunk(data=..., metadata=...) for bytes already encoded in their final
format. For example, PNG bytes belong in an image/png chunk and an HTTP body
retains its received media type. Nodes write these through put_chunk() and
read them through next_chunk(); no application-value codec is involved.
This boundary keeps serialization and transport separate: codecs map values to representations, while chunks carry any valid representation through stores, nodes, and wire streams.
Chunk¶
A unit of node data with optional metadata and ref.
Create a chunk from optional metadata, a ref, and payload data.
Metadata describing a chunk of node data.
Create chunk metadata from a MIME type, timestamp, and attributes.
property
writable
¶
Byte-string attribute map attached to the chunk.
staticmethod
¶
from_msgpack(data: Any) -> ChunkMetadata
Deserialize a value from MessagePack bytes.
classmethod
¶
Construct a native value from trusted field values.
Native records retain C++ invariants. This validates input rather than creating an invalid object.
Return the attribute bytes for a key, raising if it is absent.
Fragments and references¶
A NodeFragment associates one chunk with a
node ID, sequence position, and continuation state. Sequence numbers restore
the node's logical order even when transport messages arrive out of order.
A NodeRef identifies a range held by another node.
Support depends on the backing store; the store reference lists
the implementations that retain indexed references.
A fragment of a logical node carrying a Chunk or NodeRef.
Create a node fragment from Chunk/NodeRef data and framing fields.
property
writable
¶
Payload of the fragment as either a Chunk or a NodeRef.
staticmethod
¶
from_msgpack(data: Any) -> NodeFragment
Deserialize a value from MessagePack bytes.
classmethod
¶
Construct a native value from trusted field values.
Native records retain C++ invariants. This validates input rather than creating an invalid object.
Reference to a byte range of another logical node.
Create a node reference from an id, byte offset, and length.
Messages¶
A WireMessage batches action control messages
and node fragments. Control establishes or updates an action lifecycle; data
continues on the mapped nodes independently. One transport message may contain
work for several actions and nodes, and one action's stream may span many wire
messages.
A named input or output port of an action.
Create a port from a name and node id.
ActionMessage(id: str = '', name: str = '', inputs: Any = [], outputs: Any = [], headers: Mapping[str, bytes] | None = {})
A message invoking a named action with input and output ports.
Create an action message from id, name, ports, and headers.
staticmethod
¶
from_msgpack(data: Any) -> ActionMessage
Deserialize a value from MessagePack bytes.
classmethod
¶
Construct a native value from trusted field values.
Native records retain C++ invariants. This validates input rather than creating an invalid object.
A wire-format message bundling node fragments and actions.
Create a wire message from node fragments, actions, and headers.
property
writable
¶
Action messages carried by the message.
property
writable
¶
Node fragments carried by the message.
staticmethod
¶
from_json(value: str) -> WireMessage
Deserialize a message from its JSON wire encoding.
staticmethod
¶
from_msgpack(data: Any) -> WireMessage
Deserialize a value from MessagePack bytes.
classmethod
¶
Construct a native value from trusted field values.
Native records retain C++ invariants. This validates input rather than creating an invalid object.
Serialization¶
A registry of serializers and deserializers indexed by type and MIME.
New registries are empty. Pass register_defaults=True or call
register_defaults to install the built-in JSON and MessagePack
codecs. The process-wide registry returned by
get_global_serialization_registry already contains them.
Pin the wire tag used to identify obj_type in serialized data.
Overrides the default fully-qualified name, giving a type a short, stable identifier or disambiguating two like-named types.
Register serializer(obj) for a type and exact media type.
register_deserializer(obj_type: type, mimetype: str, deserializer: DeserializerFn, *, receives_chunk: bool | None = None) -> None
Register a data deserializer for a type and exact media type.
A deserializer may accept either data or data, obj_type. A
callback whose first argument is named chunk (or is annotated as a
Chunk) receives the complete chunk instead of chunk.data.
receives_chunk can be used to select that behavior explicitly.
register(obj_type: type, mimetype: str, serializer: SerializerFn, deserializer: DeserializerFn, *, receives_chunk: bool | None = None) -> None
Atomically register a serializer/deserializer pair.
Install the standard JSON and MessagePack registrations.
async
¶
to_chunk_async(obj: Any, mimetype: str = '') -> Chunk
to_chunk, off the event loop only when it might block it.
A caller's codec may take as long as it likes, so it goes to a worker
thread -- that is a contract, not an optimisation, and
test_object_serialization_and_deserialization_run_in_worker_threads
holds it. The registry's own codecs are a different matter: they are a
few microseconds, and asyncio.to_thread costs an order of magnitude
more than they do, so paying for a thread to run one is pure loss.
Deciding on the codec rather than on the payload's size is what makes this safe. A small value can still have a slow codec -- that is exactly what the test registers -- so size says nothing about whether the loop is about to be blocked.
async
¶
from_chunk_async(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', obj_type: type | None = None) -> Any
from_chunk, off the event loop only when it might block it.
The mirror of to_chunk_async; see it for why the decision is made on
the codec and not the payload.
to_chunk(obj: Any, mimetype: str = '') -> Chunk
Serialize obj into a chunk.
If mimetype is empty, the closest registered Python type wins and
registration order chooses its preferred representation. An explicit
MIME value selects a representation and can be exact or contain *
wildcards; the object's own type decides the tag, so a type
parameter in it is ignored.
from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str], obj_type: type[_T]) -> _T
from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', *, obj_type: type[_T]) -> _T
from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', obj_type: None = None) -> Any
from_chunk(chunk: Chunk, mimetype_patterns: str | Sequence[str] = '', obj_type: type | None = None) -> Any
Deserialize chunk.
Selectors choose the representation: they are matched in order against the chunk's media type and may contain wildcards. If the chunk has no MIME metadata, a supplied exact selector acts as the representation.
What comes back is chosen separately. An explicit obj_type always
wins and the registry makes a best effort to produce it, reporting the
deserializer's own error when the data will not fit. Otherwise the
chunk's type parameter names the type, and when it names nothing --
because it is absent, or because the module that would define it was
never imported -- the payload decodes to whatever its format describes:
a dict, a list, a scalar.
The Python type a wire tag names, or None if none is known.
The reverse of [_type_tag][a11.data.serialization.
SerializationRegistry._type_tag]: given a11.sdk.AudioBuffer, the
class it identifies. What a tag resolves to depends on what has been
registered and imported, so a caller that gets None has learned
that this process does not know the type -- not that it does not exist.