a11::stores::ChunkStore Class Reference

Abstract, pluggable backing store for the data of a node: an ordered, appendable log of fragments. More...

#include <cpp/a11/stores/chunk_store.h>

Inheritance diagram for a11::stores::ChunkStore:

Public Member Functions
virtual	~ChunkStore ()=default
a11::Future< data::NodeFragment >	Get (std::uint32_t seq)
	Get the fragment at a sequence number, waiting indefinitely.
virtual a11::Future< data::NodeFragment >	Get (std::uint32_t seq, absl::Time deadline)=0
	Get the fragment stored at a sequence number.
a11::Future< data::NodeFragment >	GetByArrivalOrder (std::uint64_t arrival_order)
	Get a fragment by arrival order, waiting indefinitely.
virtual a11::Future< data::NodeFragment >	GetByArrivalOrder (std::uint64_t arrival_order, absl::Time deadline)=0
	Get the fragment identified by the order in which it arrived, rather than by its sequence number.
a11::Future< std::vector< std::optional< data::NodeFragment > > >	Next ()
	Get the next logical-sequence fragment, waiting indefinitely.
a11::Future< std::vector< std::optional< data::NodeFragment > > >	Next (absl::Time deadline)
	Get the next logical-sequence fragment before a deadline.
virtual a11::Future< std::vector< std::optional< data::NodeFragment > > >	Next (absl::Time deadline, size_t limit)=0
	Get up to limit fragments from the shared logical-sequence cursor.
virtual a11::Future< std::uint32_t >	Put (data::NodeFragment fragment)=0
	Append a single fragment to the log.
virtual a11::Future< std::vector< std::uint32_t > >	PutMany (std::vector< data::NodeFragment > fragments)=0
	Append several fragments in one batch.
virtual a11::Future< data::NodeFragment >	ClearData (std::uint32_t seq)=0
	Erase the payload of the fragment at a sequence number while keeping its slot.
virtual a11::Future< std::uint32_t >	GetSeqForArrivalOrder (std::uint64_t arrival_order)=0
	Translate an arrival order into the sequence number of that fragment.
virtual a11::Future< std::optional< std::uint32_t > >	GetFinalSeq ()=0
	Get the sequence number explicitly marked as the final fragment.
a11::Future< absl::Status >	CloseWritesWithStatus (absl::Status status)
	Seal the store against further writes with a terminal status.
virtual a11::Future< absl::Status >	CloseWritesWithStatus (absl::Status status, bool return_status_if_already_closed)=0
	Seal the store against further writes with a terminal status.
virtual a11::Future< size_t >	Size ()=0
	Get the number of fragments currently in the store.
virtual absl::StatusOr< std::string >	GetId () const =0
	Get the store's node identifier.

Detailed Description

Abstract, pluggable backing store for the data of a node: an ordered, appendable log of fragments.

A ChunkStore is where the fragments of a node actually live – an ordered, appendable log of data::NodeFragment values that writers append to and readers pull from, addressable both by sequence number and by arrival order. Everything above it (nodes, readers, writers, actions) is written against this interface, so the storage backend is a pluggable detail.

This makes ChunkStore a deliberate extension point. The default LocalChunkStore keeps data in memory, but the interface can be implemented (or subclassed) to persist a stream to disk, a database, or a blob store, to inject faults in tests, or to enforce a custom retention policy, without touching any node, action, or session code. Streaming semantics – buffering, backpressure, cursoring – live in ChunkStoreReader and ChunkStoreWriter, which are layered on top.

Every data method returns an a11::Future: an awaitable that resolves when the operation completes, so callers never block the event loop. Retrieval methods take a deadline and resolve once the fragment is available or the deadline elapses.

Constructor & Destructor Documentation

~ChunkStore()

virtual a11::stores::ChunkStore::~ChunkStore ( )

virtualdefault

Member Function Documentation

ClearData()

virtual a11::Future< data::NodeFragment > a11::stores::ChunkStore::ClearData ( std::uint32_t  seq )

pure virtual

Erase the payload of the fragment at a sequence number while keeping its slot.

Used to reclaim memory for fragments that have already been consumed without disturbing the sequence numbering.

Parameters

seq	The sequence number whose payload should be cleared.

Returns

An awaitable that resolves with the (now payload-cleared) fragment.

Implemented in a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

CloseWritesWithStatus() [1/2]

a11::Future< absl::Status > a11::stores::ChunkStore::CloseWritesWithStatus ( absl::Status  status )

inline

Seal the store against further writes with a terminal status.

Convenience overload that does not return the previously recorded status if the store is already closed.

Parameters

status

The terminal status to record; readers awaiting Next() are released.

Returns

An awaitable that resolves when the close completes.

CloseWritesWithStatus() [2/2]

virtual a11::Future< absl::Status > a11::stores::ChunkStore::CloseWritesWithStatus ( absl::Status  status, bool  return_status_if_already_closed  )

pure virtual

Seal the store against further writes with a terminal status.

Once closed, readers blocked on Next() are released. Call this when a producer has finished (or failed).

Parameters

status	The terminal status to record.
return_status_if_already_closed	When true, a second close resolves with the status recorded by the first close instead of overwriting it.

Returns

An awaitable that resolves when the close completes.

Implemented in a11::stores::LocalChunkStore, a11::stores::RedisChunkStore, a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

Get() [1/2]

a11::Future< data::NodeFragment > a11::stores::ChunkStore::Get ( std::uint32_t  seq )

inline

Get the fragment at a sequence number, waiting indefinitely.

Convenience overload equivalent to Get() with an infinite deadline.

Parameters

seq	The sequence number of the fragment to retrieve.

Returns

An awaitable that resolves with the requested fragment once it is available.

Get() [2/2]

virtual a11::Future< data::NodeFragment > a11::stores::ChunkStore::Get ( std::uint32_t  seq, absl::Time  deadline  )

pure virtual

Get the fragment stored at a sequence number.

As storage may be non-local, errors are surfaced through the resolved fragment/status. Correct implementations resolve with an error rather than blocking forever once the fragment can no longer arrive (e.g. writes were closed with a smaller final sequence number).

Parameters

seq	The sequence number of the fragment to retrieve.
deadline	The absolute time after which the wait gives up.

Returns

An awaitable that resolves with the fragment once available, or with an error if the deadline elapses or the fragment can never arrive.

Implemented in a11::stores::LocalChunkStore, a11::stores::RedisChunkStore, a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

GetByArrivalOrder() [1/2]

a11::Future< data::NodeFragment > a11::stores::ChunkStore::GetByArrivalOrder ( std::uint64_t  arrival_order )

inline

Get a fragment by arrival order, waiting indefinitely.

Convenience overload equivalent to GetByArrivalOrder() with an infinite deadline.

Parameters

arrival_order

The zero-based rank in which the fragment arrived in the store.

Returns

An awaitable that resolves with the requested fragment once available.

GetByArrivalOrder() [2/2]

virtual a11::Future< data::NodeFragment > a11::stores::ChunkStore::GetByArrivalOrder ( std::uint64_t  arrival_order, absl::Time  deadline  )

pure virtual

Get the fragment identified by the order in which it arrived, rather than by its sequence number.

Useful for replaying fragments in ingestion order regardless of the sequence numbers assigned to them.

Parameters

arrival_order	The zero-based rank in which the fragment arrived in the store.
deadline	The absolute time after which the wait gives up.

Returns

An awaitable that resolves with the fragment once available, or with an error if the deadline elapses.

Implemented in a11::stores::LocalChunkStore, a11::stores::RedisChunkStore, a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

GetFinalSeq()

virtual a11::Future< std::optional< std::uint32_t > > a11::stores::ChunkStore::GetFinalSeq ( )

pure virtual

Get the sequence number explicitly marked as the final fragment.

Returns

An awaitable that resolves with the logical final sequence number, or an empty optional when no fragment has declared finality.

Finality and write closure are separate state transitions. A producer may mark a fragment final before closing the store, and CloseWritesWithStatus() does not invent a final fragment.

Implemented in a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

GetId()

virtual absl::StatusOr< std::string > a11::stores::ChunkStore::GetId ( ) const

pure virtual

Get the store's node identifier.

Returns

The node id, or an error status if the identifier is unavailable.

Implemented in a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

GetSeqForArrivalOrder()

virtual a11::Future< std::uint32_t > a11::stores::ChunkStore::GetSeqForArrivalOrder ( std::uint64_t  arrival_order )

pure virtual

Translate an arrival order into the sequence number of that fragment.

Parameters

arrival_order

The zero-based rank in which the fragment arrived in the store.

Returns

An awaitable that resolves with the corresponding sequence number.

Implemented in a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

Next() [1/3]

a11::Future< std::vector< std::optional< data::NodeFragment > > > a11::stores::ChunkStore::Next ( )

inline

Get the next logical-sequence fragment, waiting indefinitely.

Convenience overload equivalent to Next() with an infinite deadline and a limit of one.

Returns

An awaitable that resolves with at most one fragment, or a nullopt sentinel once the store reaches a clean logical end.

Next() [2/3]

a11::Future< std::vector< std::optional< data::NodeFragment > > > a11::stores::ChunkStore::Next ( absl::Time  deadline )

inline

Get the next logical-sequence fragment before a deadline.

Convenience overload equivalent to Next() with a limit of one.

Parameters

deadline

The absolute time after which the wait gives up.

Returns

An awaitable that resolves with at most one fragment, or a nullopt sentinel at a clean logical end.

Next() [3/3]

virtual a11::Future< std::vector< std::optional< data::NodeFragment > > > a11::stores::ChunkStore::Next ( absl::Time  deadline, size_t  limit  )

pure virtual

Get up to limit fragments from the shared logical-sequence cursor.

Next() advances through sequence numbers 0, 1, 2, and so on. It waits at a gap rather than switching to ingestion order; use GetByArrivalOrder() for that view. After the final sequence or a clean write closure, the result includes a nullopt end sentinel. Call repeatedly to follow the store as it grows.

Parameters

deadline	The absolute time after which the wait gives up.
limit	The maximum number of fragments to return in one call.

Returns

An awaitable that resolves with up to limit data fragments; a clean end may append a nullopt sentinel.

Implemented in a11::stores::LocalChunkStore, a11::stores::RedisChunkStore, a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

Put()

virtual a11::Future< std::uint32_t > a11::stores::ChunkStore::Put ( data::NodeFragment  fragment )

pure virtual

Append a single fragment to the log.

Parameters

fragment

The fragment to append.

Returns

An awaitable that resolves with the sequence number assigned to the fragment once the write is accepted.

Implemented in a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

PutMany()

virtual a11::Future< std::vector< std::uint32_t > > a11::stores::ChunkStore::PutMany ( std::vector< data::NodeFragment >  fragments )

pure virtual

Append several fragments in one batch.

Preferred over repeated Put() calls when many fragments are emitted at once, to reduce round-trips.

Parameters

fragments

The fragments to append, in order.

Returns

An awaitable that resolves with the sequence numbers assigned to the fragments, in the same order.

Implemented in a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

Size()

virtual a11::Future< size_t > a11::stores::ChunkStore::Size ( )

pure virtual

Get the number of fragments currently in the store.

Returns

An awaitable that resolves with the current fragment count.

Implemented in a11::stores::LocalChunkStore, and a11::stores::RedisChunkStore.

---

The documentation for this class was generated from the following file:

• cpp/a11/stores/chunk_store.h