06 — ECS
Code: src/Typhon.Engine/Ecs/
Typhon's data model is ECS — Entity, Component, System. Entities are 64-bit identifiers, components are blittable unmanaged structs, and archetypes declare which components an entity has. The ECS API in this folder is what application code actually touches: Spawn, Destroy, Open, OpenMut, plus the supporting types (EntityId, EntityRef, Comp<T>, EntityLink<T>, PointInTimeAccessor, ClusterRef<TArch>).
If you've used Unity DOTS, Bevy, or flecs the shape will feel familiar — but Typhon's ECS gives you a per-component storage-mode choice that other ECSs don't: each component declares whether it's Versioned (full MVCC snapshot isolation, 05-revision), SingleVersion (in-place, no isolation), or Transient (in-memory, lost on restart). Versioned components flow through the transaction model (08-transactions) and the WAL; SV/Transient bypass MVCC entirely and write straight to the slot. The storage layer (02-storage) backs all three. §8 below covers how to pick a mode and what each gives up.
The ECS data engine: application calls (
Spawn / Destroy / Open / OpenMut) run on a Transaction; entities route through the per-archetype EntityMap → EntityRecord (no PK index); ComponentTable owns segments per storage mode (Versioned → ComponentSegment + CompRevTableSegment; SV → ComponentSegment; Transient → TransientComponentSegment); secondary B+Tree indexes and the schema system sit alongside.
1. The model
| Concept | What it is |
|---|---|
| Entity | A unique identifier (EntityId). Holds no data of its own. |
| Component | An unmanaged struct attached to an entity. Components are data, not behaviour. |
| Archetype | A typed shape — the set of components an entity has. Declared as a C# class inheriting Archetype<TSelf>. |
| System | Behaviour — runs in the scheduler (10-runtime). Reads/writes components via accessors. |
Every entity belongs to exactly one archetype, fixed at spawn time. Components are stored archetype-major: all Position data for archetype Ant is contiguous, separately from the Position data for archetype Building. This is the standard ECS layout — branch-free iteration, cache-friendly scans.
Archetype declaration
[Archetype]
public sealed partial class Ant : Archetype<Ant>
{
public static readonly Comp<Position> Position = Register<Position>();
public static readonly Comp<Velocity> Velocity = Register<Velocity>();
public static readonly Comp<Health> Health = Register<Health>();
}
The archetype's identity is the CLR type name (or [Archetype(Name = "…")] if you want a name decoupled from the class). The engine auto-assigns a per-process catalog id and a per-DB routing id (persisted in ArchetypeR1, re-matched by name on reopen) — no numeric id is set in source, and a DB can hold up to 65,536 archetypes. The Register<T>() calls declare components. CRTP (Archetype<Ant>) gives compile-time type identity. Archetypes self-register up-front via a generated [ModuleInitializer] barrier at assembly load — lock-guarded and thread-safe, not lazy first-access reflection.
The class is declared partial so a source generator can extend it — see §5 Generated accessors below. Without partial the archetype still works for Spawn / Open / OpenMut, but the generated typed bulk accessors aren't emitted.
To rename an archetype (or its backing class) without losing its persisted data, keep the old identity as a hatch: [Archetype(Name = "NewName", PreviousName = "OldName")] — the engine matches PreviousName against ArchetypeR1 on reopen and adopts the existing routing id. This mirrors the component/field rename hatches in 04-schema.
Inheritance is single-parent:
public sealed partial class FlyingAnt : Archetype<FlyingAnt, Ant>
{
public static readonly Comp<Wings> Wings = Register<Wings>();
}
Parent components are inherited; slot indices start with the parent's. No diamond inheritance.
2. Identity
EntityId
8 bytes, packed:
| Bits | Field |
|---|---|
| 0–15 | routingId (16 bits, up to 65,536 per DB) — per-DB archetype routing id; routes to per-archetype storage. Extract with (ushort)value |
| 16–63 | EntityKey (48 bits, ~281 × 10¹²) — monotonic per-archetype, never recycled. Extract with value >> 16 |
public readonly struct EntityId : IEquatable<EntityId>
{
public long EntityKey { get; } // high 48 bits
public ushort ArchetypeId { get; } // low 16 bits — the per-DB archetype routing id
public bool IsNull { get; }
public static readonly EntityId Null;
}
Why monotonic, never recycled: no ABA hazards — a stale EntityId cached anywhere in the system either matches a still-live entity (correct) or matches nothing (correct). No version field needed, no recycling races.
EntityLink<T> — typed references
Compile-time-safe wrapper. EntityLink<Building> accepts Building, House, or any descendant — polymorphic at the archetype level.
public sealed partial class Worker : Archetype<Worker>
{
public static readonly Comp<EntityLink<Building>> Home = Register<EntityLink<Building>>();
}
Stored inline — implicit conversions to and from EntityId keep the call sites clean.
3. Component handles
Comp<T> — typed component handle
public readonly struct Comp<T> where T : unmanaged
{
public ComponentValue Set(in T value); // for use in Spawn
public ComponentValue Default();
}
Comp<T> carries the global ComponentTypeId — an in-memory-only handle (not persisted, never on the WAL wire; the wire addresses the per-archetype slot instead, see 11-durability). The same component type used in multiple archetypes shares the same ComponentTypeId, so the engine can route data correctly regardless of which archetype holds it. Slot resolution happens at runtime via ArchetypeMetadata.GetSlot(componentTypeId).
ComponentValue — spawn-time payload
128-byte struct: 12-byte header (ComponentTypeId + DataSize + pad) + 112-byte inline payload + 4-byte tail pad. MaxPayloadSize = 112.
Components larger than 112 bytes don't fit in a ComponentValue — for those, spawn with the smaller defaults, then OpenMut and write the payload incrementally. (In practice, most ECS components are small; if you're routinely over 112 B, that's a sign to split the component.)
4. Lifecycle: Spawn / Destroy
These live on Transaction (08-transactions) — they're mutations and need transactional context.
Spawn<TArch>
Transactions/public/Transaction.ECS.cs:147
EntityId id = tx.Spawn<Ant>(
Ant.Position.Set(new Position(10, 20)),
Ant.Health.Set(new Health(100))
);
- Allocates a fresh
EntityKey(monotonicInterlocked.Incrementon the per-archetypeNextEntityKey). - Allocates chunks in the relevant
ComponentTables (Versioned components go intoComponentSegment, Transient intoTransientComponentSegment). - Copies provided
ComponentValuepayloads; unspecified components are zero-initialized and disabled (see EnabledBits below). - The entity is held in a pending spawn list on the
Transaction; insertion into the per-archetypeEntityMaphappens at commit withBornTSN = TSN.
So a spawn is visible to this transaction immediately, visible to other transactions only after commit.
Destroy(EntityId)
Transactions/public/Transaction.ECS.cs:562
tx.Destroy(id);
Marks the entity for destruction. The EntityRecord is stamped with a DeadTSN. Other transactions still see the entity until their snapshot TSN passes DeadTSN. Cleanup of revision storage happens lazily via the DeferredCleanupManager once no active snapshot can see the entity any more.
SpawnBatch<TArch>
For mass spawning (e.g., level load). Amortizes the per-call overhead: one EnsureMutable check, one Interlocked.Add for all entity keys, one epoch refresh at the end.
Span<EntityId> ids = stackalloc EntityId[1000];
tx.SpawnBatch<Ant>(ids, Ant.Health.Set(new Health(100)));
5. Access: Open / OpenMut
Reading and mutating components flows through accessors. Three flavours, depending on the context.
EntityAccessor — base read accessor
Ecs/public/EntityAccessor.cs, EntityAccessor.ECS.cs
EntityRef e = accessor.Open(id); // read-only
if (accessor.TryOpen(id, out EntityRef e2)) { ... }
EntityRef em = accessor.OpenMut(id); // SV/Transient writes only
OpenandTryOpenresolve the entity at the accessor'sTSN, applying MVCC visibility (BornTSN ≤ TSN < DeadTSN) andEnabledBitsoverrides.- The returned
EntityRefis aref struct— stack-allocated, must not outlive the accessor that created it. OpenMuton the baseEntityAccessoris for SingleVersion / Transient components only. Versioned writes need aTransaction.OpenMutoverride (which addsEnsureMutable+ state transition).
Transaction (extends EntityAccessor)
Inside a Transaction, OpenMut upgrades to the full mutating path: the transaction is marked InProgress, revision chains get extended on commit, the UowId from the transaction's UnitOfWorkContext (01-foundation) gets stamped on every new revision element.
PointInTimeAccessor — parallel reads
Ecs/public/PointInTimeAccessor.cs
For parallel query systems that read at a single frozen MVCC snapshot across many worker threads.
var pta = new PointInTimeAccessor();
pta.Attach(dbe, workerCount); // allocates fresh TSN, resets per-worker accessors
// dispatch parallel work — each worker uses pta.GetAccessor(workerId).Open(...)
- One per-worker
EntityAccessoris stored in a flatEntityAccessor[]indexed byworkerId— no per-entity dictionary lookup. - Reused across ticks:
Attach()allocates a fresh TSN and resets accessors while preserving the warmChunkAccessorpage caches — zero allocation after first-tick warmup. - Cannot Spawn / Destroy / Commit / Rollback. Cannot write Versioned components (throws). SV/Transient writes are supported.
PTA is the right tool when the workload is "read a lot, in parallel, at one consistent snapshot" — e.g., a QuerySystem that computes statistics over all entities of an archetype.
ArchetypeAccessor<TArch> — fast path
Ecs/public/ArchetypeAccessor.cs
ArchetypeAccessor<Ant> ants = accessor.For<Ant>();
EntityRef ant = ants.Open(id);
Pre-bound to a specific archetype. Bypasses epoch checks, archetype lookup, and MVCC visibility on every Open call — intended for PTA workers in parallel QuerySystems where these checks are amortized to once per dispatch, not once per entity.
Generated accessors — ReadAll / ReadWriteAll
EntityRef resolves components one at a time (e.Read(Ant.Position)). When you want all of an archetype's components at once with compile-time field names, Typhon generates them. ArchetypeAccessorGenerator is an incremental Roslyn source generator that, for every [Archetype] partial class, emits:
- a
Refsref struct (read-only) and aMutRefsref struct (mutable), one typed field per component, and - static
ReadAll(tx, id)→RefsandReadWriteAll(tx, id)→MutRefsmethods,
into {ArchetypeName}.g.cs.
// read every component of the entity in one call
var refs = Ant.ReadAll(tx, id);
float x = refs.Position.X;
int hp = refs.Health.Current;
// mutate in place
var mut = Ant.ReadWriteAll(tx, id);
mut.Position.X = 999;
mut.Health.Current = 50;
Inheritance flows through: FlyingAnt.ReadAll(tx, id) exposes the parent's Position / Velocity and FlyingAnt's own Wings. The generated structs are ref structs — same stack-only, no-copy rules as EntityRef. The generator fires only when the class is partial; a non-partial archetype compiles fine but gets no ReadAll / ReadWriteAll.
(Two sibling generators live alongside it: SourceLocationGenerator and TraceEventGenerator, both for the profiler — see 12-observability.)
6. EntityRef — the working handle
A ref struct returned by Open / OpenMut. Carries the entity's location data inline (a fixed int[16] array of chunk IDs, one per component slot) plus optional cluster-storage fields.
EntityRef ant = accessor.Open(id);
ref readonly var pos = ref ant.Read(Ant.Position); // zero-copy
ant.Write(Ant.Position, new Position(x, y)); // requires writable EntityRef
Read<T>(Comp<T>)returns aref readonly Tdirectly into the chunk page (or cluster slot). Zero copy.Write<T>(Comp<T>, T)mutates in place; for Versioned components, this is the initial write before commit — the actual revision-chain extension happens during commit.IsValid,IsWritableproperties.IsWritablereflects whether theEntityRefwas obtained viaOpenMut(true) vsOpen(false).EnabledBitstracks per-component enable state (16-bit mask, up to 16 components per archetype). A disabled component is present in storage but logically absent from queries — used for fast component toggles without re-archetyping.
7. Cluster storage
For high-density archetypes (think hundreds of thousands of Ants), Typhon supports cluster storage: groups of 8–64 entities share a single physical chunk, components stored Structure-of-Arrays within the cluster. This dramatically improves cache density for full-scan systems.
ClusterRef<TArch>
foreach (var cluster in accessor.GetClusterEnumerator<Ant>())
{
ulong bits = cluster.OccupancyBits;
while (bits != 0)
{
int idx = BitOperations.TrailingZeroCount(bits);
bits &= bits - 1;
ref var pos = ref cluster.Get(Ant.Position, idx);
// mutate or read
}
}
The TZCNT loop walks only live slots — branch-free, very fast. OccupancyBits is a 64-bit mask at offset 0 of the cluster chunk; EnabledBits(slot) masks for component-disabled entities; ActiveBits(slot) = OccupancyBits & EnabledBits(slot) is the common case.
Cluster storage is opt-in per archetype (IsClusterEligible on ArchetypeMetadata). Eligible archetypes report ClusterRef-accessible storage; others use legacy per-entity storage.
8. Storage modes
Typhon.Schema.Definition/StorageMode.cs — set per component type via the StorageMode argument on its [Component] attribute ([Component("name", rev, StorageMode = StorageMode.SingleVersion)]); the default is Versioned. Three modes, very different contracts.
The contract per mode
| Versioned (default) | SingleVersion | Transient | |
|---|---|---|---|
| Write path | Allocates a new revision chunk; stamps the writer's UowId; chain head extended at commit |
Returns a ref T into the live SoA chunk page; the store is the page mutation |
Same as SV — ref T into the live page |
| Visibility | Snapshot-isolated; readers at TSN ≤ commit don't see the write | Immediate — every reader (any TSN, any PTA worker) sees the new bytes the moment the store retires | Same as SV — immediate |
| Conflict detection | Yes (write-write at commit, see 08-transactions) | None — last writer wins, silently | None |
Rollback (tx.Rollback) |
Voids the new revision element; chain head untouched | Cannot be reverted — no before-image is captured anywhere¹ | Cannot be reverted |
| Durability | WAL on commit (per DurabilityMode), then checkpointed |
Tick-fence WAL — dirtied chunks are persisted at the next tick boundary, regardless of whether the transaction committed | Never persisted — lost on shutdown / crash |
| Crash recovery | Re-applied from WAL records | Replayed from tick-fence WAL records | None — starts zero on every open |
| Cost | Highest (chain walk on read, allocation on write, cleanup) | Low (one indirect store) | Lowest (one indirect store, no I/O ever) |
| Storage | ComponentSegment + CompRevTableSegment |
ComponentSegment (no rev table) |
TransientComponentSegment (separate, in-memory only) |
¹ The lone exception: if you Spawn<T> an entity whose component set includes SV fields, and then Rollback, the freshly-allocated SV chunks are freed and the entity is never published to the EntityMap, so its SV bytes effectively vanish with it. Mutating an existing entity's SV slot is the unrollbackable case.
DurabilityDiscipline — a second, orthogonal axis on SingleVersion
The table above describes SingleVersion under its default discipline, DurabilityDiscipline.TickFence. A second discipline, Commit, is selected per transaction and is not a fourth StorageMode — it shares the identical SV cluster layout and only changes the rows below for the writes it covers:
| Aspect | TickFence (default) |
Commit |
|---|---|---|
| Visibility to other transactions | immediate — dirty read, before your tx commits | only at Transaction.Commit — read-committed, no partial visibility |
| Rollback | cannot be reverted | O(1) — the staged value is discarded; HEAD was never touched |
| Durability | tick-fence WAL, batched at tick boundary (≤1-tick loss) | zero-loss, atomic — WAL'd at Transaction.Commit (per DurabilityMode), no checkpoint required |
| Conflict detection | none — last-writer-wins | none — last-writer-wins (unchanged) |
| Revision chain | none | none |
Commit stages writes into a per-transaction buffer (the SV slot itself is untouched until commit), then publishes them in place — read-committed isolation and O(1) rollback without paying for MVCC. It applies only to SingleVersion (Versioned is always commit-scoped; Transient is never durable). See 11-durability for the wire format and claude/design/Ecs/committed-storage-mode.md for the full spec.
What this really means
Versioned is the only mode inside the transaction's ACID envelope — under the default TickFence discipline. SV and Transient writes leak out of the transaction in three ways:
- No isolation: another transaction (or a PTA worker) reading the same SV/Transient slot during your transaction sees the post-write value immediately, not a snapshot. Multiple concurrent writers on the same SV slot race with no synchronization — "developer owns concurrency" for SV/Transient mutations.
- No atomicity: if you write to several SV components and then crash or rollback, the writes that already retired stay. There is no "all or nothing".
- Durability follows the tick, not the commit: SV writes ride the tick-fence WAL pipeline (
DirtyBitmap→ tick-boundary persistence). Whether your transaction commits or rolls back doesn't change what's persisted.
A transaction around SV writes still gives you thread affinity and a consistent read snapshot for any Versioned components in the same archetype, but it does not give you the right to undo an SV mutation. If that matters, use Versioned — or escalate the SV component to DurabilityDiscipline.Commit, above, if you need atomicity and rollback but not snapshot isolation or temporal queries.
Picking a mode
- Versioned — anything that needs ACID. Account balances, inventory, persistent game state, anything where "did this commit?" is a question you care about answering.
- SingleVersion — hot in-place fields whose history doesn't matter and where last-writer-wins is acceptable. Examples: cached AI navigation cost, animation tween state, cached query results. SV is persisted across restarts (via tick-fence WAL), so use it when you want durability without isolation — not for ephemeral state.
- Transient — runtime-only fields that should not survive a restart. Per-tick scratch space, ephemeral coordination flags, dispatch-only metadata.
Mixing modes in one archetype
A single archetype can mix modes freely — because the mode lives on each component type (its [Component] attribute), the archetype just registers them:
[Component("Position", 1)] // Versioned (default)
public struct Position { public float X, Y, Z; }
[Component("NavCost", 1, StorageMode = StorageMode.SingleVersion)]
public struct NavCost { public float Value; }
[Component("TickScratch", 1, StorageMode = StorageMode.Transient)]
public struct TickScratch { public int Flags; }
public sealed partial class Ant : Archetype<Ant>
{
public static readonly Comp<Position> Position = Register<Position>(); // Versioned
public static readonly Comp<NavCost> NavCost = Register<NavCost>(); // SingleVersion
public static readonly Comp<TickScratch> TickScratch = Register<TickScratch>(); // Transient
}
Each Register<T>() simply picks up the mode declared on T's [Component]. Internal layout: Position lives in the archetype's main ComponentSegment with a revision chain in CompRevTableSegment; NavCost in the same ComponentSegment but no rev table; TickScratch in a separate TransientComponentSegment that's never persisted. ArchetypeMetadata.VersionedSlotMask and .TransientSlotMask drive the per-slot routing.
Who can write what
| Accessor | Versioned writes | SV writes | Transient writes |
|---|---|---|---|
Transaction.OpenMut |
✓ | ✓ | ✓ |
EntityAccessor.OpenMut (base) |
✗ throws | ✓ | ✓ |
PointInTimeAccessor workers |
✗ throws | ✓ | ✓ |
ArchetypeAccessor (fast path) |
follows the underlying accessor's rules |
The Versioned-write restriction on the base EntityAccessor and PTA is structural: Versioned writes need a Transaction to allocate a UowId, extend the revision chain, and participate in commit. PTA was designed for parallel reads at a frozen snapshot; SV/Transient writes are tolerated because they don't need any of that machinery.
9. Behind the curtain
You don't need this to use the ECS API, but here's what's happening underneath. Detail in 05-revision and 02-storage.
EntityMap— per-archetype, persistent hash map keyed byEntityKey. StoresEntityRecord(chunk IDs per slot +BornTSN/DeadTSN+EnabledBits). Implemented viaPagedHashMapon top ofComponentSegment.EntityRecord(file) — the per-entity row in the EntityMap. TheEnabledBits,BornTSN,DeadTSN, and the locations of each component chunk.ArchetypeMetadata(file) — runtime descriptor: slot count, per-slot component type IDs, Versioned/Transient slot masks, cluster layout if eligible.ArchetypeRegistry(file) — global registry; archetypes self-register up-front via a generated[ModuleInitializer]barrier at assembly load (lock-guarded, thread-safe), assigning slot indices and handling parent-child slot ordering.EnabledBitsOverrides(file) — MVCC-aware override map for per-entity enable bit changes that haven't been committed to theEntityRecordyet.ResolveEnabledBits(entityKey, baseEnabledBits, TSN)gives the snapshot-correct view.EnabledBitsHistory(file) — historical record of EnabledBits changes for MVCC visibility queries.FieldShadowBuffer(file) — captures the old index key (8 bytes,ChunkId / EntityPK / OldKey) for each indexed-field mutation so the deferred B+Tree maintenance pass can locate and update the entry. Does not buffer the new value itself — the value mutation is direct.DirtyBitmap+DirtyBitmapRing(dirty) — track which entities have been mutated this tick; consumed by view-system delta computation (09-querying).ZoneMapArray(file) — per-cluster min/max summaries used by query planning for skip-on-mismatch optimization.SimdPredicateEvaluator(file) — vectorized predicate evaluation over cluster contents.DeferredCleanupManager(file) — tail-driven garbage collection of revision chain elements that no active snapshot can see any more.
Commit pipeline (where ECS hits transactions)
The transaction commit (08-transactions) calls:
PrepareEcsDestroys— finalizeDeadTSNfor entities being destroyed; clearEnabledBitsoverrides.FlushEcsPendingOperations— walk the spawned-entity list, allocate finalEntityRecords, insert into the per-archetypeEntityMap.FinalizeSpawns— stampBornTSN = TSNon all new entries.- Cluster-versioned slot commit — for cluster-storage archetypes, the cluster slots need their per-slot revision pointer updated; this is the equivalent of revision-chain extension for the cluster layout.
After commit, the changes are visible to any new snapshot. Active snapshots with TSN < commit-TSN still see the pre-commit state — that's MVCC working as designed.
Epoch interaction
The ECS resolve path enters an EpochGuard (01-foundation §4) on every entity read — page accessors need this protection so the page cache can't evict a page mid-read. There's an EpochRefreshInterval = 128 constant (EntityAccessor.cs:32) — every 128 entity ops on a Transaction, the engine refreshes the pinned epoch and flushes the change set to release excess dirty marks (02-storage §5). This bounds how long a single transaction can hold pages dirty.
10. Quick usage map
// Setup
public sealed partial class Ant : Archetype<Ant>
{
public static readonly Comp<Position> Position = Register<Position>();
public static readonly Comp<Health> Health = Register<Health>();
}
// Spawn
using var uow = dbe.CreateUnitOfWork(DurabilityMode.GroupCommit);
using var tx = uow.CreateTransaction();
EntityId ant = tx.Spawn<Ant>(
Ant.Position.Set(new Position(0, 0)),
Ant.Health.Set(new Health(100)));
tx.Commit();
// uow flushes / fsyncs per DurabilityMode
// Read (single thread)
using var readTx = dbe.CreateTransaction(readOnly: true);
EntityRef a = readTx.Open(ant);
ref readonly var pos = ref a.Read(Ant.Position);
// Read (parallel system)
var pta = new PointInTimeAccessor();
pta.Attach(dbe, workerCount: 8);
// dispatch — each worker uses pta.GetAccessor(workerId)
// Iterate clusters (high density, full scan)
foreach (var cluster in accessor.GetClusterEnumerator<Ant>())
{
ulong bits = cluster.OccupancyBits;
while (bits != 0)
{
int idx = BitOperations.TrailingZeroCount(bits);
bits &= bits - 1;
ref readonly var pos = ref cluster.Get(Ant.Position, idx);
// ...
}
}
See also
- 01-foundation —
UnitOfWorkContext,EpochGuard, allocators that ECS sits on - 02-storage —
PagedMMF,ComponentSegment, chunk allocation, dirty tracking - 04-schema —
FieldType, attribute system, schema persistence and evolution - 05-revision — MVCC mechanics,
CompRevStorageElement, revision chains - 08-transactions —
Transaction,UnitOfWork, commit pipeline - 09-querying —
EcsQuery,EcsView, statistics