Table of Contents

Class DatabaseEngine

Namespace
Typhon.Engine
Assembly
Typhon.Engine.dll

The main database engine class providing transaction-based access to component data.

public class DatabaseEngine : ResourceNode, IResource, IDisposable, IMetricSource, IDebugPropertiesProvider
Inheritance
DatabaseEngine
Implements
Inherited Members
Extension Methods

Remarks

DatabaseEngine registers itself under the DataEngine subsystem in the resource tree. ComponentTables are registered as children of this engine.

Properties

DBD

Registry of the component and archetype schema definitions registered on this engine instance.

public DatabaseDefinitions DBD { get; }

Property Value

DatabaseDefinitions

EpochManager

Epoch manager coordinating safe, lock-free memory reclamation across concurrent readers and writers.

public EpochManager EpochManager { get; }

Property Value

EpochManager

IsDisposed

true once the engine has been disposed; further operations on it are invalid.

public bool IsDisposed { get; }

Property Value

bool

IsNewlyCreated

true when this open created the database bundle (it did not exist before); false when it reopened an existing one. Use it to gate one-time bootstrap work (initial data, first-run migrations). Note the engine already offers a crash-safe, declarative alternative via Seed(int, Action<Transaction>); prefer that for seeding, and this flag for cheaper conditional logic. It reflects file existence at open, so after a crash mid-initialisation it reads false — it is not, by itself, a "seeding completed" signal.

public bool IsNewlyCreated { get; }

Property Value

bool

MMF

Backing paged memory-mapped file store holding all persisted segments of this database.

public ManagedPagedMMF MMF { get; }

Property Value

ManagedPagedMMF

PageBodyReadCount

Number of page bodies read through TryReadPageBody(int, Span<byte>) since the engine opened. The Database File Map's detail tier is viewport-scoped, never a full-file scan — the A2 tests assert this counter grows only by the size of the requested region.

public long PageBodyReadCount { get; }

Property Value

long

Methods

BeginBulkLoad(BulkLoadOptions)

Open a new BulkLoadSession. Acquires the engine-wide exclusive bulk gate; throws BulkSessionAlreadyActiveException if a session is already open.

public BulkLoadSession BeginBulkLoad(BulkLoadOptions options = null)

Parameters

options BulkLoadOptions

Optional configuration. Defaults to a new BulkLoadOptions if null.

Returns

BulkLoadSession

A fresh, open bulk session. Caller must CompleteBulkLoad() or Dispose() when done; the engine-wide gate is held until then.

Remarks

See claude/design/Durability/BulkLoad/01-api.md for the API reference and lifecycle. A bulk session is thread-affine — only the thread that opened it may call methods on it. Regular transactions can run concurrently and see the pre-bulk snapshot.

Mutate through the returned session's Spawn<TArch>(params ReadOnlySpan<ComponentValue>), Update<T>(EntityId, in T), and Destroy(EntityId), then call CompleteBulkLoad() to commit and checkpoint the bulk durably. Disposing without completing discards it — none of the bulk's revisions become visible (UR-03).

Exceptions

BulkSessionAlreadyActiveException

A bulk session is already open on this engine.

ClassifyAllPages(Span<StoragePageType>)

Classifies every file page by semantic type into dest (length ≥ file page count). Built entirely from in-memory structures — the occupancy bitmap and the segment registry — with no data-page I/O. A page owned by no enumerated segment and not a reserved root page resolves to Unknown.

public void ClassifyAllPages(Span<StoragePageType> dest)

Parameters

dest Span<StoragePageType>

ConfigureSpatialGrid(SpatialGridConfig)

Sets the spatial grid configuration for this engine. Must be called before InitializeArchetypes(). Only required when at least one cluster-eligible archetype has a spatial component — non-spatial engines never need this call.

public void ConfigureSpatialGrid(SpatialGridConfig config)

Parameters

config SpatialGridConfig

Exceptions

InvalidOperationException

Thrown if called after InitializeArchetypes().

CreateUnitOfWork(DurabilityMode, TimeSpan)

Creates a new Unit of Work — the durability boundary for user operations. All transactions must be created through a UoW.

public UnitOfWork CreateUnitOfWork(DurabilityMode durabilityMode = DurabilityMode.Deferred, TimeSpan timeout = default)

Parameters

durabilityMode DurabilityMode

Controls when WAL records become crash-safe. Default is Deferred.

timeout TimeSpan

Lifetime timeout for this UoW. Default uses DefaultUowTimeout.

Returns

UnitOfWork

A new UnitOfWork in Pending state.

Exceptions

ResourceExhaustedException

All UoW registry slots are in use and the deadline expired.

Dispose(bool)

Releases engine resources following the standard dispose pattern. Idempotent — a no-op once IsDisposed is set. Runs the core teardown inside a try/finally so an owned service provider is still released even if a teardown step throws.

protected override void Dispose(bool disposing)

Parameters

disposing bool

true when called from Dispose(); false when called from the finalizer.

EnumerateStorageSegments()

Enumerates every live logical segment's on-disk footprint — the per-ComponentTable segments plus the occupancy-bitmap segment. Authoritative: walks the component-table registry rather than the page cache's lazy segment cache. Read-only; consumes only in-memory structures.

public IReadOnlyList<StorageSegmentDescriptor> EnumerateStorageSegments()

Returns

IReadOnlyList<StorageSegmentDescriptor>

ForceCheckpoint()

Triggers an immediate checkpoint cycle. Flushes all dirty data pages, advances CheckpointLSN, and recycles WAL segments. No-op if WAL/checkpoint is not configured.

public void ForceCheckpoint()

GetAllComponentTables()

Returns all registered ComponentTables. Used by Typhon.Engine.DatabaseEngine.StatisticsWorker to iterate tables, and by external tooling (e.g., the Workbench Schema Inspector) to enumerate the schema. Values returns a stable snapshot, so concurrent registration is safe.

public IEnumerable<ComponentTable> GetAllComponentTables()

Returns

IEnumerable<ComponentTable>

GetArchetypeClusterChunkCount(ushort)

Number of active cluster chunks for the given archetype in this engine. Returns 0 for legacy archetypes (non-cluster storage) or if the archetype has no cluster state yet. Paired with GetArchetypeEntityCount(ushort) the caller can derive occupancy for cluster archetypes: entityCount / (chunkCount * ArchetypeClusterInfo.ClusterSize).

public int GetArchetypeClusterChunkCount(ushort archetypeId)

Parameters

archetypeId ushort

Returns

int

GetArchetypeEntityCount(ushort)

Current entity count for the given archetype in this engine. Returns 0 if the archetype has no state in this engine (not registered or not yet initialized). Used by external tooling (Workbench Schema Inspector) to populate the Archetype panel — intentionally a scalar accessor so the internal Typhon.Engine.Internals.ArchetypeEngineState type does not need to leak into the public surface.

public long GetArchetypeEntityCount(ushort archetypeId)

Parameters

archetypeId ushort

Returns

long

GetComponentTable(Type)

Returns the ComponentTable registered for type, or null if it is not registered.

public ComponentTable GetComponentTable(Type type)

Parameters

type Type

The registered component type.

Returns

ComponentTable

GetComponentTable<T>()

Returns the ComponentTable registered for T, or null if none is registered.

public ComponentTable GetComponentTable<T>() where T : unmanaged

Returns

ComponentTable

Type Parameters

T

The registered unmanaged component type.

GetDebugProperties()

Gets detailed debug properties for diagnostic drill-down.

public IReadOnlyDictionary<string, object> GetDebugProperties()

Returns

IReadOnlyDictionary<string, object>

A dictionary of property names to values. Values should be primitives, strings, or types that have meaningful ToString() implementations.

Remarks

This method may allocate (returns new dictionary) since it's called infrequently. Callers should cache results if needed across multiple accesses.

Property naming convention: Use dot-notation for nested concepts (e.g., "ComponentSegment.Allocated", "Contention.MaxWaitUs").

GetIndexRef<T, TKey>(Expression<Func<T, TKey>>)

Returns an IndexRef for a secondary indexed field of component T. Resolve once (cold path), reuse many times at zero cost (hot path).

public IndexRef GetIndexRef<T, TKey>(Expression<Func<T, TKey>> keySelector) where T : unmanaged

Parameters

keySelector Expression<Func<T, TKey>>

Returns

IndexRef

Type Parameters

T
TKey

GetPKIndexRef<T>()

Returns an IndexRef for the primary key index of component T. Resolve once (cold path), reuse many times at zero cost (hot path).

public IndexRef GetPKIndexRef<T>() where T : unmanaged

Returns

IndexRef

Type Parameters

T

GetPageResidency(int, out bool, out bool)

Reports whether file page filePageIndex is currently resident in the page cache and, if so, whether it is dirty. Non-faulting — it never triggers page I/O — so it must be called before TryReadPageBody(int, Span<byte>) for that page if pre-introspection residency is wanted (the read faults the page in). An out-of-range index yields resident = false.

public void GetPageResidency(int filePageIndex, out bool resident, out bool dirty)

Parameters

filePageIndex int
resident bool
dirty bool

GetRequiredAssemblies()

The schema-assembly manifest persisted in this database: the identity of every .NET assembly that declares a stored component or archetype. Read from the AssemblyR1 catalog, which is loaded on every open (including schemaless), so this is available without any user schema DLL. The core engine assembly is intentionally excluded — it is always loaded. Consumed by tooling (the Workbench) to locate and load the schema assemblies a file depends on.

public IReadOnlyList<AssemblyName> GetRequiredAssemblies()

Returns

IReadOnlyList<AssemblyName>

GetSchemaHistory()

Returns all schema history entries from the audit trail, ordered by primary key (chronological).

public IReadOnlyList<SchemaHistoryR1> GetSchemaHistory()

Returns

IReadOnlyList<SchemaHistoryR1>

GetWalTotalBytes()

Total byte size of the write-ahead log across all segment files (0 when no WAL is active).

public long GetWalTotalBytes()

Returns

long

InitializeArchetypes()

Initialize ECS archetype storage. For each registered archetype, allocates a per-archetype RawValueHashMap and connects component slots to their ComponentTables. Must be called after all components are registered.

public void InitializeArchetypes()

MarkClusterSlotDirty(int, int, int)

Mark a single entity slot as dirty in the cluster dirty bitmap. Call from game systems that use the direct cluster iteration path (GetSpan<T>(Comp<T>)) and need migration detection or WAL tracking. The chunkId comes from ChunkId. Thread-safe (uses Or(ref int, int) internally via SetDirty).

public void MarkClusterSlotDirty(int archetypeId, int chunkId, int slotIndex)

Parameters

archetypeId int
chunkId int
slotIndex int

Open(string, Action<TyphonOptions>, ILoggerFactory)

Opens (or creates) a database backed by a self-contained service container — the standalone counterpart to the DI AddTyphon(IServiceCollection, Action<TyphonOptions>) path. The returned engine owns the container and disposes it as part of its own disposal, so using var dbe = DatabaseEngine.Open(...) is leak-free.

public static DatabaseEngine Open(string databaseFile, Action<TyphonOptions> configure = null, ILoggerFactory loggerFactory = null)

Parameters

databaseFile string

The database path; the canonical extension is .typhon. See DatabaseFile(string).

configure Action<TyphonOptions>

Optional further configuration — component/archetype registrations and advanced options.

loggerFactory ILoggerFactory

Optional logger factory. When null, logging resolves to a no-op (no output) while still satisfying the engine's required loggers.

Returns

DatabaseEngine

A ready-to-use engine that owns and disposes its private container.

ReadMetrics(IMetricWriter)

Writes current metric values to the provided writer.

public void ReadMetrics(IMetricWriter writer)

Parameters

writer IMetricWriter

The writer to report metrics to. Call one or more Write* methods.

Remarks

Called during snapshot collection (every 1-5 seconds). Implementations should:

  • Read fields and write to writer quickly (target < 100ns)
  • Not allocate (no new objects, no LINQ, no string formatting)
  • Not acquire locks (read fields directly, accept slight inconsistency)
  • Not call other IMetricSource.ReadMetrics() — the graph handles recursion

Only report metric kinds that are relevant to this resource. For example, a TransactionPool might only report Capacity and Throughput, not Memory or DiskIO.

RegisterArchetype(Type)

Finalizes an [Archetype] into the process-global catalog (feature #514 Phase 5). Called by the per-assembly generated [ModuleInitializer] for every archetype at assembly load — the barrier that replaces the manual Archetype<T>.Touch() startup calls. Runs the archetype's static ctor (declaring its Comp<T> components) then finalizes it and its parent chain. Idempotent — an already-finalized archetype is a no-op. Public because the generated registrar lives in the consumer's own assembly and cannot reach engine internals.

public static void RegisterArchetype(Type archetypeType)

Parameters

archetypeType Type

The [Archetype] class type to finalize.

RegisterByteMigration(string, int, int, int, int, ByteMigrationFunc)

Registers a byte-level migration function for scenarios where the old struct type is no longer available in code. Must be called before RegisterComponentFromAccessor<T>(ChangeSet, SchemaValidationMode) / RegisterComponentByType(Type, ChangeSet, SchemaValidationMode) for the target component.

public void RegisterByteMigration(string componentName, int fromRevision, int toRevision, int oldSize, int newSize, ByteMigrationFunc func)

Parameters

componentName string
fromRevision int
toRevision int
oldSize int
newSize int
func ByteMigrationFunc

RegisterComponentByType(Type, ChangeSet, SchemaValidationMode)

Non-generic entry point for registering a component when the type is only known at runtime — e.g. types discovered via reflection from a plugin or user-supplied schema DLL, as the Workbench does when loading *.schema.dll into a collectible AssemblyLoadContext.

public bool RegisterComponentByType(Type componentType, ChangeSet changeSet = null, SchemaValidationMode schemaValidation = SchemaValidationMode.Enforce)

Parameters

componentType Type

A closed unmanaged value type tagged with [Component]. The unmanaged constraint from the generic overload is verified at runtime by the CLR when the method is specialized; non-blittable or reference-type inputs will throw from deep inside MakeGenericMethod(params Type[]).

changeSet ChangeSet

Optional transactional change set. See RegisterComponentFromAccessor<T>(ChangeSet, SchemaValidationMode).

schemaValidation SchemaValidationMode

Schema validation policy (default: Enforce).

Returns

bool

Forwarded from RegisterComponentFromAccessor<T>(ChangeSet, SchemaValidationMode) — true on success.

Remarks

Internally invokes the generic RegisterComponentFromAccessor<T>(ChangeSet, SchemaValidationMode) via MakeGenericMethod(params Type[]). Any TargetInvocationException raised by reflection is unwrapped with ExceptionDispatchInfo so callers observe the real underlying exception — e.g. SchemaValidationException, SchemaDowngradeException, SchemaMigrationException — with its original stack trace preserved.

Exceptions

ArgumentNullException

componentType is null.

ArgumentException

componentType is not a closed value type.

See Also

RegisterComponentCollectionFactory<T>()

Registers the AOT-safe backing-store factory for a ComponentCollection<T> element type. Called by source-generated component schema providers (feature #514) for each collection field — the element type is a compile-time generic argument, so no MakeGenericType/Activator is needed. Idempotent.

public static void RegisterComponentCollectionFactory<T>() where T : unmanaged

Type Parameters

T

The collection's element type.

RegisterComponentFromAccessor<T>(ChangeSet, SchemaValidationMode)

Registers component type T with the engine: builds its schema definition from the component accessor, validates it against any persisted schema for the same component name, and creates or loads the backing ComponentTable.

public bool RegisterComponentFromAccessor<T>(ChangeSet changeSet = null, SchemaValidationMode schemaValidation = SchemaValidationMode.Enforce) where T : unmanaged

Parameters

changeSet ChangeSet

Optional change set to enlist the registration writes in.

schemaValidation SchemaValidationMode

How a persisted schema is reconciled with the runtime type; default Enforce.

Returns

bool

true on success; false when the component definition could not be built.

Type Parameters

T

A closed unmanaged value type tagged with [Component].

Remarks

When a persisted schema exists for this component, schemaValidation governs how differences are reconciled. A Transient component may not declare a ComponentCollection field — that combination is rejected at registration.

Exceptions

InvalidOperationException

A Transient component declares a ComponentCollection field.

RegisterMigration<TOld, TNew>(MigrationFunc<TOld, TNew>)

Registers a strongly-typed migration function that transforms component data from TOld to TNew. Both types must have [Component] attributes with the same Name but different Revisions. Must be called before RegisterComponentFromAccessor<T>(ChangeSet, SchemaValidationMode) / RegisterComponentByType(Type, ChangeSet, SchemaValidationMode) for the target component.

public void RegisterMigration<TOld, TNew>(MigrationFunc<TOld, TNew> func) where TOld : unmanaged where TNew : unmanaged

Parameters

func MigrationFunc<TOld, TNew>

Type Parameters

TOld
TNew

ResetPeaks()

Resets all high-water mark metrics (PeakBytes, MaxWaitUs, MaxUs, etc.) to current values.

public void ResetPeaks()

Remarks

Called by ResetAllPeaks() to enable windowed peak measurements. After displaying or exporting peak values, this method resets them so subsequent peaks represent new maxima since the last reset.

If this node has no high-water marks, implement as an empty method. Plain writes are sufficient — snapshots are taken every 1-5 seconds, so eventual visibility is acceptable.

RunStorageIntegrityCheck()

Audits storage-level invariants and returns every violation found. Pure read-only — touches the occupancy bitmap, the segment registry, and each segment's forward header chain; no data-page mutation, no allocation beyond a small issue list. Safe to call at any time on a live engine.

public StorageIntegrityReport RunStorageIntegrityCheck()

Returns

StorageIntegrityReport

Remarks

Two classes of check, each independent:

  • Popcount canary — the count of set bits in the occupancy bitmap must equal the sum of every registered segment's Pages.Length, plus each segment's directory-map extension pages (the pages outside the root that hold the page-index list when the segment owns more than 500 data pages — they are bit-set but not part of Pages), plus each directory page's CK-05 twin (the alternate slot, bit-set but in no segment list), plus the reserved root pages (0..InitialReservedPageCount-1), plus the occupancy-reserve pages (data, map-extension, and the map-extension twin) held by the page-allocator machinery. Any orphan (bit set, no claimant) or phantom (claimant, bit clear) is reported as a hard durability/structural bug.
  • Chunk-segment capacity — for every Typhon.Engine.Internals.ChunkBasedSegment<TStore>, AllocatedChunkCount + FreeChunkCount must equal ChunkCapacity. Desync indicates the segment's chunk free-list drifted from its on-page chunk bitmaps.

SetSpatialBarrierOnly<TArch>(bool)

Declare that TArch uses ClusterRef.WriteSpatial as the exclusive writer of its spatial component. Skips the fence-time legacy scan for this archetype — both DetectClusterMigrations's dirtyBits sweep and RecomputeDirtyClusterAabbs's ActiveClusterIds iteration are replaced with sparse iteration over Typhon.Engine.Internals.ArchetypeClusterState.ClusterProcessBitmap.

Only set when EVERY spatial-field write on this archetype goes through WriteSpatial. Mutations via raw GetSpan or OpenMut + Write will be invisible to the engine's spatial maintenance after this is enabled. The TYPHON009 analyzer flags non-WriteSpatial mutation sites — once those are zero, it's safe to opt in. 

public void SetSpatialBarrierOnly<TArch>(bool value = true) where TArch : Archetype<TArch>

Parameters

value bool

Type Parameters

TArch

TryReadPageBody(int, Span<byte>)

Copies the raw page (8 KiB — PageBaseHeader + metadata region + chunk data) of file page filePageIndex into dest. The read goes through the page cache via the no-clock-sweep path (RequestPageEpochNoSweep): it does not bump the eviction heuristic and does not satisfy the page's pending CRC verification — the caller recomputes the CRC itself. Returns false on an out-of-range index or a lost eviction race; true with dest filled otherwise.

public bool TryReadPageBody(int filePageIndex, Span<byte> dest)

Parameters

filePageIndex int
dest Span<byte>

Returns

bool

WriteTickFence(long, ChangeSet)

Serializes dirty SingleVersion component data to WAL at tick boundary. One TickFence chunk per SV ComponentTable. Called by the game loop at each tick boundary.

public long WriteTickFence(long tickNumber, ChangeSet changeSet = null)

Parameters

tickNumber long

Monotonic tick identifier.

changeSet ChangeSet

Caller-supplied ChangeSet for shared dirty-page tracking across the whole tick fence (typically the per-tick UoW's shared ChangeSet — see Typhon.Engine.UnitOfWork.ChangeSet). When null, a one-shot local ChangeSet is created and committed by this method itself (test/admin path: tests that invoke WriteTickFence directly without a UoW retain their original behaviour).

Returns

long

Highest LSN written, or 0 if nothing was serialized.

Events

OnMigrationProgress

Raised during schema migration to report progress to subscribers.

public event EventHandler<MigrationProgressEventArgs> OnMigrationProgress

Event Type

EventHandler<MigrationProgressEventArgs>