Component storage and system queries

Examine typed columns, sparse row indexes, query caching, change tracking, schema adoption, and parallel field commits.

Asterune stores game-defined components in typed columns beside the instance arena. Component membership maps the arena's physical slots to dense rows. Queries cache the matching identities and row positions, while revision stamps separate structural changes from ordinary field updates.

This entity-component system (ECS) extends the instance model without replacing its hierarchy or built-in class state. A part still carries the engine's typed physical state. A game-defined component adds another record attached to that same instance identity.

Sparse membership and dense field columnsCopy link

Each component definition owns a set of rows. A sparse array maps an instance's arena slot to a row index, using an absent marker when the instance lacks that component. Dense arrays hold the corresponding instance IDs and arena slots. Every field column uses the same dense row positions.

Primitive fields use specialized vectors: numbers, integers, booleans, text, enum indices, vectors, transforms, and references have dedicated storage forms. Lists, maps, structs, optional values, and buffers use columns of whole values. The engine does not expand every nested collection member into its own column.

Removing a row swaps the final row into the vacant dense position. It performs the same swap in every field and tracking array, then repairs the moved instance's sparse entry. This keeps columns dense without shifting all later rows. It also means a row index is a storage position rather than a permanent entity identity.

Component handles retain identity and definition information rather than assuming a cached row remains valid indefinitely. Field access resolves the instance's present row. The binding caches definition and field-name lookup, invalidating those caches when the store identity, name revision, or definition allocation changes. A removed component produces a missing-membership result instead of reading a different instance's swapped row.

The store shares whole component sets between snapshots. A write makes the modified set private through copy-on-write. Unmodified sets remain shared. This differs from the instance arena's per-node sharing and from an archetype layout that moves an entity's entire collection of components together.

Query drivers and cached membershipCopy link

A query starts from required component membership or a required tag. It chooses the smallest available required component set or tag set as its driver. The remaining component, exclusion, tag, class, and ancestry conditions filter those candidates. A query without either kind of positive driver is rejected instead of scanning every instance implicitly.

For example, if one required component has 20,000 rows and another has 200, the smaller set supplies the candidates. The query tests those 200 identities against the other conditions. A required tag with fewer members can supply an even smaller driver. These counts determine candidate enumeration, not the final number of matching rows.

The resulting cache stores matching IDs and the dense row positions for each term. Matches are sorted by instance ID, giving the cached iteration a stable identity order independent of the columns' swap-removal order. An ancestor term additionally records the nearest ancestor carrying that component.

A cache stamp includes the store identity and structural revisions for the component sets that affect membership. Tag-filtered queries also include the tag revision. Queries constrained by ancestry include topology revision. If the stamp matches, the query reuses its membership and row cache.

A field write does not change component membership, so it does not by itself rebuild that cache. Adding or removing a row does: swap-removal can change the row position of another matching instance even when that instance's fields stay unchanged. Replacing the store also invalidates the cached mapping.

Luau iteration maintains a second mirror of instance values and component handles in VM tables. It rebuilds that mirror when store identity or query generation changes, reusing surviving instance wrappers when possible. A Luau iterator then walks those tables, with specialized paths for up to 3 component terms. Field access still reads live component state; this mirror caches handles rather than copying every field into an independent Lua table.

Structural stamps, change ticks, and removalsCopy link

Each component row records when it was added and when a field last changed. The store advances its mutation tick for actual field changes. Assigning an equal value does not create another changed tick. A per-row field mask records which columns need outgoing dirty-state processing.

A query with added or changed filters captures the store tick at the beginning of iteration. Its next iteration compares row ticks with that captured point. Writes made after the iteration began remain eligible for the following run. Each query keeps its own baseline; another consumer does not consume or clear that query's change window.

Multiple change filters are conjunctive. If a query requires changed state in 2 component terms, both row timestamps must exceed its baseline. The filter checks timestamps over cached membership rather than rebuilding the complete match set for each value change.

Removing a row also appends its identity and tick to the component's removal log. A removed-reader cursor is independent of ordinary added or changed queries. Returned identities are sorted and deduplicated, so several removal records do not imply several distinct entities.

The log retains at most 16,384 records before dropping its older half. A reader whose cursor precedes the retained floor receives an explicit missed-removals condition. It must reconstruct its dependent state rather than treating the remaining suffix as a complete history. Revision tracking therefore supports incremental consumers without promising an unbounded event journal.

Schema evaluation and row adoptionCopy link

A ComponentDefinition is evaluated as data in a restricted Luau environment. That environment includes field constructors and selected value operations, with an 8 MiB memory budget. It does not expose the live DataModel or the ordinary game-service surface. The engine caches evaluation results by source hash, including failed evaluations, so unchanged definitions do not execute again during every synchronization pass.

Definition synchronization tracks topology and the definition-family revision. When those indicate a possible change, it compares the relevant names and source fingerprints before rebuilding typed state. The component's name comes from its definition instance; the schema does not independently assign a competing name.

Saved rows can arrive before their definition. The store retains their raw records as orphans rather than interpreting unknown fields as arbitrary typed columns. When a compatible definition becomes available, adoption constructs the declared columns and moves readable rows into typed storage. A definition that excludes the instance's class leaves that row unadopted.

Redefinition converts existing typed rows and orphan rows through the same adoption process. It matches fields by their declared name, then by recognized former names. Missing fields use their defaults; values that cannot convert reset to defaults; unrecognized fields are dropped from the adopted typed record. The adoption report distinguishes those outcomes.

A row saved under an older schema version also retains its preceding values as pending migration input. Default adoption establishes a usable typed row, while a migration can inspect that saved version and data. This separation keeps schema interpretation distinct from the game-specific transformation of older state.

Reference indexes and lifecycle policiesCopy link

Direct reference columns maintain a reverse index from target identity to holder, component, and field. Replacing a reference removes its old index entry and adds the new one. Destruction can then find affected holders without scanning every stored record.

The schema's destruction policy decides whether to clear a reference, remove the referring component and its dependents, or destroy the holder. Applying a policy checks that the holder is still live because an earlier action can already have removed it. Observed components queue change events when clearing a reference changes the field.

Cloning and grafting also remap component references alongside built-in and attribute references. The store retains the necessary representation for unadopted raw rows, so reference rewriting is not limited to rows whose schema has already loaded. The surrounding instance lifecycle is described in Instance state and lifetime.

Parallel field overlaysCopy link

Parallel component writes record intended field values instead of mutating the shared columns immediately. Each executing writer owns an overlay indexed by component, row, and field. Repeated writes by that writer replace its previous recorded value and update the local sequence number.

A read by the same writer first checks its overlay, so it observes its own latest write. Another writer reads the phase's established component state, not that uncommitted overlay. This holds even when the writers happen to run one after another on the same lane. Lane placement does not silently create cross-writer read-after-write visibility.

After all lanes join, the engine sorts writes by writer rank and sequence. Execution-group callbacks use hierarchy-derived group ranks; Systems use their scheduled positions in a later writer tier. Applying that ordered list makes the final field value independent of which CPU worker happened to finish first. Later applied writes replace earlier values when writers target the same field.

The engine records such conflicts and checks whether the participating Systems have a declared ordering relationship. A reproducible commit order does not establish that competing updates were intentional. Conflict reporting lets the schedule expose that ambiguity rather than treating the last value as proof of a valid dependency.

Each recorded write carries the instance identity as well as its cached row. If that row no longer identifies the same live instance, application falls back to the checked field setter. An invalid target causes a reported refusal. Structural operations, including adding or removing component rows, stay in serial execution. The overlay is a field-update mechanism, rather than a transactional replacement for arbitrary instance mutation.

System order and measured accessCopy link

The System scheduler builds its dependency graph from registered Systems and named sets. It first establishes hierarchy order for otherwise unordered candidates, then expands before and after relationships into graph edges. Cross-phase edges are unnecessary because the phases already establish order; missing or nonrunning targets produce diagnostics rather than invented nodes.

Strongly connected groups reveal dependency cycles. The scheduler stops Systems in those cycles and reports the chain, while other schedulable Systems remain available. A topological pass establishes each phase's final order and records reachability for subsequent conflict checks.

An interval System accumulates phase time until it is due. It runs once with the accumulated duration and clears that accumulator, rather than running once for every interval missed by a long frame. Fixed-phase callbacks instead participate in the engine's fixed block. Their phase determines the cadence that supplies elapsed time.

During execution, component reads, field writes, visited rows, duration, and errors contribute to System statistics. Access tracking records the work that actually ran. It supports diagnostics and conflict detection; it does not infer a new dependency graph that automatically makes game logic correct.

The component guide covers schemas and query syntax. Systems and scheduling covers System definitions, phase options, and schedule inspection.