Grain Topology & Scaling

Design page

This page describes the design of the server’s batch signal pipeline. It is not the default path in current releases. The pipeline runs only where it is enabled for a deployment; by default, devices update per tag and live views reach the browser as wire patches (see Multi-renderer UI). Its per-session coalescing stage is not yet in use. The scale properties below are design goals, not measured results. Its acquisition stages (connection, planning and sharding) are planned to move into an acquisition core shared with the edge. The cluster topology section at the end of the page describes how current releases run.

The batch pipeline is designed around what it deliberately does not do: it does not allocate a grain per tag. In the design, grain activation count scales with infrastructure — controllers, shards, devices, sessions — not with the number of tags being polled, so two sites with the same equipment activate roughly the same number of grains whether they poll a few thousand tags or many more.

This page describes the stages of that pipeline, how a batch flows from controller to browser, and why the activation budget is designed to stay flat as tag counts grow.

Note

This is the server-side pipeline. It reads Logix-family (ControlLogix) controllers directly from a site server over EtherNet/IP, read-only and without an edge, using the same driver library the edge uses. This is in early access: our engineers enable it per controller, and it has run at a site trial since 28 October 2026. Off by default. A cloud cell never reads plant controllers: site and edge data reach it through synchronisation. The edge path (an edge runtime that reads its controllers and streams values to the site server over gRPC, with store-and-forward) is described under PLC communication. Edge-streamed values currently pass through to their devices without this pipeline’s batch evaluation; bringing them fully onto it is roadmap. See Architecture overview for where the server tier sits.

Why per-tag grains do not scale

The naive approach — one grain per tag — does not hold at industrial scale. A large site carries a very large number of tags; a grain per tag means an activation, a timer and per-tag persistence churn for every one of them. Every UI update would also flow per tag through the system with no coalescing, loading the SignalR transport.

The batch pipeline is designed to replace per-tag work with batch envelopes. Reads are planned into contiguous blocks, executed by a small fixed set of shard grains, and emitted as a single batch carrying many tag updates. Each downstream device evaluates once per batch, not once per tag, so grain count and message volume track the equipment rather than the tag count.

Pipeline stages

In the design, the path from controller to browser passes through five grain roles. Each is keyed so that activation count is bounded by physical or configured topology, never by tag count.

Grain role	Count	Scales with	Responsibility
Controller connection	1 per controller	Controllers	Connection lifecycle, health, dispatches read plans
Read planner	1 per controller	Controllers	Groups the controller’s tags into read blocks; emits a versioned read plan
Controller shard	Configured per controller	Controllers × shards	Executes block reads; holds shard-local current values; emits a batch
Device	1 per device instance	Devices	Evaluates a batch once; emits the signal changes
Session publisher (planned)	1 per session	Sessions	Planned: subscribes to device grains and coalesces changes per tick into one SignalR message

Read planning and sharding

The read planner takes the full tag map for a controller and computes a read plan — a set of read blocks distributed across the controller’s shards. Each replan produces a new plan version, which is distributed to the shards; a shard applies a plan only when it is newer than the one it holds.

The shard count is fixed per controller at configuration time, so it is bounded by topology rather than tag count. Each shard keeps an in-memory current-value dictionary, polls its assigned blocks, diffs against the dictionary, and emits only the changed tags as a batch. There are no per-tag timers and no per-tag persistence.

Batch evaluation, once per device

The critical rule is that a device grain runs its evaluation graph exactly once per batch, regardless of how many tags changed. On receiving a batch it filters to the tags this device cares about (a pre-computed set from its binding config); if nothing relevant changed it returns with no cost. Otherwise it applies the relevant updates and evaluates state-machine transitions and alarm conditions in a single pass, then emits changes for only the signals whose values changed. Interlocks are not evaluated here: they stay in the controller’s own logic.

If a batch carries 200 tag updates and only 3 are relevant to a given pump, the pump evaluates once with those 3 updates applied — not three times, and not 200 times.

Fanout and coalescing

This stage is planned and not yet in use; live views reach the browser as wire patches today (see Multi-renderer UI). In the design, signal changes reach sessions by direct subscription: each session publisher registers with the relevant device grains and is notified when signals change.

Each browser session would have one session publisher. It accumulates incoming deltas and, on a coalescing tick, emits a single update per tick — one SignalR message — which the browser’s wire renderer applies as wire patches.

Advisory-only AI

Operator commands do not travel through this read pipeline. They take the platform’s separate, audited command path, which applies the site’s command policy and checks the command’s preconditions before anything is sent. AI never participates in that command path — it is advisory-only: it reads plant state, explains it and recommends; it never commands equipment. Hard interlocks live in deterministic controller logic and hardwired safety relays. AI tooling that browses tags does so read-only. See AI architecture.

Data flow

The diagram shows the design; the session publisher stage is planned.

flowchart TD
  HW[Controller<br/>ControlLogix over EtherNet/IP]
  CONN[Controller connection<br/>1 per controller]
  PLAN[Read planner<br/>emits a read plan]
  SHARD[Controller shards<br/>block reads, current values]
  DEV[Device grain<br/>evaluate once per batch]
  PUB[Session publisher, planned<br/>coalesce per tick]
  HIST[Historian<br/>PostgreSQL, synced to cloud]
  UI["Wire patches over SignalR<br/>browser wire renderer · CLI"]

  HW -->|EtherNet/IP CIP| CONN
  PLAN -->|read plan| CONN
  CONN -->|shard assignments| SHARD
  SHARD -->|batch: value, ts, quality| DEV
  DEV -->|signal changes| PUB
  DEV -.samples, alarm events.-> HIST
  PUB -->|one message per tick| UI

The historian leg

The browser is not the only consumer. In the design, a device whose binding config names a historian also feeds it: the device grain maps its signal changes to samples (and alarm transitions to events) and emits them fire-and-forget — never blocking the live path — to its own historian grain (one per device), which persists to PostgreSQL. Like the rest of the pipeline, this leg is designed to scale with devices and historians, not tags.

Separately from this pipeline, a site server linked to the cloud pushes its historian samples up to the cloud tier in blocks on a schedule (every five minutes by default). The push follows a cursor, so delivery is ordered and duplicate-safe.

Quality travels with the data

In the design, every value carries a quality flag, set where it is acquired and carried through device evaluation, so the UI can show how far each value can be trusted:

Quality	Meaning
Good	Freshly read and confirmed
Uncertain	Value validity in doubt — for example a sensor in manual override
Cached	Loaded from a snapshot, not yet refreshed by a live read
Bad	Read failure or comms loss
Stale	Exceeded its max-age threshold

A value recovered from a snapshot rather than a live read is marked Cached until a fresh read confirms it, so an operator can tell a live value from a recovered one.

Activation budget — infrastructure, not tags

Because reads are sharded per controller and each device evaluates once per batch, the design sets the number of active grains by the site’s infrastructure — its controllers, devices, and sessions — not by how many tags are polled. Adding tags to an existing device adds no grains and no timers; the activation budget grows only when you add controllers, devices, or connected sessions. With per-session coalescing (planned), SignalR traffic would be bounded by sessions × tick rate rather than by tag-changes-per-second. The shape is the same from a single-controller demo to a large site; only the counts move, and they move with infrastructure.

Observability

Where the pipeline is enabled, its stages report metrics aggregated per site every five seconds: per-controller read cycle time, tags polled and batches emitted, batches and signal changes per second, and the total Orleans activation count. These are the measurements against which the design goal — activation count linear with infrastructure, not tags — is to be checked. Per-device evaluation timing and per-session SignalR metrics are not yet reported.

Cluster topology: multi-silo in the cloud, single-silo by default on-site

Cloud data cells run as multi-silo Orleans clusters — several server replicas forming one cluster, with cluster membership held in the cell’s PostgreSQL. A site server, by contrast, typically runs as a single silo: an on-prem plant does not need cluster-scale compute, and one silo is the simplest thing to operate offline. The same code runs both shapes, and grains are written to be multi-silo-correct so a deployment scales out without a rewrite.

Cross-server delivery uses an Orleans affinity relay, not a SignalR backplane. A browser’s WebSocket terminates on one silo, but the grain that produces its updates can be placed on any silo; the producing grain hops its push to a relay pinned on the silo that holds the connection, which delivers through that silo’s local hub. There is no Redis or hosted SignalR service in the path — the relay is the delivery mechanism on both cloud and on-prem.

Two design rules keep grains correct across silos:

• No authoritative in-memory-only state. A grain that holds data in memory is either fully persistent (Orleans persistent state or write-through to PostgreSQL) or regenerable from the database, so a grain rebalancing across silos never loses state.
• Clock synchronisation. Orleans reminders coordinate through the cluster’s reminder table, so NTP synchronisation is a deployment prerequisite for multi-silo clusters.

Each site server runs its own Orleans cluster with hard isolation by design; there are no grain calls between site servers, and a site server synchronises to the cloud tier only through dedicated sync grains. See Tenant isolation and the Architecture overview.

Related

• Architecture overview — where the server tier and grain runtime sit in the deployment model
• Multi-renderer UI — how live views reach the browser as wire patches over SignalR
• PLC communication — the edge-side controller path
• AI architecture — the advisory-only model that bounds AI to non-control roles