Resources, Live Data, and History

TL;DR

A resource is a named data contract that tells clients what a running strategy publishes, how it delivers data, and whether history is available. The resource catalog is the single source of truth — clients discover keys from the catalog rather than guessing them from display labels.

Why resources exist

A strategy declares typed topics at compile time:

on(TOPIC<"bar://okx/BTCUSDT@1m">, handler);

This topic is internal to the process. An external client — Terminal, an API integration, a monitoring script — has no access to the C++ type system. It needs a runtime description of what data is available, what shape it has, and how to consume it.

The resource catalog bridges this gap. When your strategy is running, the SDK publishes a catalog entry for every resource the strategy exposes. That catalog is available through the Forge API and the runtime WebSocket. Clients use it to discover exact keys, schemas, delivery models, and history availability.

Note

Think of TOPIC<...> as the internal compile-time identity and the resource catalog as the external runtime identity of the same data. They describe the same thing from different perspectives.

Resource identity

Every live resource has a canonical key. The key encodes the resource family, venue, instrument, and parameters in a fixed format:

bar://okx/BTCUSDT@1m
book://okx/BTCUSDT@full
factor://spread_mean/btcusdt.okx@bar:1m
order://paper/filled
scalar://signal-watch/confidence

Warning

Keys are case-sensitive at the authoring boundary and normalized by the runtime. External clients must use the catalog’s returned key verbatim. Do not derive a resource key from a Terminal label, a display name, or a venue channel naming convention. Labels change; keys are the contract.

The key format is designed to be self-describing: the scheme identifies the resource family, the authority identifies the source (venue or strategy name), and the path encodes the specific instrument and parameters.

Contract dimensions

Each catalog entry answers a set of independent questions. A client that understands the contract does not need to guess how to consume a resource.

Field

Question answered

model

What domain value does this resource represent? (Bar, Book, Factor, Order, Trade, Account, Portfolio)

payloadSchema

What physical payload is delivered on the wire?

deliveryModel

Latest value, snapshot plus deltas, append events, query snapshot, or in-process signal?

historyModel

Is it live-only, absent from history, or backed by a durable model?

replayPolicy

What initial state can a new subscriber receive? (None, latest, snapshot from history)

mergeModel

How should multiple frames form one client view? (Replace, upsert by key, append, snapshot + ordered deltas)

dropPolicy

Can updates be coalesced or resynchronized, or must none be dropped?

visibility

Is the data public market, private account, derived, or internal?

Why each field matters:

  • deliveryModel: Tells you whether to expect a stream of values (bars, factors) or a snapshot-then-deltas pattern (order books). Treating a book update as a bar value will produce nonsense.

  • historyModel: Tells you whether to query history for this resource. A resource can be live-only by design — querying history for it will always return empty, and that is correct behavior.

  • mergeModel: Tells you how to combine multiple frames into a single view. Upsert-by-key is correct for bars; append is correct for trades. Using the wrong merge model produces duplicate or missing data.

  • dropPolicy: Tells you whether the runtime can coalesce rapid updates. Lossless resources must not be dropped; bars and factors can be.

Tip

When debugging a data display issue in Terminal, check the catalog contract before checking the data. A resource with deliveryModel: SnapshotAndDeltas that is missing a snapshot will appear empty even if updates are arriving.

A worked example

Here is a bar topic, the resource it generates, and how Terminal consumes it.

Step 1: Strategy declares the topic.

bind<Datafeed>(venue::okx::Bar("BTCUSDT", "1m"));

on(TOPIC<"bar://okx/BTCUSDT@1m">,
   [](const Tick<schema::BarRow>& tick) {
  // Process bar.
});

Step 2: SDK publishes a catalog entry.

The runtime resource catalog includes an entry approximately like this:

{
  "key": "bar://okx/BTCUSDT@1m",
  "contract": {
    "model": "BAR",
    "payloadSchema": "bar",
    "deliveryModel": "LATEST_VALUE",
    "historyModel": "DURABLE",
    "replayPolicy": "LATEST",
    "mergeModel": "UPSERT_BY_KEY",
    "dropPolicy": "COALESCE",
    "visibility": "PUBLIC"
  }
}

Step 3: Terminal discovers and consumes.

Terminal fetches GET /runtime/resources?instance=<id>, reads the contract, and decides how to render the data:

  • deliveryModel: LATEST_VALUE → subscribe for live updates, no snapshot needed.

  • historyModel: DURABLE → backfill the chart with historical bars from GET /history/instances/{id}/market/bars.

  • mergeModel: UPSERT_BY_KEY → use the bar timestamp as the merge key; each new bar replaces the previous bar at the same timestamp.

  • dropPolicy: COALESCE → the runtime may skip intermediate values during backpressure; the chart does not need every tick.

This is the core loop for every resource in the system. The contract is the algorithm for correct consumption.

Live delivery

Different delivery models require different client behavior:

Delivery model

Client behavior

Example resource

Latest value

Subscribe, render each value as it arrives.

Bars, factors, scalars

Snapshot + deltas

Apply snapshot, then ordered deltas. On gap: RESYNC.

Order books

Append events

Append each event to the local log. Do not drop.

Trades, order events

Query snapshot

Poll for current state. No streaming.

Account balances

A generic WebSocket consumer must branch on the contract. Treating every frame as a chart point works for bars and factors but silently corrupts book and trade data.

History is separate

A resource appearing in the live catalog does not imply persisted rows. Always check historyModel before querying history:

  • DURABLE: history is stored and queryable. Use the history endpoint.

  • LIVE_ONLY: the resource is intentionally not persisted. History queries will return empty — this is correct.

  • ABSENT: no history model is declared. Do not query.

History is also run-scoped. The same instance can have multiple starts with distinct run_id values. Restarting the same project version creates a new run. Use an exact run_id in exported reports and automation. current is convenient for interactive use but can change between requests.

Warning

current is a moving pointer. If you query /history/instances/{id}/summary without a run_id, you get the latest run. If the project restarts between your discovery call and your data call, you may get a different run’s data. Automation must store the resolved run_id from the discovery step.

Availability states

These states answer different questions. No one state implies the others:

State

What it means

declared

The runtime described the resource in its catalog. The resource exists as a concept.

active

The resource has recent live publication evidence. Data is flowing.

persisted

Matching durable rows exist in ClickHouse for the selected run. History queries will return data.

available capability

The runtime supports the corresponding query or stream family. The infrastructure can serve this resource type.

Terminal keeps these distinctions visible in Inspect, Data Store, and Debug. A resource that is declared but not active may simply be waiting for the next bar interval. A resource that is active but not persisted may be live-only by design. A capability that is absent may be the correct contract for that project (a factor server with no ledger).

Next steps