Resources

Federation and materialization

The right access strategy depends on the workload. Examine where computation happens, how current the result must be and who carries the operational cost.

Source queries and retained views at 09:20

Order 2042 · 24 September 2026 · UTC

At 09:12 the source changed from Picked v7 to Dispatched v8.

Query the source

Read the permitted source view when the request arrives.

Source observed09:20
Returned evidenceDispatched v8

Results or intermediate data may move. The source and query plan determine where computation occurs.

Read a retained view

Read the result of the last successful refresh.

Source observed09:00
Returned evidencePicked v7

The evidence is 20 minutes old. Reading this stored result does not refresh it.

What if the source is unavailable at 09:20?

Source query: no fresh result until the source recovers. Any cached response needs its own age and acceptance policy.

Retained view: a healthy projection store can still return Picked v7. It cannot confirm whether the order has since been dispatched.

Both paths need current authorization. The projection owner defines acceptable age, refresh recovery and how source deletions reach retained copies. Observation time determines the age of the evidence returned.

Start with the workload and its limits

Federation queries information through an access layer while data remains governed by its source architecture. Materialization persists selected data or computed results for subsequent use. Neither choice is automatically superior. A sporadic investigation, an executive dashboard and a latency sensitive operational screen place different demands on source systems, network paths and the acceptable age of a result.

Describe representative queries before comparing technologies. Record expected concurrency, result size, join complexity, freshness requirements and the impact of slow responses. Identify which sources are transactional systems with protected operating windows. A technically possible query may still be an inappropriate workload for a production application database during a financial close or a manufacturing shift.

Inspect where execution actually happens

A federated query can be efficient when filters, projections or aggregations execute close to the data. Trino documents pushdown optimizations and the fact that support depends on connectors and query conditions. A filter that appears early in SQL is not evidence that the remote source executed it. Inspect the query plan and measure transferred data using representative predicates.

Cross source joins deserve particular attention. The engine may need to move intermediate results before completing the join, and estimates may be imperfect. Test skewed keys, selective filters and large intermediate sets. Measure source load as well as response latency. A query can appear convenient to one analyst while consuming resources that belong to a critical operational workload.

Define freshness as a business requirement

Persisted results can improve predictable access, but the refresh process creates a freshness responsibility. PostgreSQL materialized views retain query results and update them through a refresh operation. The relevant business question is how an older result affects a decision, not whether a platform uses the word real time. Some reporting tolerates a daily refresh; a stock allocation decision may not.

Publish the observation time and freshness expectation with the data product. Distinguish a normal refresh interval from an actual refresh failure. When a result exceeds its limit, decide whether consumers should see a warning, receive partial information or stop a dependent action. This behavior should be designed with the business owner rather than left to an incidental cache setting.

Follow policy across every copy and cache

Querying data in place can reduce selected copying requirements, but it does not mean that data never moves. Results, intermediate buffers, exports and caches can all contain sensitive information. Trace those locations and establish retention and access controls. Verify whether the query identity is propagated to the source or whether a shared service identity changes the effective authorization model.

Materialized products need explicit ownership of policy propagation. If source access is revoked or a record must be removed, specify how derived stores respond and how completion is verified. Document lineage from source fields to persisted outputs. Compare the governance burden of both strategies in the actual deployment rather than assuming that either automatically satisfies a regulatory obligation.

Evaluate failure and recovery behavior

Federated access inherits dependencies on source availability, connectors and network paths. A source outage may prevent a result even when the query layer itself is healthy. A persisted product can remain available during an outage, but its age continues to increase. Decide whether a usable older result is acceptable and make that condition visible to the people relying on it.

Test timeouts, source maintenance, credential rotation and partial refresh failure. Establish whether a failed refresh leaves the previous complete version available or exposes incomplete data. Record recovery objectives for the product and the dependencies required to meet them. Availability should describe the business result a user can obtain, not merely whether a query endpoint returns a response.

Combine access patterns by workload

An architecture can combine both approaches. It may query a source for detailed investigation, persist a governed summary for repeated analysis and consume events for selected operational changes. The boundaries should follow workload evidence. Avoid duplicating the same data through several paths unless there is a clear reason and a reconciliation strategy for inconsistent observations.

Maintain a decision record for each important data product: chosen access mode, acceptable age, source impact, policy owner, recovery behavior and review trigger. Revisit the choice when usage or source constraints change.

Order 2042 observation paths comparing a source read at 09:20 returning Dispatched v8 with a retained view from 09:00 returning Picked v7.
The same order request returns evidence of different ages, depending on the observation path.