An AI brief layer over multi-project analytics.

01 Problem

Running several products at once creates a specific failure mode: the dashboards exist, but nobody reads them. Metrics are spread across projects, each with its own baseline, and the signal that matters — what actually changed since last time — is buried under charts that all look the same.

SoleOS consolidates those metrics into one place and adds a layer that writes the read: what moved, in which project, and whether it is worth attention.

02 Constraints

A written brief is only useful if it is trustworthy. That sets the binding constraint: the generated text must never assert a figure that is not present in the underlying aggregate. A brief that hallucinates a number once is worse than no brief at all, because it teaches the reader to re-check everything, which removes the entire benefit.

The second constraint is cost. A brief that runs across every project on every refresh is a recurring inference bill with no natural ceiling, so spend has to be bounded by design rather than discovered on an invoice.

03 Role

Sole engineer. Architecture, implementation, and ongoing operation — this is my own product, so I also own what happens when it misbehaves at an inconvenient hour.

04 Architecture

The system separates two responsibilities that are tempting to merge. Aggregation is deterministic: metrics are collected and reduced into a compact, typed summary per project — the deltas, the periods, the direction of movement. Only that reduced structure is passed to the model.

The language layer's job is narrow by design: turn a structured summary into readable prose. It is not asked to compute, compare, or infer anything from raw rows. Keeping arithmetic out of the model is the single decision that does most of the work, because a model cannot misreport a number it was never asked to derive.

05 Evaluation strategy

The property worth testing is groundedness: every figure that appears in the output should be traceable to the structured input. That is checkable without human review — the numbers in the generated text can be extracted and reconciled against the summary that produced them, and a mismatch is a hard failure rather than a style note.

This is the shape I recommend for most generative features. Pick the one property whose violation would destroy trust, make it mechanically checkable, and run it on every change. Broad subjective quality scoring can come later; the correctness gate cannot.

06 Cost & latency

Because only a reduced summary reaches the model, input size is bounded by the number of tracked projects rather than the volume of underlying events. That keeps per-brief cost predictable as the data grows, which is the difference between a feature that scales and one that quietly becomes the largest line item.

Briefs are generated on a cadence rather than on every page view, so reads are served from stored output. Interface latency is therefore a database read, not an inference call.

07 Failure modes

Three worth naming. A project reporting no data produces an empty summary, and a naive prompt will happily narrate nothing as though it were something — the empty case needs an explicit path. A provider outage must degrade to the last stored brief rather than an error state, since the dashboard is still useful without new prose. And a model change is a silent behavioural change: without the groundedness check running in CI, a swap can alter output quality with no visible signal.

08 What transfers

This is the pattern I bring to client work: keep deterministic computation out of the model, reduce before you generate, pick the one correctness property that matters and enforce it automatically, and bound cost structurally rather than by monitoring it after the fact.