Skip to content
MSO CloudDocumentation

The decision engine

On this page

In short#

The decision engine turns an organization's own signals into ranked, recorded proposals that a person approves, carries out and later confirms. It abstains when the evidence cannot support a number, and never executes an action.

1 · CollectDated signalsnapshot2 · DetectNumeric rules,three-valued3 · ExplainThe receipt asone sentence4 · ProposeRank thematched set5 · GuardrailBlocking checksbefore approval6 · ApproveA person, witha stored reason7 · ExecuteA person, outsidethe platform8 · MeasureDeclared conversion,declared horizon9 · ConfirmOwner ratifiesor overrides10 · LearnRebuild thetrack recordconfirmed outcomes change the rankBlue: machine. Amber: owner or org admin. Only an owner-confirmed outcome reaches Learn; a machine verdict is never learned from.

Selecting a playbook is a numeric rule evaluation; trusting a playbook or lever is statistics over confirmed results. Neither uses a language model or semantic similarity. How far each number can be trusted is graded on the evidence ladder. Upstream: From signal to decision. Outcomes: Measurement and honest numbers. Screens: Decision engine screens.

Rule selection#

Numeric predicates compare an observed signal with a threshold. Evaluation is three-valued and unknown never matches, so a missing signal fails closed. Each selection emits a per-predicate receipt (predicate, observed value, threshold) re-derivable from stored inputs; display text is not part of the rule. Grammar: Marketing playbooks.

Off-take statistics#

On the commerce lever plane one pure module holds the statistics, as scaled integers, so no surface re-derives them. The forecast is a regression with confidence bands, not a trained model.

Method What it is
Correlation Lagged partial-rank correlation of a lever with GMV, controlled for campaigns, winsorized inputs
Forecast Least-squares regression on lever, campaign and season; cold-start prior; walk-forward evaluation
Confidence Correlation strength weighted by the amount of evidence, in basis points

Abstain gates#

Each gate returns a named state instead of a number; downstream inherits it, so no recommendation is produced.

Condition Result
Fewer than twenty weeks of history learning
Correlation below the reliability threshold no reliable signal
A lever that never varied Nothing: a constant carries no information
Recommended discount above the cap Rejected by the guardrail
No data from the declared conversion feed Recorded as learning, with the reason
Diagnostic scan or off-policy evaluation below floor Abstains with the floor named
No lever with confirmed positive lift cold start: no lever is recommended

Policy learner and off-policy evaluation#

Each lever's normal posterior starts at 0 bp and folds the measured lift of owner-confirmed, treatment-arm decisions only; a lever is served once it has at least three observations and a mean above zero. Selection is deterministic via a seeded draw (no clock, no ambient randomness), and the ledger records the true probability: one for the chosen action, zero for the rest. A randomized selector with a replayable draw is tested; no production path calls it. Off-policy evaluation reports IPS, self-normalized IPS and doubly-robust estimates with intervals and an effective-sample-size check, or refuses with a named reason (deterministic_log, zero_overlap, ess_below_floor). On deterministic logs it refuses, so no policy is promoted on an off-policy figure; a decision enters the estimator only with an in-range recorded probability, a real variance and a known lever (learned policy on Project status).

Guardrails and approval#

Blocking guardrails run before display: a hard discount cap, an objective-against-cap check and the off-policy floor. They gate; they do not simulate. Each mutation checks owner or org admin on the stored row and claims it with a status-predicated update before any command is written, so a repeated approve inserts nothing. One decision in ten, chosen by a hash of the decision, is held out as control: its commands are stored as held out, the board tells the owner not to run it by hand, its outcome abstains, and confirming it or approving its lesson is refused. An approved action becomes manual steps; the engine writes to no marketplace, ad account or channel. A playbook started by an alert or another system trigger proposes as a viewer, so each of its steps waits for a person.

The episode ledger#

Each lifecycle event is one immutable row. Writes are idempotent per organization and brand, so a re-run scan writes nothing new.

Evaluatedeligible set snapshotProposeda decision rowApprovedowner said yesActivatedexecution startedMeasuredmachine verdictConfirmedofficial verdictapprove and execute in one stepAbstainednamed reason, no cardRejectedwith a stored reasonCancelledbefore any outcomeback to the inboxLesson confirmedowner blesses the textLearntrack record, memoryonly from Confirmeddisplay onlyWho writes which rowBlue: the worker writes evaluated, proposed, measured, abstained. Amber: an owner or org admin, through the decision router, with an ownership check.Every event is one append-only ledger row. A policy promotion or rollback is a row too (policy_promoted, policy_rolled_back) and carries the approving identity.
Episode states. Blue arrows are written by the worker; amber arrows need an owner or org admin and an audit-log row. Only the amber 'Confirmed' event is evidence.
Field Records
Eligible set Every candidate, its three-valued outcome and its rank in the deterministic order
Chosen action Its true probability and the logging policy's name and version; probabilities sum to one
Reward window Conversion, days and a late cutoff three days after close, declared before any outcome
Version pins Engine and rule version, a hash of the signals seen, a hash of the receipt
Abstention reason Learning, no reliable signal, missing signal, below floor, estimate, no match, no data

Replay#

Replay re-runs the pinned rule version on the frozen signal values and compares receipt hashes. Where the episode pins the champion policy that ranked it, replay re-ranks the candidates with that configuration and reports reproduced or divergent; an episode ranked with the defaults reports unpinned.

Verdict Meaning
Identical The receipt re-derives exactly
Divergent Names the leaf and field that differ: signal, operator, threshold, observed value, outcome
Unreplayable Abstained, no rule version, version gone, or another engine version; not a pass

Memory#

The card cites what happened the last time a rule matched. Memory is derived at read time from confirmed outcomes, so no separate store can drift. Relevance is lexical, not embedding similarity, and memory never changes which rule fires.

Aspect Rule
Eligible Owner-confirmed hit, partial or miss; same playbook; last year; not superseded; not sample data
Ranked by Specificity (entity, segment, rule), recency halving in about five weeks, a bonus for a lesson
Supersession Same rule, segment and period: the later confirmation wins and names its predecessor
Shown Three citations by default; nothing when nothing is confirmed

What it reads and what it never reads#

Direction Scope
Reads First-party signals, the declared conversion, the playbook corpus, owner confirmations
Never reads as evidence Third-party market estimates; a diagnostic may cite one as a hypothesis (Data planes)
Never writes Approvals, activations, rejections or confirmations from a machine path; anything outside the platform

Where it stops#

  • No trained machine-learning model runs in production; prediction is regression on the organization's own history.
  • Intervention readouts are observational: a before-and-after readout is disclosed as confounded and is directional evidence. The measured lift compares each decision with its own earlier window and is not yet adjusted by the control arm. Geo designs are out of scope.
  • Memory has expiry and supersession but no deletion audit.
  • There is no trade-off simulator. A randomized logged selection, a data watermark writer and a causal compute service are not built; dated status per capability is on Project status.

Decisions#

Decision Why Enforced by
Every lifecycle event is one row in an append-only ledger A field not captured at decision time can never be added honestly afterwards An update-and-truncate trigger, no runtime delete grant, verify:data checking probabilities
A person approves before anything is activated; the worker proposes and measures A loop that approves its own proposals, or learns from its own verdicts, grades itself Router ownership checks, an audit-log entry per mutation, confirmed-only track record
Every selection receipt can be replayed from frozen inputs An explanation that cannot be re-derived is a story The replay build gate: zero divergences to pass, "skipped" (not "passed") when nothing replays

Sources#