Skip to content
MSO CloudDocumentation

Event contracts

On this page

Every event and record shape that crosses a process or system boundary, with where it is written and read, what identifies it, and what lineage it keeps. Five shapes exist in the code. The one ingest envelope at the end (ADR 0060 seam S4, docs/adr/0060-self-learning-seams.md) is built: the webhook ingest route and the eventbus carry it; connector streams, the realtime feed and platform events do not yet.

Contracts#

Contract Schema Transport Event id Version field
Connector stream records STREAM_SCHEMAS packages/connectors/core/src/mapping/streams.ts:836 Postgres: raw_events (bronze), import_staging_rows (staging) raw_events.id / import_staging_rows.id none; "adding a new field is a release" (streams.ts:16-19)
Marketplace webhook payload webhookPayloadSchema apps/web/src/app/api/ingest/webhook/[connectorType]/[connectorId]/route.ts:24-42, .passthrough(); envelope :157 HTTP POST into raw_events, then a job_outbox row envelope id (derived); raw_events.id envelope schema_version
Eventbus topics TOPICS packages/eventbus/src/bus.ts:16-47; EventEnvelope interface :55-69 Redis Streams yng:bus:<topic>.<orgId> or yng:bus:<topic> envelope id (randomUUID, bus.ts:167) envelope schema_version
Realtime ops feed BaseEventSchema packages/eventbus/src/realtime.ts, imported by apps/worker/src/events-bus.ts and apps/web/src/server/routers/subscription.ts Redis Stream yng:events:<orgId> Redis entry id only (events-bus.ts:12) none
Platform events platformEventSchema packages/platform-contract/src/webhooks.ts:66-72 platform_outbox row, then signed HTTP POST envelope id (webhooks.ts:13-17) none

Connector stream records#

Canonical post-transform shape per target stream; 29 streams in TARGET_STREAMS (packages/connectors/core/src/mapping/spec.ts:14). Bronze raw_events holds orders, creators, inventory; every other stream lands in import_staging_rows (§17).

Field or step Bronze raw_events Staging import_staging_rows
Table packages/schema/src/tables/data-lake.ts:33-69 packages/schema/src/tables/imports.ts:92
Payload payload jsonb, verbatim (data-lake.ts:45) source_payload verbatim, normalized_payload post-transform (imports.ts:110-111)
Idempotency key unique (org_id, connector_type, stream_name, external_id, payload_hash) (data-lake.ts:61-67) dedupe_key, non-unique index (imports.ts:127-132)
payload_hash sha256 of JSON.stringify(payload, sortedTopLevelKeys) (packages/connectors/core/src/bronze-util.ts:9-15) n/a
Batch lineage source_batch_id = sync_jobs.id (data-lake.ts:47) upload_job_id, template_id, template_version (imports.ts:100-104)
Transport marker ingest_source (data-lake.ts:53) source_kind (imports.ts:108)
Processing state processed_at, processing_error (data-lake.ts:50-51) status, promoted_at (imports.ts:113-116)
Writer polling: writeBronze apps/worker/src/processors/bronze.ts:58; push: webhook route route.ts:158-172 apps/worker/src/processors/manual-upload.ts:434, apps/worker/src/processors/sheet-staging.ts:241
Validated against STREAM_SCHEMAS not for orders/creators: promoters read fields one by one (streams.ts:43-47) packages/connectors/google-sheets/src/values.ts:155, :280; apps/worker/src/processors/sync-dataset.ts:511; wizard packages/connectors/core/src/mapping/dictionary.ts:185
Replay reader readBronzeForReplay bronze.ts:85; markBronzeProcessed bronze.ts:111 re-run of an upload at its template version (§16)

Marketplace webhook payload#

POST /api/ingest/webhook/<tiktok-shop|shopee>/<connectorId>; HMAC header x-yng-signature (route.ts:101-120). The derived values below become one eventEnvelopeSchema envelope (route.ts:157-175); a delivery whose values do not fit it answers 400 invalid_payload. The body lands in raw_events.payload unchanged. Body cap and rate limit: Limits and budgets.

Derived value Source fields, in order Fallback Code
eventType eventType, type, code, topic 'webhook.received' route.ts:240-242
streamName inventory when the type contains inventory/stock or skuId/item_id is set orders route.ts:244-250
externalId orderId, order_id, resourceId, item_id, skuId, id <connectorType>-<first 12 hex of payload_hash> route.ts:251-259
occurredAt updatedAt, updated_at, update_time, eventTime, timestamp receive time route.ts:260-267
Job payload WebhookJobPayloadSchema apps/worker/src/queues.ts:722-731, carries rawEventId enqueued only when the bronze insert wrote a row (route.ts:201-219) reader processWebhookJob apps/worker/src/processors/phase-b.ts:236

Eventbus topics#

Producers validate the payload (bus.ts:164); subscribers parse the payload only, not the envelope (packages/eventbus/src/subscribe.ts:188-189, :500-501). EventEnvelope fields: id, schema_version (both absent on entries published before S4), topic, payload, publishedAt (ISO), producer (bus.ts:55-69). Broadcast topics use one stream for all orgs (bus.ts:126-135).

Topic Payload fields Publisher Subscriber
chat.tool_called orgId, brandId, conversationId, toolName, durationMs, ok (bus.ts:40-47) apps/web/src/server/chat/tools/factory.ts:118-129 none; stream length counted by apps/worker/src/metrics.ts:384
materialization.refreshed (broadcast) orgId, brandId (bus.ts:64-67) apps/worker/src/binding-cache-invalidation-publisher.ts:33 apps/web/src/server/binding-cache-eventbus-hook.ts:4-13
dataset_rollup.refreshed (broadcast) orgId, brandId (bus.ts:74-77) binding-cache-invalidation-publisher.ts:46 binding-cache-eventbus-hook.ts:4-13

Realtime ops feed#

One stream per org, fields type, orgId, at, payload (untyped record). Read by tRPC subscription.events (subscription.ts:23-58) for apps/web/src/app/(dashboard)/workspaces/ops/realtime-feed.tsx:18.

Type Publisher and payload keys
order.created one OrderEventPayloadSchema (realtime.ts): platform, platformOrderId, optional orderId, status; webhook (phase-b.ts) sets all four, polling sync (apps/worker/src/processors/sync.ts) the first two
order.status_changed webhook only, same keys as above (phase-b.ts:445)
inventory.below_reorder_point productId, productName, quantityOnHand (phase-b.ts:526-535)
creator.hour_spike none (grep at eb676254)

Platform events#

Envelope id, type, org_id, occurred_at, data; id is also X-MSO-Event-Id (webhooks.ts:5-17). Payload fields, signing, retries and deduplication: Platform API and events, section "Signing, retries and idempotency".

Type Direction Writer Reader
org.app.changed MSO to app apps/web/src/server/platform/org-app-changed.ts:32 via writePlatformEvent apps/web/src/server/platform/outbox.ts:26 delivery sweep apps/worker/src/processors/platform-webhook-delivery.ts:43-78
membership.changed MSO to app apps/web/src/server/platform/membership-events.ts:15 delivery sweep
user.disabled MSO to app membership-events.ts:24 delivery sweep
trends.updated MSO to app apps/worker/src/promoters/silver-upsert.ts:2109 via apps/worker/src/lib/platform-outbox-write.ts:13 delivery sweep
content.published App to MSO apps/web/src/app/api/platform/v1/events/route.ts:34-40 → acceptInboundEvent outbox.ts:85 none in apps/ or packages/ (grep at eb676254)

Lineage by layer#

Layer Table Link to the layer below
Bronze raw_events source_batch_id → sync_jobs.id; webhook rows leave it null (route.ts:186-199)
Silver, commerce orders (packages/schema/src/tables/orders.ts:23), inventory_snapshots raw_event_id → raw_events.id, source_batch_id → sync_jobs.id (0253, soft refs); writers
Silver, content content_mentions ingest_batch_id → content_ingest_batches (packages/schema/src/tables/content-intelligence.ts:577)
Silver, market intelligence market_intel_* ingest_batch (packages/schema/src/tables/market-intel.ts:193), capture_batch (:618); no raw_events link
Gold agg_org_day, agg_org_sku_day, agg_org_creator_day, agg_org_shopper_month, agg_org_campaign refreshed_at; source_watermark jsonb (0253), no writer yet
Decisions decision_episodes (packages/schema/src/tables/mso-episodes.ts:169) receipt_hash, signal_event_id; source_watermark jsonb (0253), no writer yet (decision engine)
Silver writer raw_event_id source_batch_id
Polling sync, upsertOrders/upsertInventory (apps/worker/src/processors/sync.ts) the row writeBronze stored or matched for that record (bronze.ts) the sync run
Bronze replay, runReplay (sync.ts) the replayed raw_events row the replay run
Webhook, processWebhookJob (apps/worker/src/processors/phase-b.ts) the job's rawEventId NULL
Upload promoter (apps/worker/src/promoters/orders.ts) NULL, cleared on overwrite NULL

Rows written before 0253 stay NULL; there is no backfill (83,065 of 91,047 prod orders have no Bronze row).

Gaps#

Gap Evidence
Schema version on two contracts of five webhook ingest and eventbus carry schema_version; connector streams, the realtime feed and platform events do not; a stream field change is a code release (streams.ts:16-19)
Realtime feed carries no event id each publish mints a Redis entry id (apps/worker/src/events-bus.ts:12)
Declared types with no publisher creator.hour_spike
payload_hash ignores nested values a key array as the JSON.stringify replacer filters every level (bronze-util.ts:13): {data:{status:'PAID'}} and {data:{status:'SHIPPED'}} hash equal, so the second hits onConflictDoNothing on both bronze writers (route.ts:171, apps/worker/src/processors/bronze.ts:72, :76)
Webhook identity falls back to receive time and a hash route.ts:224-240; the bronze row has no brand_id or source_batch_id (route.ts:160-170)
Webhook payload is open-ended .passthrough() (route.ts:37)
Gold and decisions have no watermark writer source_watermark exists on agg_* and decision_episodes (0253); nothing writes it yet

Envelope (ADR 0060, S4)#

One Zod eventEnvelopeSchema in packages/platform-contract/src/event-envelope.ts (@younetgroup/platform-contract 0.3.0); eventEnvelopeOf(dataSchema) types data. Guard: scripts/audit/event-envelope-single-schema.py (§205).

Field Type Webhook ingest Eventbus
id string 1-128; idempotency key deriveEventEnvelopeId: sha256 of the raw_events dedupe tuple randomUUID() per publish
type string 1-160 derived eventType topic
org_id uuid the authenticated connector's org orgId in the payload
brand_id uuid, nullable, optional not set (a webhook carries no brand, ADR 0028 D5) brandId in the payload
occurred_at ISO 8601 with offset derived occurredAt publishedAt
source_system string 1-64 connector type producer
source_ref string 1-256 derived externalId none
schema_version integer ≥ 1; EVENT_ENVELOPE_SCHEMA_VERSION 1 1
data any the vendor body, verbatim payload

The eventbus keeps its wire field names (the right column) so entries already in Redis still parse, and adds id and schema_version. Platform events keep their 0.2.0 shape (id, type, org_id, occurred_at, data).