P1 Foundation — Build Plan (workstream zero opened 2026-08-11)
Status: Active build plan v1.0 · F6 of INTEGRATION PROMPT v2.3, opened on owner "Start"
after F1+F2 landed · Checklist source: docs/roadmap/SIGNAL_SHOPIFY_PHASE_BINDING.md ·
Exit: clean Shopify-vs-tracked reconciliation across 10 design partners (Stage-1 ≥80%
identity-coverage precondition per partner).
Discipline: the inherited checklist was measured against the tree before building
(rule: a handed-down list is a claim, not a fact). Most of the ingest half already exists —
P1 builds the gaps, never parallels.
A. Measured inventory — already built, do not rebuild
| Workstream-zero item | Measured state (2026-08-11) |
|---|---|
| Shopify webhooks (orders/refunds/inventory/fulfillments) | Built + deployed in services/service-marketing-connectors — POST /webhooks/shopify is topic-generic with HMAC verify, replay-cache dedupe, GDPR-topic ACK-never-ingest, inventory-topic routing to Cell 37, FAIL-CLOSED consent gate (merge 59782a39; deploy+verify green). ~~Blocked on operator secrets (SHOPIFY_WEBHOOK_SECRET unset ⇒ all deliveries 401 — register item 6)~~ Corrected 2026-09-30: the Shopify secrets have been populated and mounted since 2026-08-25 (docs/reports/SHOPIFY_SECRETS_STATUS_2026-08-25.md; the 08-11 "unset" reading was already corrected 08-12); zero live traffic has crossed the gate because no store has ever installed, and the shared-secret verifier is defect D1 of the 2026-09-30 lane plan (Shopify signs app webhooks with the app client secret — fix in PR #1275). |
| Bulk-operation backfill | Built + deployed: POST /api/v1/shopify/sync — paginated Admin GraphQL pull → CanonicalRecord → service-canonical-ingestion. Single-tenant credential posture (Secret Manager), caller-verified. |
| Canonical Event Envelope mapping | Built for the gateway streams (_shopify_record → canonical ingestion; no Shopify-special data model). Per-topic field-mapping review for fulfillments is a P1 verification item, not a rebuild. |
| Web Pixel / micro-signals → Cell 33 | Built + deployed (services/intent-shopify-extender, consent fail-closed, 503-when-unconfigured), same operator-secret block. |
| CAPI / Enhanced-Conversions transports | Built, flag-off, dry-run in services/measurement-rails (meta_capi.py, google_enhanced_conversions.py, writeback hard gates — 334-test suite green). The F6 "skeleton" exists; what was missing was the NCM valuation in front of it (§B). |
B. Built by this workstream-zero session (new, tested, deliberately landed)
| Artifact | What it is |
|---|---|
services/measurement-rails/ncm_v1.py |
NCM-v1 metric contract: the §2.5 formula with frozen term set (TERMS_FINGERPRINT — editing a term without shipping NCM-v2 fails the suite), consumer pinning (require_version), bundle-decomposed COGS that rejects rather than guesses, miNCM, and the batched/rate-limited true-up policy (owner decision 4 interaction documented, not pre-empted). |
services/measurement-rails/ncm_value_feed.py |
The L1 gate in front of the existing rails: E[NCM]-per-conversion fragment (margin, never raw revenue), FEEDS_DEFAULT_ENABLED = False as a test-asserted source literal, refuses without a ReconciliationAttestation of ≥14 clean days with a named basis. Nothing imports it into a serving path yet — wiring is step C3. |
test_ncm_v1.py + test_ncm_value_feed.py |
17 tests, both directions per rule (the violation each gate must catch AND the legal case it must pass). Full measurement-rails suite after addition: 334 passed. |
docs/onboarding/COGS_WORKSHEET.md + cogs_worksheet_template.csv |
ERP-less merchant cost intake: per-variant landed COGS, bundle decomposition, 3PL pick/pack, return-cost defaults — blank-beats-guessed rule matching the code's refusal semantics. |
Landing note: services/measurement-rails/** is a deploy-boss-agent-core push path — this
landing rebuilds the boss image with two pure-additive stdlib modules nothing imports at
runtime (a no-op at serving time; the pipeline was green twice today). That trade was taken
deliberately over inventing a new package home — consolidation beats a deploy-silent side door.
C. Remaining P1 build (classed; sequence top-down)
- [build] COGS validated import — BUILT 2026-08-12 (
cogs_import.py, besidencm_v1): worksheet CSV → row-level-reasoned rejects, blank-beats-guessed exclusions visible in a coverage report → tenant-scoped cost table →OrderEconomicsassembly with refuse-never-guess semantics (unknown/uncosted variants, missing platform fee, unresolved return costs all refuse). The repo template CSV is parsed inside the suite, so the interchange shape and the code cannot drift apart silently. No serving surface (per plan). - [build] Reconciliation harness — BUILT 2026-08-12 (
reconciliation.py): day-by-day Shopify-vs-tracked comparator over the full contiguous span (an absent day is never agreement), exact order-count matching, tighten-only revenue tolerances whose ceilings are test-asserted source literals, trailing-consecutive clean-day count, content-addressed dated artifact per merchant;attest()is the blessedReconciliationAttestationproducer, and the suite proves the 14-clean-day path opensncm_value_feed's gate end to end. Measurement-rails suite after both landings: 364 passed (was 334). Honest limit: the harness is the instrument, not the evidence — no attestation exists for any merchant until a runner feeds it real Shopify-Admin + canonical-stream totals, which ~~stays blocked on the operator secrets (register item 6)~~ waits on the first store install (corrected 2026-09-30: the secrets landed 2026-08-25). - [build] Value-feed wiring — BUILT 2026-08-12 (
ncm_feed_wiring.py): E[NCM] flows ONLY fromncm_value_feed.value_fragmentinto the rails' OWN constructors and send paths (meta_capi,google_enhanced_conversionsRESTATEMENT) — EMQ hashing, shared dedup ids, validate-before-send, dry-run defaults,require_rail/require_transportall the rails' own, none modified. Adds: attestation tenancy check (merchant must equal tenant — clean days are not transferable,FLEET_INTEGRITY.md), loud refusal of negative E[NCM] (never clamped to zero), and an input shape with no revenue field so raw revenue is inexpressible.WIRING_DEFAULT_ENABLED=Falsesource literal, ast-asserted, anti-drift-pinned toFEEDS_DEFAULT_ENABLED. 32 tests; rails suite 364→396. No serving path imports it; a live feed still requires a REAL per-merchant attestation + the rails' flags + operator credentials — all absent today. - [build] Multi-merchant OAuth install flow — DESIGN NOTE AUTHORED 2026-08-12
(
docs/architecture/SHOPIFY_OAUTH_INSTALL_DESIGN.md): auth-code grant (embedded token-exchange as designed swap-in), Level-1 protected-data scope set with per-stream justification, expiring offline tokens with single-flight refresh, schema-2 extension of the EXISTINGSecretManagerCredentialStore(measured: custody mechanism already exists unwired — the build is smaller than this line previously implied),shopify_shop_registryshop→tenant resolution with aDEFAULT_TENANT_IDmigration shim, uninstall/compliance jobs behind the shipped ACK-never-ingest boundary, everything flag-gated OFF. Ordered build B0–B6+OP, each step with both-direction tests; every gateway-touching step is a deliberate deploy. Measured gap registered by the note (§1):resolve_tenant/ tenant- registry enforcement is invoked on NO gateway route today — the connectors production gate is unmet on this surface; lands at B2/B3. IMPLEMENTED 2026-08-12 (B0–B4+B6):shopify_install.py(registry lifecycle, schema-2 custody, nonce+query-HMAC verifiers, compliance job queue) + flag-off routes + webhook tenant resolution (single-tenant baseline byte-identical, 34-test suite unmodified) + syncresolve_tenantenforcement w/ per-tenant credentials and proven single-flight refresh + uninstall/compliance jobs behind the unchanged ACK boundary + README.SHOPIFY_OAUTH_ENABLED=Falseast-asserted twice; +126 tests, combined 205 green. B5 BUILT 2026-08-12:activate_web_pixel(installed-rows-only per §8 — activation never precedes step-5 completion; settings ALLOWLIST so nothing person-shaped can ship in pixel config;userErrors TAKEN= already-active, so re-runs are idempotent) wired as the defaultPIXEL_ACTIVATION_HOOK— unconfigured lane ⇒ deferred-LOUD (metric + warning, install completes: installs never couple to the optional pixel lane; since the 2026-08-19 pixel ingress perimeter package the lane's config isSHOPIFY_PIXEL_COLLECT_URL+SHOPIFY_PIXEL_INGEST_SECRET, and settings carry the per-install token + O-2 knobs —docs/architecture/PIXEL_INGRESS_PERIMETER.md); extender per-shop config landed asINTENT_TENANT_MAP(mapped shop → its tenant; unmapped shop refused when a map is configured; set-but-invalid map fails the surface closed — a tenant boundary is never guessed at; no map ⇒ single-tenant byte-identical, proven by the unmodified pre-existing suite) + a suspended-hold activation guard (§5.1 names the status, no writer exists yet — guard lands ahead of the writer). Suites: gateway+connectors 205→220, extender 12→20. The pixel extension ARTIFACT (JS + settings schema readingingest_url/shop_domain) ships with the Shopify app project — [operator] OP row; the extender's/pixel/eventsreceiving endpoint is designed WITH that artifact, not before it. Remaining: the OP operator row (app creation, client secret custody, Level-1 request, item-6 secrets — noteSHOPIFY_WEBHOOK_SECRETsemantics change with live registry rows — that semantics note is defect D1, fixed by PR #1275, 2026-09-30), canonical- store erasure leg — ~~measured 2026-08-12: BLOCKED,service-canonical-ingestionexposes no subject/erasure path at all (grep zero), so the leg needs its own design+build on that service's lane; the compliance-job leg stays honestlypending_in_buildand keeps jobs OPEN~~ corrected 2026-09-30: the path EXISTS since 2026-08-20 (POST /api/v1/subjects/eraseonservice-canonical-ingestion; gateway leg_canonical_store_erasurebehindCANONICAL_ERASER_URL, unset in production, so compliance jobs stay honestly OPEN until an operator arms it — register item 27) (rule 01: fail-closed needs somewhere to close onto — wiring a pretend-eraser would be worse than the open job). Owner decisions 1 and 2 DECIDED 2026-08-12 (owner-delegated; rulings in master §3.6a + design note §9): direct/unlisted distribution P1–P3 with App Store at P4, and merchant-owned ad accounts only — both select the already-landed implementation, so no code changed with the ruling; the OP row now instructs direct/unlisted app creation definitively. - [build] Fulfillments-topic mapping review — DONE 2026-08-12, verdict: GAPS — 6
findings, fixes landed (
docs/reports/FULFILLMENTS_MAPPING_REVIEW_2026-08-12.md, measured by driving the real route with HMAC-signed deliveries, not by reading): "generic path accepts the topic" was TRUE; "likely small" was FALSE — G1 (HIGH): the fail-closed consent gate leaked the fulfillmentdestinationship-to block (full name, street, geo) because the redaction set covered order address containers but not fulfillments'. G1 + G2 (order linkage on the Identity axis) + G4 (provenance.backfillon sync pulls) + G5 (deliveryX-Shopify-API-Versioninto provenance) + G6 (10 seeded both-direction tests; gateway suite 24→34) fixed in the same landing;addresses(customer address book, same leak class) covered too;origin_addressdeliberately NOT redacted (merchant org data — reasoned in the report). Zero live traffic had ever crossed the gate (secret unset), so no production exposure occurred. Follow-ups — ALL CLOSED 2026-08-12: C5a fulfillments sync BUILT (rides the orders connection, ship-todestinationnever selected,provenance.backfillstamped — G3 closed; note: anoracle-liiC5a takeover claim raced this landing after the board's >6h-stale rule fired on a clock-regime jump — the build landing here is the settlement, same PII posture both sides intended); C5b detector-set symmetry landed both twins byte-identical; C5c found + fixedpayment_details(cardholder name, masked PAN, BIN, AVS/CVV) surviving consent denial — allad6f0fc1. - [build] Reconciliation runner — BUILT 2026-08-12 (
ops/reconciliation/run_reconciliation.py, 29 tests): Shopify-Admin vs canonical-stream daily totals →reconcile()→ dated content-addressed artifact + attestation; INERT until credentials (operator secrets item below; register item 6) AND — measured while building — ~~the canonical envelope stream has NO BigQuery landing table (events land in Firestore via the outbox;mizoki-eventshas zero subscribers)~~ corrected 2026-09-30: the landing exists since 2026-08-20 — DDLbigquery/schemas/canonical_events.sql, andservice-canonical-ingestionterminates at BigQuerycanonical_eventswhen itsCANONICAL_EVENTS_BQ_TABLElane is armed on the serving revision (an operator read, not asserted here) — so--events-tableis required-no-default and the tracked side names that table before any real attestation can exist. Register item 26. - [measured] Canonical-store erasure leg — ~~no surface exists (branch c)~~ built
2026-08-20, unarmed in production (corrected 2026-09-30)
(
docs/reports/CANONICAL_ERASURE_LEG_2026-08-12.mdis the 08-12 measurement): the routePOST /api/v1/subjects/eraseexists onservice-canonical-ingestionand the gateway calls it from_canonical_store_erasurewhenCANONICAL_ERASER_URLis set — it is unset on the serving revision, so the compliance-job leg stays honestly OPEN and alerting (never silently done); arming it is the operator's (register item 27). Cell 33's cascade still covers the intent family + Cell 35 graph; no gateway side door was built. - [operator] Secrets — ~~
SHOPIFY_WEBHOOK_SECRET+ provider credentials (register item 6 / 6c): the whole ingest half stays 401/fail-closed until these land.~~ Corrected 2026-09-30: the three Shopify secrets (client id, client secret, webhook secret) are populated and mounted since 2026-08-25; the ingest half now waits on the first store install and on D1 (PR #1275 — Shopify signs app webhooks with the app client secret). Provider credentials (Klaviyo key, ad-platform tokens) remain operator items. Nothing in P1 code can substitute for this. - [owner] Design partners — recruiting the 10 partners gates the P1 exit, not the
build. Of decisions 1–3: 1 (App Store vs direct) and 2 (account custody) are DECIDED
2026-08-12 (master §3.6a — direct/unlisted P1–P3 + merchant-owned accounts, so partner
onboarding is unblocked on both); 3 (Profit Truth Audit wedge go/no-go) ~~stays open~~
RULED GO 2026-09-15 (
docs/OPEN_ITEMS.mdD-3; corrected here 2026-09-30).
D. Standing constraints on every P1 step
- L0/L1 only: no spend authority anywhere in P1 — value feeds are L1's ceiling and stay
off until the attestation exists; promotion beyond L1 is P2+ under
docs/architecture/DEL_AUTHORIZATION_FUNCTION.md(canon — PR #656 merged by the owner 2026-08-11, deciding owner item 11) and the phase-binding exits. - Consent fail-closed, no audio, deny-list, HMAC-tokenized identifiers — inherited platform law, already enforced at the built boundaries.
- Fleet integrity (
docs/architecture/FLEET_INTEGRITY.md): merchant cost data and NCM outputs never cross the tenant boundary. - Every landing on gateway or rails paths is a deliberate deploy-path change (item-27 lesson); docs and plan updates ride auto-merge freely.