P1 Foundation — Build Plan (workstream zero opened 2026-08-11)

Status: Active build plan v1.0 · F6 of INTEGRATION PROMPT v2.3, opened on owner "Start" after F1+F2 landed · Checklist source: docs/roadmap/SIGNAL_SHOPIFY_PHASE_BINDING.md · Exit: clean Shopify-vs-tracked reconciliation across 10 design partners (Stage-1 ≥80% identity-coverage precondition per partner). Discipline: the inherited checklist was measured against the tree before building (rule: a handed-down list is a claim, not a fact). Most of the ingest half already exists — P1 builds the gaps, never parallels.

A. Measured inventory — already built, do not rebuild

Workstream-zero item Measured state (2026-08-11)
Shopify webhooks (orders/refunds/inventory/fulfillments) Built + deployed in services/service-marketing-connectors — POST /webhooks/shopify is topic-generic with HMAC verify, replay-cache dedupe, GDPR-topic ACK-never-ingest, inventory-topic routing to Cell 37, FAIL-CLOSED consent gate (merge 59782a39; deploy+verify green). ~~Blocked on operator secrets (SHOPIFY_WEBHOOK_SECRET unset ⇒ all deliveries 401 — register item 6)~~ Corrected 2026-09-30: the Shopify secrets have been populated and mounted since 2026-08-25 (docs/reports/SHOPIFY_SECRETS_STATUS_2026-08-25.md; the 08-11 "unset" reading was already corrected 08-12); zero live traffic has crossed the gate because no store has ever installed, and the shared-secret verifier is defect D1 of the 2026-09-30 lane plan (Shopify signs app webhooks with the app client secret — fix in PR #1275).
Bulk-operation backfill Built + deployed: POST /api/v1/shopify/sync — paginated Admin GraphQL pull → CanonicalRecord → service-canonical-ingestion. Single-tenant credential posture (Secret Manager), caller-verified.
Canonical Event Envelope mapping Built for the gateway streams (_shopify_record → canonical ingestion; no Shopify-special data model). Per-topic field-mapping review for fulfillments is a P1 verification item, not a rebuild.
Web Pixel / micro-signals → Cell 33 Built + deployed (services/intent-shopify-extender, consent fail-closed, 503-when-unconfigured), same operator-secret block.
CAPI / Enhanced-Conversions transports Built, flag-off, dry-run in services/measurement-rails (meta_capi.py, google_enhanced_conversions.py, writeback hard gates — 334-test suite green). The F6 "skeleton" exists; what was missing was the NCM valuation in front of it (§B).

B. Built by this workstream-zero session (new, tested, deliberately landed)

Artifact What it is
services/measurement-rails/ncm_v1.py NCM-v1 metric contract: the §2.5 formula with frozen term set (TERMS_FINGERPRINT — editing a term without shipping NCM-v2 fails the suite), consumer pinning (require_version), bundle-decomposed COGS that rejects rather than guesses, miNCM, and the batched/rate-limited true-up policy (owner decision 4 interaction documented, not pre-empted).
services/measurement-rails/ncm_value_feed.py The L1 gate in front of the existing rails: E[NCM]-per-conversion fragment (margin, never raw revenue), FEEDS_DEFAULT_ENABLED = False as a test-asserted source literal, refuses without a ReconciliationAttestation of ≥14 clean days with a named basis. Nothing imports it into a serving path yet — wiring is step C3.
test_ncm_v1.py + test_ncm_value_feed.py 17 tests, both directions per rule (the violation each gate must catch AND the legal case it must pass). Full measurement-rails suite after addition: 334 passed.
docs/onboarding/COGS_WORKSHEET.md + cogs_worksheet_template.csv ERP-less merchant cost intake: per-variant landed COGS, bundle decomposition, 3PL pick/pack, return-cost defaults — blank-beats-guessed rule matching the code's refusal semantics.

Landing note: services/measurement-rails/** is a deploy-boss-agent-core push path — this landing rebuilds the boss image with two pure-additive stdlib modules nothing imports at runtime (a no-op at serving time; the pipeline was green twice today). That trade was taken deliberately over inventing a new package home — consolidation beats a deploy-silent side door.

C. Remaining P1 build (classed; sequence top-down)

  1. [build] COGS validated import — BUILT 2026-08-12 (cogs_import.py, beside ncm_v1): worksheet CSV → row-level-reasoned rejects, blank-beats-guessed exclusions visible in a coverage report → tenant-scoped cost table → OrderEconomics assembly with refuse-never-guess semantics (unknown/uncosted variants, missing platform fee, unresolved return costs all refuse). The repo template CSV is parsed inside the suite, so the interchange shape and the code cannot drift apart silently. No serving surface (per plan).
  2. [build] Reconciliation harness — BUILT 2026-08-12 (reconciliation.py): day-by-day Shopify-vs-tracked comparator over the full contiguous span (an absent day is never agreement), exact order-count matching, tighten-only revenue tolerances whose ceilings are test-asserted source literals, trailing-consecutive clean-day count, content-addressed dated artifact per merchant; attest() is the blessed ReconciliationAttestation producer, and the suite proves the 14-clean-day path opens ncm_value_feed's gate end to end. Measurement-rails suite after both landings: 364 passed (was 334). Honest limit: the harness is the instrument, not the evidence — no attestation exists for any merchant until a runner feeds it real Shopify-Admin + canonical-stream totals, which ~~stays blocked on the operator secrets (register item 6)~~ waits on the first store install (corrected 2026-09-30: the secrets landed 2026-08-25).
  3. [build] Value-feed wiring — BUILT 2026-08-12 (ncm_feed_wiring.py): E[NCM] flows ONLY from ncm_value_feed.value_fragment into the rails' OWN constructors and send paths (meta_capi, google_enhanced_conversions RESTATEMENT) — EMQ hashing, shared dedup ids, validate-before-send, dry-run defaults, require_rail/require_transport all the rails' own, none modified. Adds: attestation tenancy check (merchant must equal tenant — clean days are not transferable, FLEET_INTEGRITY.md), loud refusal of negative E[NCM] (never clamped to zero), and an input shape with no revenue field so raw revenue is inexpressible. WIRING_DEFAULT_ENABLED=False source literal, ast-asserted, anti-drift-pinned to FEEDS_DEFAULT_ENABLED. 32 tests; rails suite 364→396. No serving path imports it; a live feed still requires a REAL per-merchant attestation + the rails' flags + operator credentials — all absent today.
  4. [build] Multi-merchant OAuth install flow — DESIGN NOTE AUTHORED 2026-08-12 (docs/architecture/SHOPIFY_OAUTH_INSTALL_DESIGN.md): auth-code grant (embedded token-exchange as designed swap-in), Level-1 protected-data scope set with per-stream justification, expiring offline tokens with single-flight refresh, schema-2 extension of the EXISTING SecretManagerCredentialStore (measured: custody mechanism already exists unwired — the build is smaller than this line previously implied), shopify_shop_registry shop→tenant resolution with a DEFAULT_TENANT_ID migration shim, uninstall/compliance jobs behind the shipped ACK-never-ingest boundary, everything flag-gated OFF. Ordered build B0–B6+OP, each step with both-direction tests; every gateway-touching step is a deliberate deploy. Measured gap registered by the note (§1): resolve_tenant / tenant- registry enforcement is invoked on NO gateway route today — the connectors production gate is unmet on this surface; lands at B2/B3. IMPLEMENTED 2026-08-12 (B0–B4+B6): shopify_install.py (registry lifecycle, schema-2 custody, nonce+query-HMAC verifiers, compliance job queue) + flag-off routes + webhook tenant resolution (single-tenant baseline byte-identical, 34-test suite unmodified) + sync resolve_tenant enforcement w/ per-tenant credentials and proven single-flight refresh + uninstall/compliance jobs behind the unchanged ACK boundary + README. SHOPIFY_OAUTH_ENABLED=False ast-asserted twice; +126 tests, combined 205 green. B5 BUILT 2026-08-12: activate_web_pixel (installed-rows-only per §8 — activation never precedes step-5 completion; settings ALLOWLIST so nothing person-shaped can ship in pixel config; userErrors TAKEN = already-active, so re-runs are idempotent) wired as the default PIXEL_ACTIVATION_HOOK — unconfigured lane ⇒ deferred-LOUD (metric + warning, install completes: installs never couple to the optional pixel lane; since the 2026-08-19 pixel ingress perimeter package the lane's config is SHOPIFY_PIXEL_COLLECT_URL + SHOPIFY_PIXEL_INGEST_SECRET, and settings carry the per-install token + O-2 knobs — docs/architecture/PIXEL_INGRESS_PERIMETER.md); extender per-shop config landed as INTENT_TENANT_MAP (mapped shop → its tenant; unmapped shop refused when a map is configured; set-but-invalid map fails the surface closed — a tenant boundary is never guessed at; no map ⇒ single-tenant byte-identical, proven by the unmodified pre-existing suite) + a suspended-hold activation guard (§5.1 names the status, no writer exists yet — guard lands ahead of the writer). Suites: gateway+connectors 205→220, extender 12→20. The pixel extension ARTIFACT (JS + settings schema reading ingest_url/shop_domain) ships with the Shopify app project — [operator] OP row; the extender's /pixel/events receiving endpoint is designed WITH that artifact, not before it. Remaining: the OP operator row (app creation, client secret custody, Level-1 request, item-6 secrets — note SHOPIFY_WEBHOOK_SECRET semantics change with live registry rows — that semantics note is defect D1, fixed by PR #1275, 2026-09-30), canonical- store erasure leg — ~~measured 2026-08-12: BLOCKED, service-canonical-ingestion exposes no subject/erasure path at all (grep zero), so the leg needs its own design+build on that service's lane; the compliance-job leg stays honestly pending_in_build and keeps jobs OPEN~~ corrected 2026-09-30: the path EXISTS since 2026-08-20 (POST /api/v1/subjects/erase on service-canonical-ingestion; gateway leg _canonical_store_erasure behind CANONICAL_ERASER_URL, unset in production, so compliance jobs stay honestly OPEN until an operator arms it — register item 27) (rule 01: fail-closed needs somewhere to close onto — wiring a pretend-eraser would be worse than the open job). Owner decisions 1 and 2 DECIDED 2026-08-12 (owner-delegated; rulings in master §3.6a + design note §9): direct/unlisted distribution P1–P3 with App Store at P4, and merchant-owned ad accounts only — both select the already-landed implementation, so no code changed with the ruling; the OP row now instructs direct/unlisted app creation definitively.
  5. [build] Fulfillments-topic mapping review — DONE 2026-08-12, verdict: GAPS — 6 findings, fixes landed (docs/reports/FULFILLMENTS_MAPPING_REVIEW_2026-08-12.md, measured by driving the real route with HMAC-signed deliveries, not by reading): "generic path accepts the topic" was TRUE; "likely small" was FALSE — G1 (HIGH): the fail-closed consent gate leaked the fulfillment destination ship-to block (full name, street, geo) because the redaction set covered order address containers but not fulfillments'. G1 + G2 (order linkage on the Identity axis) + G4 (provenance.backfill on sync pulls) + G5 (delivery X-Shopify-API-Version into provenance) + G6 (10 seeded both-direction tests; gateway suite 24→34) fixed in the same landing; addresses (customer address book, same leak class) covered too; origin_address deliberately NOT redacted (merchant org data — reasoned in the report). Zero live traffic had ever crossed the gate (secret unset), so no production exposure occurred. Follow-ups — ALL CLOSED 2026-08-12: C5a fulfillments sync BUILT (rides the orders connection, ship-to destination never selected, provenance.backfill stamped — G3 closed; note: an oracle-lii C5a takeover claim raced this landing after the board's >6h-stale rule fired on a clock-regime jump — the build landing here is the settlement, same PII posture both sides intended); C5b detector-set symmetry landed both twins byte-identical; C5c found + fixed payment_details (cardholder name, masked PAN, BIN, AVS/CVV) surviving consent denial — all ad6f0fc1.
  6. [build] Reconciliation runner — BUILT 2026-08-12 (ops/reconciliation/run_reconciliation.py, 29 tests): Shopify-Admin vs canonical-stream daily totals → reconcile() → dated content-addressed artifact + attestation; INERT until credentials (operator secrets item below; register item 6) AND — measured while building — ~~the canonical envelope stream has NO BigQuery landing table (events land in Firestore via the outbox; mizoki-events has zero subscribers)~~ corrected 2026-09-30: the landing exists since 2026-08-20 — DDL bigquery/schemas/canonical_events.sql, and service-canonical-ingestion terminates at BigQuery canonical_events when its CANONICAL_EVENTS_BQ_TABLE lane is armed on the serving revision (an operator read, not asserted here) — so --events-table is required-no-default and the tracked side names that table before any real attestation can exist. Register item 26.
  7. [measured] Canonical-store erasure leg — ~~no surface exists (branch c)~~ built 2026-08-20, unarmed in production (corrected 2026-09-30) (docs/reports/CANONICAL_ERASURE_LEG_2026-08-12.md is the 08-12 measurement): the route POST /api/v1/subjects/erase exists on service-canonical-ingestion and the gateway calls it from _canonical_store_erasure when CANONICAL_ERASER_URL is set — it is unset on the serving revision, so the compliance-job leg stays honestly OPEN and alerting (never silently done); arming it is the operator's (register item 27). Cell 33's cascade still covers the intent family + Cell 35 graph; no gateway side door was built.
  8. [operator] Secrets — ~~SHOPIFY_WEBHOOK_SECRET + provider credentials (register item 6 / 6c): the whole ingest half stays 401/fail-closed until these land.~~ Corrected 2026-09-30: the three Shopify secrets (client id, client secret, webhook secret) are populated and mounted since 2026-08-25; the ingest half now waits on the first store install and on D1 (PR #1275 — Shopify signs app webhooks with the app client secret). Provider credentials (Klaviyo key, ad-platform tokens) remain operator items. Nothing in P1 code can substitute for this.
  9. [owner] Design partners — recruiting the 10 partners gates the P1 exit, not the build. Of decisions 1–3: 1 (App Store vs direct) and 2 (account custody) are DECIDED 2026-08-12 (master §3.6a — direct/unlisted P1–P3 + merchant-owned accounts, so partner onboarding is unblocked on both); 3 (Profit Truth Audit wedge go/no-go) ~~stays open~~ RULED GO 2026-09-15 (docs/OPEN_ITEMS.md D-3; corrected here 2026-09-30).

D. Standing constraints on every P1 step

← All docsView source on GitHub →