Register item 15 — Step-0 verification (2026-08-20)

Read-only verification of the register line "mycocoons ads/GA buckets not in KG — ~9.5k Facebook / ~8k Google / GA4 objects unprojected; close via service-marketing-connectors → canonical ingestion, never bucket→Firestore" before building it (rule 01: measure the inherited inventory before acting on it). Every claim below is measured in-repo at file:line unless labeled otherwise. Verifier: growth-control-completion session, Step-0 sweep run under the item-15 lane claim (3433b74).

Headline: the register line is materially stale

The objects are not unprojected — they are ingested nightly and projected into a different Firestore collection pair than the one the KG census measured. The real gap is a graph-store split plus a missing projector, not an ingest gap.

The two disjoint graphs

writer collections reader
mycocoons nightly ingest services/gemini-kg-pipeline/src/firebase/firebase_kg_client.py:134-136 nodes / edges / metadata the pipeline's own census only (scripts/kg_census.py:56,71,83)
journey/GraphRAG path services/gemini-kg-pipeline/src/journey_events/kg_sinks.py:163,169 kg_nodes / kg_relationships Cell 3 GraphRAG (src/cells/cell03/v2/services/graphrag_service.py:372,388,642)

Same project, same database, two disjoint graphs. Facebook/Google/GA4 entities live in nodes/edges; retrieval reads kg_nodes/kg_relationships.

Findings

  1. "Unprojected" is false. The nightly 10 6 * * * job walks the whole bucket (setup_ingest_schedulers.sh:63-80 → /api/v1/ingest/mycocoons/full; orchestrator.py:63-69 defaults to prefix ""), with dedicated parsers for both Facebook export vintages, Google Ads, and GA4 device/demographics (src/mycocoons/parsers.py:149-219,894-905,462,846). Module memory records 8 consecutive nightlies and ads/GA4 content in the graph (1,391 GA4 segments, campaign hierarchies, 2,456 order→ad attribution edges — .claude/memory/active/gemini-kg-pipeline.md:11-27, recorded evidence, not re-measured here). No source-based filter exists; skips are per-file (orchestrator.py:832-851).

  2. The prescribed governed path terminates at BigQuery, not the KG. Gateway batch door → _send_to_canonical_ingestion (service-marketing-connectors/main.py:637-641,260-316) → envelope + outbox + Pub/Sub mizoki-events (service-canonical-ingestion/main.py: 215-232) → BigQuery canonical_events (deployment/terraform/canonical_events_landing/main.tf:102-137, the ONLY subscriber). No mizoki-events → kg_nodes/kg_relationships projector exists — item 26 closed the BigQuery landing leg only, and current-priorities.md itself records "the governed ingestion path cannot reach kg_relationships until item 26 builds the projector" (item 8) and "mizoki-events subscribers: ZERO in code and in terraform — this IS item 26's build half" (item 6). Item 15 therefore has an undeclared hard dependency on a projector that was never built.

  3. The facebook/google/ga4 = 0 census figure is partly an artifact. (a) Wrong collection (nodes vs kg_nodes); (b) the mycocoons ingest sets source to the gs:// source-file URI, not a channel name (src/mycocoons/accumulator.py:175-176), so source == 'facebook' cannot match by construction; (c) the journey-lane vocabulary is meta / google_ads (journey-event.json:11), never the strings facebook/ga4 — a re-run census grepping those strings returns 0 even after closure. The Shopify-heavy kg_nodes content (~16k Customer) came from different writers (scripts/process_mycocoons_to_kg.py:420,452 demo + Cell 3's BigQuery processor lineage), not from the bucket walk.

  4. scripts/process_mycocoons_to_kg.py is an active violation of this item's own rule — it reads gs://mycocoons CSVs and writes kg_nodes/kg_relationships/kg_events directly (:395-470), i.e. bucket→Firestore. It is listed under item 8a's writer inventory (wave 0 landed its edge-shape fix) but belongs under item 15's scope for retirement once the governed path serves the same data.

  5. Counts: the only current in-repo measurement is .claude/memory/active/marketing-commerce-connectors.md:269 (2026-08-07 live scan): Facebook ~9.5k, Google ~8k, Shopify ~1.2k, GA4Custom ~2.2k, GAAnalytics ~8.3k. docs/reports/CELL2_INGESTION_STATUS_REPORT.md:56-79 ("700+ files", nodes_count: 1000) predates real ingestion and is obsolete for sizing. Note Klaviyo files live under the Facebook/ prefix (router.py:58-60), so "~9.5k Facebook" is not purely Meta Ads.

  6. Consent/PII posture of the bucket data: the ads rows carry no person identifiers (Facebook/Google column maps are account/campaign/adset/ad/ UTM/metrics only — parsers.py:149-219,894-902; GA4 exports are aggregate reports, corroborated by docs/INTENT_CORPUS_CARD_MYCOCOONS.md:70-73). The exception is Klaviyo (person×campaign engagement); the standing decision is NO per-person nodes (parsers.py:519-525). The gateway's fail-closed consent gate is bound to Shopify person-topics only (main.py:1396-1409,1635-1645); the batch door has no consent gate and no resolve_tenant (measured at main.py:637-641 — unlike sync_shopify at :650).

Verdict — smallest governed build (plan of record for the lane)

Already exists (do not rebuild): the authenticated batch door accepting meta_ads/google_ads/ga4 (PROVIDER_CONFIG), the gateway→canonical forwarder, the atomic outbox + publish + drainer, the BigQuery landing, the BaseAdapter.pull → CanonicalRecord pattern (direct_connectors.py:102-118, 426-444), CSV column knowledge (parsers.py), the path-family classifier (router.py:48-98), and the CI deploy lanes for both services.

Must be written, in dependency order:

  1. THE BLOCKER — a canonical-events → KG projector downstream of the canonical door (connectors rule: "provider projections belong downstream of the canonical door, as a post-ingestion hook — never as a second door"). Without it the governed path ends in BigQuery and item 15 cannot close as written. This is also item 26's unbuilt leg.
  2. A GCS batch-pull adapter in service-marketing-connectors (new google-cloud-storage dependency; watermark/checkpoint store; Klaviyo prefix policy preserving the no-per-person-nodes decision).
  3. resolve_tenant on the batch door; consent decision for Klaviyo rows.
  4. Tests under tests/connectors/ (they gate the gateway deploy).

Operator preconditions (not agent work): roles/storage.objectViewer on gs://mycocoons for mizoki-platform@; a Cloud Scheduler job for the new sync/projection routes; CONNECTOR_CONSENT_REGISTRY_ENABLED if the consent gate is extended.

Alternative noted and not taken: migrating the existing nodes/edges graph into kg_nodes/kg_relationships would be cheaper but does not satisfy connectors rule 1 (canonical ingestion before KG projection) and contradicts the register's own "never bucket→Firestore" prescription — the ungoverned writer would remain the source. If the owner wants the cheap path anyway, that is an owner decision to record, not a shortcut an agent should take.

← All docsView source on GitHub →