kg_edges Retirement Plan — writers-first migration design
Status: EXECUTED — closed 2026-08-24. All waves landed and Wave 7's
delete ran (operator, 2026-08-20: 422 twins verified, 422 deleted, unmappable 0);
kg_relationships is the sole edge store and the last open decision (D8 kg-proxy)
closed 2026-08-24. One live hazard outlives the plan: its D8 row explains why
the remaining "operator gcloud builds submit" line must NOT be run — see §3c.
Body below is design-vintage (measured 2026-08-19, read-only pass at main
be339abe6; line numbers verified against that tree) and is kept as the
execution record, not as outstanding work. Register item 8 is the tracking row;
marketing-commerce-connectors.md rule 6a carries the standing HOLD.
Decision context: item 8a settled 2026-08-19 — writers-first sequencing, no
unscoped --apply; the delete (scripts/retire_kg_edges.py, operator) is the
LAST step.
Register rows: docs/reports/MIGRATION_CLOSURE_REGISTER_2026-09-02.md MC-01…MC-06 (Wave 1, 2026-09-02) — closure tests in tests/test_migration_closure_register.py.
0. The corrected inventory (supersedes the "seven writers / five readers" map)
A wide multiline grep (collection("kg_edges") … .set|.add|.update|.delete
within 250 chars) found what the recorded same-line grep missed: five writers
use asyncio.to_thread(<ref>.document(x).set, payload) — .set with no
parenthesis. Excluded as different collections: threat_kg_edges,
mt_kg_edges, gemini_kg_edges, gdp_kg_edges, fl_kg_edges,
fl_causal_kg_edges, agent_registry_kg_edges, mart_kg_edges_for_neo4j.
15 write sites (12 live in-service, 1 local-dev seeder, 2 delete sites),
12 read sites. Not one write site double-writes kg_relationships — every
edge written to kg_edges is invisible to Cell 3 GraphRAG retrieval today.
CORRECTED 2026-08-20: even this multiline grep missed the kg_projections
cluster — a live writer that names its collection through a variable
(config["collections"]["edges"]) — so the true totals are 16 write sites /
13+ read sites; see §0b. The lesson (variable-collection-name writes evade
every literal grep) is recorded cross-repo in shared-memory.
Writers (all collection names hardcoded)
| # | Path:line | Doc id | Kind | Note |
|---|---|---|---|---|
| 1 | miz-oki-adk-agents/boss/connector_gateway/kg_writer.py:141 |
auto-id (.add()) |
new edge | non-idempotent; _safe() swallows failures (:170-174); dead Neo4j branch :142-153. (No services/ copy exists) |
| 2 | miz-oki-adk-agents/boss/boss_agent_core.py:33347 |
edge_{uuid4[:12]} |
new edge | reached by unauthenticated routes — see Wave 2 |
| 3 | miz-oki-adk-agents/boss/policy_engine_integration.py:960 |
edge.idempotency_key |
audit append | uses from_id/to_id — unmappable by the migration script |
| 4 | miz-oki-adk-agents/boss/ads_causal_controller.py:1361 |
edge_{user}_{asset}_{ts} |
new edge | paired reader :1434 range-queries top-level timestamp |
| 5 | miz-oki-adk-agents/boss/eshkg_signal_integration.py:1203 |
pre-existing id, .update() |
enrichment | 404s if the doc's creator migrates first; paired reader :1480 |
| 6 | miz-oki-adk-agents/boss/privacy_safe_joins_attribution.py:656 |
caller id or edge_{uuid4[:8]} |
new edge | merge=True; caller-supplied shape |
| 7 | scripts/process_mycocoons_to_kg.py:448 |
deterministic | new edge, batch | easiest: already id/source/target/properties; flip relation→type (:160,203,299 + kg_events mirror :459) |
| 8 | miz-oki-adk-agents/boss/budget_reallocator_integration.py:604 |
budget_{channel} |
new edge | no source id — cannot be a canonical relationship as written |
| 9 | miz-oki-adk-agents/boss/autonomous_budget_reallocation_mvp.py:620 |
roi_{aud}_{camp} |
new edge | from_id/to_id convention |
| 10 | miz-oki-adk-agents/boss/edge_roi_inference_integration.py:531 |
cate_{impression} |
new edge | |
| 11 | miz-oki-adk-agents/boss/creative_test_deploy_integration.py:818 |
exp_{experiment} |
new edge | |
| 12 | miz-oki-adk-agents/boss/journey_scoring_autopilot_integration.py:404 |
journey_score_{user}_{ts} |
new edge | no target id — reclassify, cannot be an edge |
| 13 | boss_agent_core.py:33359 |
— | DELETE (delete_edge) |
reached by unauthenticated DELETE /api/v1/kg/edges/{id} (:38536) |
| 14 | boss_agent_core.py:33313,33317 |
— | DELETE cascade (delete_node) |
query-then-delete on source_id/target_id |
| 15 | miz-oki-adk-agents/boss/local_dev/local_server.py:176 |
edge_{i:03d} |
local-dev seeder | mock-guarded (:169); out of scope |
Writers 1–6 and 8–14 all ship in the boss-agent-adk image (Dockerfile.v5:27
COPYs all of boss/; imports verified at boss_agent_core.py module level).
Readers
Boss get_edges (boss_agent_core.py:33573 — filters relation, needs
type after the flip), /api/v1/kg/stats dual-ledger counts (:36157 +
kg_controller:33508 — the deliberate row; decision needed: keep reporting 0
as the retirement receipt, or remove the key + update boss.ts:218,242 and
two UI tests asserting kg_edges: 310), delete_node cascade (:33312),
command-center BFF/adapter/browse/hook (proxy + copy-only changes;
PREFERRED_COLUMNS already handles both type and relation),
srpaldl-cells/kg-proxy/app.py:416 (needs source/target/type + a composite
index; IAM-locked, no CI deploy path — manual gcloud builds submit from
a clean git archive), plus paired readers eshkg:1480,
autonomous_budget:582 (4-field composite query), ads_causal:1434,
policy_shaped_decision_layer.py:259, temporal_kg_snapshots.py:272,277
(checksums change when the collection empties), change_detection:291
(already reads type). One orphan reader:
budget_reallocator_integration.py:285 (uplift_{campaign} docs) — dead
against live data (nothing writes those docs to kg_edges; note the writer in
the SAME file at :604 writes budget_{channel} docs, a different id scheme).
CORRECTION (measured 2026-08-20): the previously-listed second "orphan"
knowledge_graph_brain_integration.py:759 (src.nodeId/dst.nodeId/rel) is
NOT dead — it is the live reader of the kg_projections cluster (§0b), a
writer the literal collection("kg_edges") grep missed. Some of the 5
firestore.indexes.json composites serve that cluster's live query_edges, not
only the one true orphan — re-audit before treating them as removable.
0b. The kg_projections cluster — a LIVE writer the grep missed (added 2026-08-20)
Both the §0 inventory and the recorded same-line grep were built from a literal
collection("kg_edges") search. kg_projections_integration.py writes through a
variable collection name, so it was invisible to every prior pass:
kg_projections_integration.py:238—KG_PROJECTIONS_CONFIG["collections"]["edges"] = "kg_edges".kg_projections_integration.py:455—self.db.collection(edges_collection).document(edge_id).set(edge_data), whereedge_data = {ownerId, src:{nodeId,type}, dst:{nodeId,type}, rel, props, weight, ts, validUntil}, doc idedge_{uuid4[:12]}. Nestedsrc/dst+relshape — NOT flatsource_id/target_id.- Wired live:
boss_agent_core.py:26612callsinitialize_kg_projections(...); gated byENABLE_KG_PROJECTIONS(boss_agent_core.py:1860, code defaulttrue), registered as MCP toolkg_upsert_edge(:1196). No deploy YAML overrides the flag. - Readers of this shape:
kg_projections_integration.py:487(query_edges, filtersownerId+rel+src.nodeId+dst.nodeId,order_by(ts DESC)) andknowledge_graph_brain_integration.py:759(§0 reader mislabel corrected).
Consequences for the retirement:
1. It is an additional live writer + two readers not in the §0 table.
2. Its nested src/dst/rel docs are unmappable by canonical_from_legacy()
(flat-field only) → they ABORT the all-or-nothing retire, same class as
writers 3/8/9/12. A dedicated nested→canonical mapper is required
(src.nodeId→source, dst.nodeId→target, rel→type, rest→properties).
3. query_edges needs a composite index on the canonical mapping.
4. Operator verification (cannot close from the repo): whether the deployed
boss-agent-adk env sets ENABLE_KG_PROJECTIONS=false. Code default is live;
an env override to false makes the cluster dormant and drops it in priority.
Confirm the served env before scheduling this cluster's migration.
Neither kg_projections_integration.py nor knowledge_graph_brain_integration.py
is on the deploy-boss-agent-core.yml allowlist (they ship in the image but a
reader-only PR to either alone merges-not-serving — bundle with an allowlisted
file, per Wave 1).
1. Target decision: direct canonical shape (Target A)
The governance-preferred path (MAPPERS[source] → ingest_gate →
service-canonical-ingestion) publishes to mizoki-events and stops —
no subscriber projects to kg_relationships (register item 26; verified
end-to-end in service-canonical-ingestion/main.py:194-234). Routing writers
there today is silent edge loss. Therefore: writers repoint DIRECTLY to
kg_relationships in the canonical_relationship() shape
({id, source, target, type, properties}, doc id = id, .set(merge=True);
kg_sinks.py:124-151 is the reference; Cell 3's reader consumes exactly this,
graphrag_service.py:526-614). Re-route onto governed ingestion when item 26
lands a projector — recorded follow-on, not this lane.
2. Dual-write: NO
(a) nothing double-writes today, so there is no window to preserve; (b) the
Boss/UI reads flip with their writers; (c) a dual-write keeps kg_edges
growing, which the all-or-nothing retire verify cannot tolerate; (d) writers
1–2 use random ids, so dual-writes cannot be reconciled by id. Instead:
cutover per writer, then re-run
scripts/migrate_kg_edges_to_kg_relationships.py (idempotent, additive,
skip-existing — proven 2026-08-11: +422, 0 unmappable) as the sweep
reconciler after each wave.
3. Waves (lowest blast radius first; deploy-coupled last)
- Wave 0 — offline reference diff. DONE 2026-08-19. Writer 7
(
process_mycocoons_to_kg):relation→typeat every edge dict + collection →kg_relationships(thekg_eventsaudit mirror keeps its ownrelationevent field, now read fromedge["type"]). No service, no deploy. Pinned bytests/test_kg_write_perimeter.py::test_mycocoons_writes_canonical_relationships_not_legacy_kg_edges. - Wave 1 — deploy-trigger repair, no behavior change.
deploy-boss-agent-core.ymltriggers are an explicit path ALLOWLIST; eight writer files (3, 5, 6, 8–12) are NOT on it — a fix to any of them merges and never deploys. Add them first and verify a run fires, or every later wave is merged-not-serving. - Wave 2a — close the unauthenticated write surface. DONE 2026-08-19.
The six KG mutation HTTP routes (
POST/PUT/DELETE /api/v1/kg/nodes,POST/DELETE /api/v1/kg/edges,POST /api/v1/kg/batch) carried no auth dependency on a service whose registry row isauth: public(production/service-registry.yaml:182). Closed with a fail-closedrequire_kg_writerdependency (auth module absent → 503; no bearer → 401; else delegate to the sameverify_supabase_tokenthe SRPVDAL/v1/reasonand/v1/traceroutes use). Reads (GET /api/v1/kg/nodes|edges) stay open — the dual-ledger browse surface. No mapped caller broken: every in-repo caller of these paths is a GET (UI/BFF); the MCP invoke callers carry auth already. Verified: no in-repo caller POSTs/DELETEs these routes. Pinned bytests/test_kg_write_perimeter.py(AST route-gate assertions + fail-closed branch behavior). Landed with Wave 0; boss-agent-adk deploys viadeploy-boss-agent-core.yml(boss_agent_core.pyis on its allowlist). - Wave 2b — MCP write tools. DONE 2026-08-19. The
write_kg_edge/write_kg_nodeMCP tools were reachable through the genericPOST /api/v1/mcp/tools/{name}/invokeexecutor, which invokes without anenforce_authcontext, so its RFC-8707 scope check is skipped — an unauthenticated write path parallel to the now-gated HTTP routes. The invoke route is NOT gated withverify_supabase_token— its real callers authenticate with Cloud Run IAM identity tokens (authedFetch), which that verifier would reject. Instead,invoke_toolrefuses the two tool names (_KG_WRITE_TOOLS_DISABLED_VIA_INVOKE) before handler dispatch, routing KG writes through the authenticated/api/v1/kg/*routes. Safe: no in-repo caller invokes these two tools by name (grep-verified); the registrations stay, so the skill catalog and the>= 100tool-count integration assertion are unaffected — only the ungated execution path is closed. Pinned bytests/test_kg_write_perimeter.py::test_mcp_invoke_refuses_kg_write_tools. - Do NOT repoint any writer before its wave — repointing a still-public write route would convert a public write into the legacy ledger into one into the canonical Cell 3 ledger; 2a/2b close that door first.
- Wave 3 — self-contained writers 8/9/10/11 (+ their paired readers + indexes, one PR each). Writers 8 and 12 must first gain a real second endpoint or be reclassified out of the edge store entirely.
- Writer 11 DONE 2026-08-19 (
creative_test_deploy_integration.py:818): the creative-experimenttest_resultedge now writeskg_relationshipsin canonical{id, source, target, type, properties}shape (endpoint types + status/winner relocated underproperties). Chosen first because it is write-only — grep-verified zero readers of theexp_/test_resultedge anywhere — so no paired-reader flip and no composite index were needed. File is on thedeploy-boss-agent-core.ymlallowlist (wave 1), so it deploys. Pinned bytests/test_kg_write_perimeter.py::test_creative_test_writer_migrated_to_canonical_ledger. - Writer 10 DONE 2026-08-19 (
edge_roi_inference_integration.py:531): the CATE-prediction (predicted_cate) edge now writeskg_relationshipsin canonical shape (source_id→sourceimpression,target_id→targetad_asset, endpoint types + CATE metrics intoproperties). Also write-only — thepredicted_cate/cate_docs have no reader (thepredicted_catehits elsewhere are in-memory dataclass/dict fields, not KG reads), so no reader flip, no index. On the allowlist ⇒ deploys. Pinned bytest_edge_roi_writer_migrated_to_canonical_ledger. - Writer 9 RECLASSIFIED to Wave 4 (measured 2026-08-19): it is NOT a
standalone wave-3 writer. The
ROI_DELTAedge it creates (autonomous_budget_reallocation_mvp:620,delta/confidence) is a coupled cluster shared with writer 5's enrichment (eshkg_signal_integration:1203,.update()addsroi_delta/roi_confidence/roi_updated_at) and TWO readers:autonomous_budget:582(readsdelta; 4-field compositeedge_type+to_id+confidence+order_by(delta)— no composite index exists inboss/firestore.indexes.jsontoday, so this query most likely returns empty in production now, a latent pre-existing bug) andeshkg:1480(readsroi_delta/roi_score, filtersedge_type+channel). Field-name divergence is load-bearing: writer 9 writesdelta, writer 5 writesroi_delta, the eshkg reader reads both. Migrating writer 9 alone would split the edge across two collections and 404 writer 5's.update(). So Wave 3's clean set is writers 10 + 11 (DONE); the ROI_DELTA cluster moves to Wave 4. - Wave 4 — coupled clusters (each = all writers + all readers + the
composite index in ONE PR, cutover-ordered so no
.update()404s): - ROI_DELTA cluster — CODE-COMPLETE (2026-08-19), activation operator-gated.
All four sites flipped to
kg_relationshipsin one change: writer 9 (autonomous_budget:620,.setfull doc) + writer 5 (eshkg:1203,.update()enrichment via dottedproperties.roi_delta/roi_confidence/roi_updated_atpaths); readersautonomous_budget:582+eshkg:1480. Mapping:from_id/audience_id→ top-levelsource,to_id/campaign_id→ top-leveltarget,edge_type→ top-leveltype;delta/confidence/roi_delta/roi_confidence/sample_size/channel/updated_at/the provenancesourcestring →properties.*. Thedeltavsroi_deltadivergence is preserved. Writer 5 keeps.update()(still 404s if the doc is missing — unchanged failure mode). Two composite indexes declared inboss/firestore.indexes.json:(type, target, properties.confidence, properties.delta DESC)for reader 582 and(type, properties.channel)for reader 1480's channel path. Behavior-preserving: reader 582 was already dead in prod (primary Neo4j path retired). ROOT CAUSE CORRECTED 2026-08-20 — the "no matching index" diagnosis was wrong. The query paired an inequality onproperties.confidencewith an ordering on the delta field. Firestore requires the inequality field to be the FIRST order_by, so the query was malformed and rejected server-side on every call — no index could ever have made it legal, and "apply the index to activate it" was therefore never true. Measured in-session: the Python client does NOT validate this (its only such check is cursor-specific,base_query.py:915, gated on_has_snapshot_cursor), so the rejection happens at.stream(), inside thetry, and degrades to[]via theexcept— which is exactly why it read as "empty because unindexed". The order_by now matches the inequality field; delta ranking and the limit moved in-process over a bounded, logged over-fetch. Pinned by 7 tests intests/test_kg_write_perimeter.py(both directions on all four sites, the index declaration, and a safety pin that no workflow auto-applies the index). Both files are on the boss deploy allowlist ⇒ the code deploys. - Touchpoint cluster — CODE-COMPLETE (2026-08-19). Writer 4
(
ads_causal_controllerTemporalKGUplift.record_touchpoint) + its reader (get_time_decayed_uplift), both flipped tokg_relationships: the (User)-[TOUCHED]->(Asset) endpoints are top-levelsource/target/type, everyTouchpointEdgeattribute preserved underproperties. The reader gains the requiredtype == "TOUCHED"discriminator (the canonical collection is multi-type; legacy kg_edges only worked because touchpoints were the sole timestamped edges — the governed journey writer also timestamps). Two composite indexes declared inboss/firestore.indexes.json:(type, properties.timestamp)and(type, target, properties.timestamp). Unlike ROI_DELTA this reader is a LIVE MCP surface (kg_time_decayed_uplift): the type filter forces an index the legacy no-asset path did not need, so between deploy and the operator index-apply the read degrades to empty with an ERROR log, not a crash (rule 01: read paths degrade and say so). OnlyTemporalKGUplift's kg_edges usage moves — thekg_brain/marketing_kgtouchpoint writers target other collections and are untouched.ads_causal_controller.pyis on the boss deploy allowlist ⇒ deploys. Pinned by 5 tests intests/test_kg_write_perimeter.py. -
Wave 5 — the boss-CRUD cluster + governance-shaped writers (measured 2026-08-20). Each bullet marks MECHANICAL (safe to build) vs DECISION (needs an owner ruling first).
-
5a. Boss CRUD cluster — CODE-COMPLETE (2026-08-20, GATE-1 local). All five sites in
boss_agent_core.pyflipped tokg_relationships:write_edge(source_id→source,target_id→target,relation→type,attributes+created_at→properties, +properties.origin="kg_controller"scope marker),get_edges(D1 — scopedwhere("properties.origin","==","kg_controller")+optionaltype; the API response still echoesrelation),delete_nodecascade (scoped byproperties.originon bothsourceandtargetso a node delete never touches system-generated canonical edges),delete_edge(id-only rename). Three composite indexes declared:(properties.origin, type),(source, properties.origin),(target, properties.origin).CORRECTED 2026-08-20 — the cascade line above was wrong, and the error was load-bearing.
get_edgesdoes degrade (caught error → local store, ERROR logged).delete_nodedid NOT: it returns{"success": False}from itsexcept, so the local-store fallback below it is unreachable oncefirestore_clientis set. Worse, it deleted the node BEFORE cascading, so on an unindexed project every call deleted the node, raisedFAILED_PRECONDITIONon the two-filter cascade, and left the node's kg_controller edges ORPHANED in the canonical ledger while reporting failure for a deletion that had already happened. A write path must not depend on an index an operator may not have applied. Fixed: the cascade now runs FIRST and filterskg_controllerin-process behind a single-field (auto-indexed) endpoint query, so it needs no composite index at all; scoping is unchanged and a partial failure is retryable and orphan-free.(source|target, properties.origin)are consequently retained-but-optional; only(properties.origin, type)still serves a live query, and that one is a read that degrades. Pinned by 9 tests (4 mutation-proved against the unfixed source). On the allowlist ⇒ deploys. Original design below. - 5a design (pre-build). Writer 2 (boss_agent_core.py:33347,relation/source_id/target_id/attributes) + readerget_edges(:33573,relation→type) +delete_nodecascade (:33312/:33315,source_id/target_id→source/target) +delete_edge(:33359, id-only rename). OneKGController, one doc shape; all on the allowlist ⇒ deploy together; splitting them zeroes the open browse surface. DECISION embedded —get_edgesscope: flipping it tokg_relationshipsbroadens the open, unauthenticatedGET /api/v1/kg/edgesfrom user-CRUD edges to ALL canonical types (journey, ROI, touchpoints…). Keep it scoped (filter to the CRUDtype, or atype in (...)clause) so the public browse surface is not silently widened. - 5b. Writer 1 — CODE-COMPLETE (2026-08-20, GATE-1 local); D5 FINAL.connector_gateway/kg_writer.py_write_edge. As built:edge_id = "edge_" + hashlib.md5(f"{tenant_id}|{source_id}|{edge_type}|{target_id}" .encode()).hexdigest()[:16]; record{id, source: source_id, target: target_id, type: edge_type, properties: {tenant_id, confidence, source (provenance), timestamp, provenance, policy_scope, **properties}}→collection("kg_relationships").document(edge_id).set(record, merge=True)via_safe. Both dead Neo4j branches (_write_edge,_upsert_node) removed;kg_nodeswrite kept. Guarded bytest_writer1_migrated_canonical_md5_natural_key. Spec/rationale below. Replace.add()(auto-id, non-idempotent — the anomaly: the file's retired Neo4j path alreadyMERGEd on(a)-[type]->(b), so Firestore was the only leg creating a fresh doc per write) with a deterministic natural-key id: ```python import hashlib key = f"{tenant_id}|{source_id}|{edge_type}|{target_id}" edge_id = "edge_" + hashlib.md5(key.encode()).hexdigest()[:16]write to kg_relationships as {id: edge_id, source, target, type, properties}, .set(merge=True)
`` Rationale: keyed on the edge's identity(source, target, type, tenant)so re-processing the same link dedupes (restores the MERGE semantics); mutable attributes (confidence,timestamp,provenance,policy_scope,properties) stay underpropertiesand update via.set(merge=True); **md5 hex** — never Pythonhash()(non-deterministic across runs — platform rule,gemini-kg-pipeline.md) — and hex output is Firestore-doc-id-safe where a rawf"{source}{type}"is not (ids can contain/, and it would omittenant_id). **Drop both dead Neo4j branches** in the file (_write_edge:142-153,_upsert_node:112-121)._safe()swallows failures (:170-174) — pre-existing, note it. On the allowlist ⇒ deploys. - **5c. Writer 6 — CODE-COMPLETE (2026-08-20).**privacy_safe_joins_attribution.pyKGResultWriter.write_results: caller-suppliededge_updatesdicts are now normalized into canonicalshape — extractssource_id→source,target_id→target,relation→type(defaultPSJ_EDGE), remaining attrs intoproperties. Collection flipped tokg_relationships,.set(merge=True)preserved. No reader exists (MCP toolpsj_write_kg_resultsis write-only). On the allowlist ⇒ deploys. Pinned bytest_writer6_psj_migrated_to_canonical_ledger. - **Writer 3 — CODE-COMPLETE (2026-08-20, GATE-1 local; D2).**policy_engine_integration.py_emit_transition_edge:from_id/to_id, **no reader**; the same class already appends the audit row tokg_transitions. Ruled collapse (drop thekg_edgeswrite, do not repoint) — an audit append has no graph consumer and does not belong in the retrieval ledger. **As built:** the whole method is now a documented no-op (the deadPolicyEdgeconstruction was also removed);kg_transitionsaudit in_save_transitionuntouched. Guarded bytest_writer3_transition_edge_dropped_audit_kept. - **Writers 8 & 12 — CODE-COMPLETE (2026-08-20, GATE-1 local; D3/D4).** Mis-modeled non-edges. Writer 8 (budget_reallocator:604,BUDGET_ALLOCATION) had **nosource_id** (onlysource_type:"system"); writer 12 (journey_scoring:404,journey:score) had **notarget_id**. Neither can be arelationship, and both already write their real store (budget_allocations:590;autopilot_scores:426). Ruled reclassify OUT of the edge store — thekg_edgeswrite is dropped, the real-store write kept, no data lost. **As built:** eachkg_edges.set()replaced by a D3/D4 rationale comment; real-store writes untouched. Guarded bytest_writer8_budget_edge_dropped_allocation_keptandtest_writer12_journey_score_edge_dropped_score_kept`. (A real graph edge later needs a second endpoint first — separate design task.) -
Wave 5c — kg_projections cluster (§0b, CODE-COMPLETE 2026-08-20). Confirmed LIVE (
ENABLE_KG_PROJECTIONSdefaults"true", not overridden). Config flipped"edges": "kg_relationships", writer remapped nested→canonical{id, source, target, type, properties}, both readers (query_edges:487 andknowledge_graph_brain:759) remapped to canonical field paths. Three composite indexes declared (properties.ownerId× {type,source,target} -
properties.ts). Pinned by 5 tests intests/test_kg_write_perimeter.py. -
Wave 6 — remaining readers flip. Clean readers CODE-COMPLETE (2026-08-20).
change_detection:291(already readtype— collection rename only),temporal_kg_snapshots:272/277(source_id→source; checksums + snapshot size grow once it scans the full canonical ledger — expected),policy_shaped_decision_layer:259(source_id→source+ remapedge_type/target_id/target_type/weightintotype/target/properties, the downstream dict-key contract preserved). All single-field / unfiltered queries ⇒ no composite index. Pinned by 3 tests intests/test_kg_write_perimeter.py(suite 33). DEPLOY-PATH: none of the three files are on the boss allowlist (§0b), so this change auto-merges but is merge-not-serving — the migrated readers ship in the boss image and go live on the next boss deploy (e.g. the reclassify-writer waves, which touch allowlisted files). Safe: until then the old code keeps readingkg_edges, which still holds data. Remaining Wave-6 item: the/api/v1/kg/statsdual-ledger counts (D7 — keep-0-as-receipt). - kg-proxy LAST (
srpaldl-cells/kg-proxy/app.py:416, DECISION + operator): needssource/target/type+ a composite index(source, type). Separate service, no CI deploy path — operatorgcloud builds submitfrom a pristinegit archive. IAM lock is carried by INGRESS config (internal-and-cloud-load-balancing), NOT the cloudbuild's--allow-unauthenticated(cloudbuild.yaml:46) — a naive re-deploy re-asserts theallUsersinvoker binding while ingress keeps it locked, so re-verify ingress after any manual deploy (rule 04). Live (consumed by thereasoncell) — apply the(source,type)index to the boss Firestore BEFORE flipping, or the proxy degrades to empty. - Wave 7 — reconcile → verify → delete. ✅ EXECUTED 2026-08-20 (operator).
migrate dry: 422 read,
migrate: 0,skip existing: 422,unmappable: 0, 0 written. retire dry: 422 twins verified, 0 BLOCKING. retire--apply: 422 documents deleted. The legacy collection is gone;kg_relationshipsis the sole edge store.unmappable: 0empirically settles precondition-2 class (b) — no single-endpoint docs existed — and means the class-(a) mapper extension was defensive, closing an abort class that held zero documents.
Wave 7 stale-reference sweep list (measured 2026-08-20, post-5b)
A repo-wide re-measure (revised 2026-08-20 post-item-8a, after a parallel
session landed writer 6, D6, D8 and the orphan reader) confirms no executing
kg_edges Firestore site remains in the boss lane: what is left in *.py is
comments, the migration/retire scripts that legitimately name the legacy
collection, the intentional D7 dual-ledger count, and unrelated collections
(gemini_kg_edges, gdp_kg_edges, fl_kg_edges, threat_kg_edges,
mt_kg_edges, agent_registry_kg_edges). But the earlier inventory was scoped
to *.py, so these non-executing references were never enumerated.
None is a migration blocker; all become wrong once the collection is deleted,
so the retire sweep must include them. SWEPT 2026-08-20 — all six flipped to
kg_relationships and pinned in both directions by
test_wave7_swept_reference_is_canonical (parametrized, 6 cases). Each safety
claim below was re-verified before editing, not taken from this table:
| Reference | Why it is not live |
|---|---|
boss/archive/boss_agent_merged_v5.0.0.py (3 collection() handles of 7 mentions) |
archive/ — superseded copy, not imported |
~~boss/local_dev/local_server.py:176~~ |
RESOLVED on main (fb6411064) — seeder now writes kg_relationships. The remaining :519/:600 hits are a route name and an in-memory dict key, not Firestore handles |
boss/lib/agents/knowledge/firestore-graph-store.ts:133, kg-ingest-client.ts:99 |
TypeScript inside a python:3.11-slim image (Dockerfile.v5); imported only by the lib/agents/index.ts barrel — never executes in the boss service |
boss/lib/agents/mcp/server.ts:456 |
hardcoded stub (count: 0, "Connect to actual KG store") — 'kg_edges' is a response label, not a collection handle |
miz-oki-command-center-ui/lib/firebase-admin.ts:84 (kgEdges: 'kg_edges') |
constant declared but never consumed (the other kgEdges hits are an unrelated useState in NeuralBrainVisualization.tsx). Path corrected 2026-08-20: an earlier row said command-center-ui/..., which does not exist — the narrower path would have read as absence |
boss/config/budget_reallocation_config.yaml:263, config/kg_ingestion_config.yaml:388 |
inert — neither file has any Python loader (git grep -ln "<name>" -- '*.py' ⇒ empty), so no config-driven collection-name indirection exists here |
Two near-misses worth recording so they are not re-investigated: the UI browse
surface (/api/bff/kg/edges) proxies the boss API rather than reading
Firestore, so it already rides wave 5a's migrated origin-scoped get_edges; and
roi_edges / kg_roi_edges / abr_roi_edges / gdp_kg_edges are different
collections, not kg_edges — out of scope for this retirement.
Per-writer tests ship with each wave: a source-literal assertion on the
collection constant and a shape assertion on the emitted doc (rule 01). Waves
2a/2b/3/4 (both wave-4 clusters), 5a, 5b + writers 3/8/12, the wave-6 clean
readers, (from the parallel session) writer 6, the D6 kg_projections cluster and
D8, and the wave-7 stale-reference sweep are all pinned by
tests/test_kg_write_perimeter.py (52 tests, green 2026-08-20). The sweep
pin also guards the opposite direction: /api/v1/kg/stats' intentional D7
dual-ledger count and its local_dev mirror must NOT be swept, or the evidence
the retirement is measured by disappears.
3c. Open decisions (owner rulings needed before Wave 5+ build)
The measurement surfaced choices that are NOT mechanical repoints. Each has a recommendation; a one-line ruling per item unblocks the build.
| # | Decision | Recommendation |
|---|---|---|
| D1 | get_edges open-surface scope (5a) |
Keep scoped to the CRUD type — do NOT broaden the public browse surface to all canonical edges |
| D2 | Writer 3 transition_executed (no reader) |
BUILT 2026-08-20 — collapsed into kg_transitions; kg_edges write dropped (method now a documented no-op). See writers §. |
| D3 | Writer 8 BUDGET_ALLOCATION (no source_id) |
BUILT 2026-08-20 — reclassified OUT; kg_edges write dropped, budget_allocations kept. See writers §. |
| D4 | Writer 12 journey:score (no target_id) |
BUILT 2026-08-20 — reclassified OUT; kg_edges write dropped, autopilot_scores kept. See writers §. |
| D5 | Writer 1 deterministic id | FINAL 2026-08-20 — "edge_" + md5(f"{tenant_id}\|{source_id}\|{edge_type}\|{target_id}")[:16] (natural-key idempotent; md5 not Python hash(); drop both dead Neo4j branches). See 5b. |
| D6 | kg_projections cluster (§0b) | BUILT 2026-08-20 — confirmed LIVE; config→kg_relationships, writer+2 readers remapped canonical, 3 composite indexes declared, 5 perimeter tests |
| D7 | /api/v1/kg/stats dual-ledger count |
DECIDED 2026-08-20 — keep reporting kg_edges count as retirement receipt (naturally 0 once drained); no code change needed, current dual-count code is correct |
| D8 | kg-proxy | CLOSED 2026-08-24 — the service is ALREADY DEPLOYED; do not run the completion step. Code was CODE-COMPLETE 2026-08-20 (reader flipped to kg_relationships with canonical field names; (source,type) composite index declared) and a deploy has since landed, so the plan's remaining "operator gcloud builds submit" line is spent. Running it now buys nothing and costs something: cloudbuild.yaml:46 carries --allow-unauthenticated and an unauth-curl smoke test, so a redeploy re-asserts the allUsers invoker binding. Ingress (internal-and-cloud-load-balancing) still holds the lock, so that is a standing least-privilege gap rather than an immediate public opening — but it is a hole that opens the moment ingress is ever loosened, and rule 04 is explicit that a later --no-allow-unauthenticated does not remove the binding. There is no CI path either (grep -rl "kg-proxy" .github/workflows/ = empty), so this file stays armed for whoever reads the plan literally next. The durable close is fixing the cloudbuild — dropping the flag and the unauth smoke test — not deploying again. |
Once D1–D5 and D7 are ruled, the buildable order is: 5a boss-CRUD cluster
(mechanical) ✅ → Wave 6 clean readers (bundled to deploy) ✅ → 5b writer 1
+ writers 3/8/12 per ruling ✅ (all CODE-COMPLETE 2026-08-20, GATE-1 local) →
5c writer 6 ✅ (CODE-COMPLETE 2026-08-20) →
kg_projections (D6) ✅ (CODE-COMPLETE 2026-08-20) →
orphan reader budget_reallocator:285 ✅ (dead read removed 2026-08-20) →
D7 ✅ (DECIDED: keep dual-count as retirement receipt) →
D8 kg-proxy ✅ (CODE-COMPLETE 2026-08-20; deploy operator-only) →
Wave 7 reconcile + retire (operator --apply sweep).
3b. Wave-4 operator index apply (kg_relationships) — GATE-2
Wave 4 declared four composite indexes in boss/firestore.indexes.json that
no CI applies (verified: zero firestore.indexes refs under
.github/workflows/, pinned by a test). Apply them to the boss Firestore
database with the operator index-apply path (firebase deploy --only
firestore:indexes, or gcloud firestore indexes composite create per index).
Agents never run gcloud/firebase (AGENTS.md) — this is operator/GATE-2. Two
tiers:
Tier 1 — RECOVERY (apply promptly; the reader is LIVE). The Touchpoint
reader kg_time_decayed_uplift degrades to empty + ERROR-log until these land:
- kg_relationships (type ASC, properties.timestamp ASC) — no-asset query
- kg_relationships (type ASC, target ASC, properties.timestamp ASC) — by-asset query
Tier 2 — RESOLVED 2026-08-20; it was a code defect, not an activation gate. This tier previously said applying the reader-582 index "reactivates that budget input". That was false, and the doubt embedded in its own text ("confirm Firestore accepts the shape, or reorder the query first") was the correct instinct: Firestore does NOT accept it. The query was malformed — inequality on one field, first order_by on another — so it failed regardless of indexing, and no index applied or withheld ever gated it. The query is fixed; the index declaration below now describes the legal query.
Activation posture is unchanged and was never carried by the index: the module's
public entrypoints default dry_run=True, create_decision yields
PROPOSED/PENDING_APPROVAL rather than APPLIED, actuation is a separate
explicit call, and no scheduler, cron, or terraform drives the module (measured
2026-08-20). Restoring a read leg is not arming an actuator.
- kg_relationships (type ASC, target ASC, properties.confidence DESC)
— reader 582, matching the corrected query. properties.delta is no longer an
index field: delta ranking happens in-process.
- kg_relationships (type ASC, properties.channel ASC) — reader 1480 channel path.
4. Preconditions for scripts/retire_kg_edges.py --apply (item 8b)
- Writer quiescence proof — the script snapshots then deletes
(
plan_retirement(),:70-78); a write between passes is deleted unverified. Run only after Wave 6, with the count stable across two passes. - Zero unmappable — status re-measured 2026-08-20:
- (a)
from_id/to_id/edge_type(writer 3'sPolicyEdge.to_dict(), confirmed from source history) — ✅ CLOSED: the mapper now reads these as canonical endpoints and type, and excludes them from the properties catch-all so they are not duplicated.edge_typehad to join the type fallback too, or such rows would map as"RELATED"with the real type demoted — a quieter defect than the abort it replaces. 4 tests, 3 of them mutation-proved red against the prior mapper. - (b) single-endpoint docs (writers 8, 12) — OPEN BY DESIGN. An edge without both endpoints is not an edge;Noneis correct and retire failing closed on it is correct. Only an operator can prove the class is empty (a Firestore read). If it is non-empty, that is a decision — delete the orphans or exclude them by id — not a mapper change. - (c) nestedsrc/dst/rel(kg_projections, §0b) — ✅ CLOSED, mapper readssrc.nodeId/dst.nodeId/rel.
Note the twin check is by id, not by re-mapping: an unmapped doc simply
never got a twin, which is exactly how an unmappable class becomes an abort.
3. Dual-ledger stats decision (reader table above).
4. Non-code cleanup the script does not touch — re-measured 2026-08-20:
- miz-oki-adk-agents/boss/firestore.rules match /kg_edges/{edgeId} +
doc comment — ✅ removed 2026-08-20 (post-delete). SOURCE-ONLY: no workflow
applies the rules file, so the served ruleset changes only when an operator
deploys rules. Removing it is also the safer resting state — with no match
block, a recreated kg_edges is denied by default instead of writable.
- 5 orphan kg_edges composite indexes — ✅ removed 2026-08-20. Also
declaration-only (no workflow applies the index file, by design).
- firebase-admin.ts:84 constant — ✅ swept (wave-7 sweep).
- both inert config mentions — ✅ swept (wave-7 sweep).
None of these blocks the delete; all become dead references after it.
5. The script authenticates via gcloud auth print-access-token —
operator-only, unchanged.