RKA gdrive binary stubs: real root cause + evidence-grounded gate design

Claim#

The 171/435 [Google Drive binary reference] stubs in RKA are NOT caused by extract_binary=False or would_extract=False (both hypotheses wrong; 165/171 were routed to binary extraction). Measured metadata.gdrive.text_extraction.status breakdown: skipped_oversize 94 (55%, flat 25MB cap; pptx median 162MB max 1.18GB), empty 43 (25%, scanned/image PDFs 25 + image pptx 17 + docx 1), failed 28 (16%, 27 = HWP), skipped_unsupported 6.

Key findings (live-validated)#

HWP failure is a CODE BUG, not a format limit. extract_hwp_text uses OleStorage(io.BytesIO(data)) + StringIO; pyhwp's lxml transform emits bytes → TypeError: string argument expected, got 'bytes', masked by a broad except. Live: hwp5txt <tempfile> extracted 42,165 chars from a 4.4MB hwp. Fix = temp-file + hwp5txt subprocess (shell=False, timeout, cleanup).
Oversize is format-dependent: live 298MB pptx → ~30 chars (media-heavy); 63MB PDF → 18,858 chars. Use per-format caps (PDF 200 / DOCX·HWP·HWPX 100 / XLSX·PPTX 50 MB), rest → on-demand queue. download_bytes buffers whole file in BytesIO → add ZIP-bomb + memory guards.
Stored Drive file_ids are STALE (all sampled stub IDs now 404 though SA can list the drive; files re-uploaded post-2026-06-30). content_hash includes file_id so move-detection won't catch replacements. Backfill must re-list + match by folder_path+name.

Evidence-gate root cause (problem: ghost suggestions)#

_has_grounded_evidence = len(evidence_text.strip())>=12 only; a 20-char folder-path echo passes.
_evaluate_extraction_signal gate is NEGATIVE: fires only when max_confidence<=0.45, so ONE conf>0.45 suggestion bypasses grounding. Fixing _has_grounded_evidence alone won't close it — rewrite gate POSITIVELY.
Two independent ghost producers: LLM extraction (stopped by is_reference_source stub-skip, which must run BEFORE allowlist_strong at source_filter.py:142) AND co_location_suggestions_from_metadata (folder-path producer, runs after doc store in ingest_gdrive_api.py). The latter is mislabeled ("asserted","deterministic","structured_source") at trust.py:35 → downgrade to ("inferred","probabilistic","structured_source"), review-only, excluded from signal counts. provenance_class enum has NO structural value.
The 28 stub-confirmed suggestions are all machine/auto (0 human approvals) → safe to re-evaluate.

Design#

Full converged design (with Codex/gpt-5.5) at repo docs/41_evidence_grounded_pipeline_hardening.md: A binary re-extraction, B mirror option C (body_digest(body, max_chars=_BODY_FETCH_MAX_CHARS) at call site only, keep _descriptive_body_text filter — dropping it regressed matching 0.871→0.816), C evidence gate/producers/dedup/edges. Split into code-only rounds (no prod write, keep 723-test baseline) vs approval-gated backfill + graph reconciliation (backfill ≠ cleanup; deprecate_suggested_for_document does not undo graph writes).

Sagwan Revalidation 2026-07-15T08:57:09Z#

verdict: refresh
note: Drive ID·추출 파이프라인 상태가 변동성 높아 현재 코드/목록 재검증 필요.

Sagwan Revalidation 2026-07-17T09:27:11Z#

verdict: ok
note: 2일 전 검증된 내부 계측·재현 결과라 현재 변경 근거가 없다

Sagwan Revalidation 2026-07-19T11:09:38Z#

verdict: ok
note: 직전 검증이 최근이며 주장·권장안에 시간 민감 변화 신호가 없다.

Sagwan Revalidation 2026-07-21T12:19:35Z#

verdict: ok
note: 직전 검증 후 2일뿐이며 주장·권장안 모두 현재 practice와 충돌 없음