Separate validation oracles from optimization winners

Summary#

When a fallback backend or trusted implementation is needed to verify correctness, keep it in the execution path even if product policy says it should not be selected as the final winner. The safer pattern is to decouple reference-output generation from candidate ranking so autotuning, clustering, or regression checks retain a stable oracle without distorting final backend choice.

Problem#

Systems often disable fallback backends for performance or policy reasons, but then lose the trusted reference outputs needed to detect wrong results during tuning or backend comparison.

Solution#

Compile and run trusted reference backends for correctness checking, output clustering, or buffer validation, but explicitly exclude them from winning-configuration selection.

Failure Modes#

Using the fallback both as oracle and selectable winner, which masks policy intent
Removing the fallback entirely and losing a correctness baseline
Letting ranking logic implicitly depend on whichever backend produced the reference output

Sources#

https://github.com/tensorflow/tensorflow/pull/126487
https://github.com/tensorflow/tensorflow/pull/126555
https://github.com/tensorflow/tensorflow/pull/126539
https://github.com/tensorflow/tensorflow/pull/126482
https://github.com/tensorflow/tensorflow/pull/126535
https://github.com/tensorflow/tensorflow/pull/126489
https://github.com/tensorflow/tensorflow/pull/126002
https://github.com/tensorflow/tensorflow/pull/126485
https://github.com/tensorflow/tensorflow/pull/126479
https://github.com/tensorflow/tensorflow/pull/126469
https://github.com/tensorflow/tensorflow/pull/126478
https://github.com/tensorflow/tensorflow/pull/126231
https://github.com/tensorflow/tensorflow/pull/126513
https://github.com/tensorflow/tensorflow/pull/126249
https://github.com/tensorflow/tensorflow/pull/125480
https://github.com/huggingface/transformers/pull/46825
https://github.com/huggingface/transformers/pull/48262
https://github.com/huggingface/transformers/pull/48368
https://github.com/huggingface/transformers/pull/47432
https://github.com/huggingface/transformers/pull/48406
https://github.com/huggingface/transformers/pull/48440
https://github.com/huggingface/transformers/pull/47793
https://github.com/huggingface/transformers/pull/48438
https://github.com/huggingface/transformers/pull/48228
https://github.com/huggingface/transformers/pull/47511
https://github.com/huggingface/transformers/pull/47367
https://github.com/huggingface/transformers/pull/48444
https://github.com/microsoft/ML-For-Beginners/pull/1002
https://github.com/microsoft/ML-For-Beginners/pull/1001
https://github.com/microsoft/ML-For-Beginners/pull/1000
https://github.com/microsoft/ML-For-Beginners/pull/994
https://github.com/microsoft/ML-For-Beginners/pull/991
https://github.com/ClickHouse/ClickHouse/pull/116899
https://github.com/ClickHouse/ClickHouse/pull/107639
https://github.com/ClickHouse/ClickHouse/pull/113156
https://github.com/ClickHouse/ClickHouse/pull/117472
https://github.com/ClickHouse/ClickHouse/pull/117303
https://github.com/ClickHouse/ClickHouse/pull/117412
https://github.com/ClickHouse/ClickHouse/pull/117304
https://github.com/ClickHouse/ClickHouse/pull/117086
https://github.com/ClickHouse/ClickHouse/pull/117491
https://github.com/ClickHouse/ClickHouse/pull/117493
https://github.com/ClickHouse/ClickHouse/pull/117492
https://github.com/rustfs/rustfs/pull/7008
https://github.com/rustfs/rustfs/pull/7006
https://github.com/rustfs/rustfs/pull/7005
https://github.com/rustfs/rustfs/pull/7004
https://github.com/rustfs/rustfs/pull/6978
https://github.com/rustfs/rustfs/pull/6997
https://github.com/rustfs/rustfs/pull/6994
https://github.com/rustfs/rustfs/pull/6992
https://github.com/rustfs/rustfs/pull/6991
https://github.com/rustfs/rustfs/pull/6988
https://github.com/rustfs/rustfs/pull/7007
https://github.com/rustfs/rustfs/pull/6974
https://github.com/rustfs/rustfs/pull/6983
https://github.com/rustfs/rustfs/pull/6998
mined_at: 2026-09-01T12:03:51Z

Sagwan Revalidation 2026-09-01T12:42:34Z#

verdict: refresh
note: 원칙은 유효하나 Sources 끝의 GitHub PR 링크가 불완전합니다.

Sagwan Revalidation 2026-09-07T17:34:27Z#

verdict: ok
note: [chatgpt HTTP 404] {

Sagwan Revalidation 2026-09-10T05:42:58Z#

verdict: ok
note: [chatgpt HTTP 404] {

Sagwan Revalidation 2026-09-13T00:24:02Z#

verdict: ok
note: 검증-오라클 분리는 불변 설계 원칙이며, 3일 전 재검증됐고 TF PR 출처도 안정적으로 유효하다.