병렬 실행 계획은 결과뿐 아니라 배치도 검증

Summary#

분산 모델의 출력이 맞더라도 각 작업자가 모든 데이터를 보유하면 병렬화의 메모리·확장성 이점이 사라진다. 기능 테스트와 별도로 실행 계획에서 실제 분할 대상이 각 작업자에 나뉘어 배치됐는지 확인한다.

Problem#

MoE 병렬화 계획에서 전문가 분할이 빠져도 출력은 정상이라 기존 정확성 테스트만으로는 복제 배치를 발견할 수 없었다.

Solution#

각 TP/EP 계획에 전문가 분할을 포함하고, 작업자별 전문가 배치를 직접 검증하는 회귀 테스트를 추가한다.

Failure Modes#

출력 동등성만으로 병렬화 성공을 판단함
일부 병렬화 계획만 검사해 다른 계획의 전체 복제를 놓침

Sources#

https://github.com/tensorflow/tensorflow/pull/123516
https://github.com/tensorflow/tensorflow/pull/129244
https://github.com/tensorflow/tensorflow/pull/128465
https://github.com/tensorflow/tensorflow/pull/128629
https://github.com/tensorflow/tensorflow/pull/123572
https://github.com/tensorflow/tensorflow/pull/129161
https://github.com/tensorflow/tensorflow/pull/129184
https://github.com/tensorflow/tensorflow/pull/129173
https://github.com/tensorflow/tensorflow/pull/128300
https://github.com/tensorflow/tensorflow/pull/129035
https://github.com/huggingface/transformers/pull/43112
https://github.com/huggingface/transformers/pull/29967
https://github.com/huggingface/transformers/pull/49690
https://github.com/huggingface/transformers/pull/49206
https://github.com/huggingface/transformers/pull/49642
https://github.com/huggingface/transformers/pull/49657
https://github.com/huggingface/transformers/pull/49370
https://github.com/huggingface/transformers/pull/49610
https://github.com/ClickHouse/ClickHouse/pull/124531
https://github.com/ClickHouse/ClickHouse/pull/126354
https://github.com/ClickHouse/ClickHouse/pull/126153
https://github.com/ClickHouse/ClickHouse/pull/113572
https://github.com/ClickHouse/ClickHouse/pull/123514
https://github.com/ClickHouse/ClickHouse/pull/119353
https://github.com/ClickHouse/ClickHouse/pull/119814
https://github.com/ClickHouse/ClickHouse/pull/108469
https://github.com/ClickHouse/ClickHouse/pull/125467
https://github.com/ClickHouse/ClickHouse/pull/126581
https://github.com/ClickHouse/ClickHouse/pull/126580
https://github.com/ClickHouse/ClickHouse/pull/125899
https://github.com/rustfs/rustfs/pull/8657
https://github.com/rustfs/rustfs/pull/8656
https://github.com/rustfs/rustfs/pull/8654
https://github.com/rustfs/rustfs/pull/8562
https://github.com/rustfs/rustfs/pull/8559
https://github.com/rustfs/rustfs/pull/8650
https://github.com/rustfs/rustfs/pull/8651
https://github.com/rustfs/rustfs/pull/8640
https://github.com/rustfs/rustfs/pull/8649
https://github.com/rustfs/rustfs/pull/8647
https://github.com/rustfs/rustfs/pull/8648
https://github.com/rustfs/rustfs/pull/8643
https://github.com/rustfs/rustfs/pull/8542
https://github.com/rustfs/rustfs/pull/8641
mined_at: 2026-10-18T16:34:55Z

Sagwan Revalidation 2026-10-18T17:07:44Z#

verdict: ok
note: 출력 검증과 별도로 작업자별 전문가 배치를 확인하라는 권장안은 여전히 유효하다.