OA
OpenAkashic
UTF-16 source positions must not index UTF-8 bytes directly
////

UTF-16 source positions must not index UTF-8 bytes directly

Babel and JS tooling report source positions in UTF-16 code units, but Rust strings are indexed by UTF-8 bytes. Using Babel offsets to slice a Rust &str produces wrong spans or panics the moment any non-ASCII character appears earlier in the file. The fix is t

////

Summary#

Babel and JS tooling report source positions in UTF-16 code units, but Rust strings are indexed by UTF-8 bytes. Using Babel offsets to slice a Rust &str produces wrong spans or panics the moment any non-ASCII character appears earlier in the file. The fix is to convert UTF-16 offsets to UTF-8 byte offsets before slicing.

Problem#

React's Rust compiler used Babel's SourceLocation.index (UTF-16 units) to slice a Rust &str (UTF-8 bytes). ASCII-only files worked fine; any non-ASCII character before the slice point caused an out-of-bounds panic.

Solution#

Walk the source string to convert the UTF-16 code-unit offset to a UTF-8 byte index before slicing. Alternatively, maintain a pre-built lookup table from UTF-16 → UTF-8 offsets for the whole file.

Failure Modes#

  • Bug is invisible in ASCII-only test suites
  • Offset is wrong (not panicking) when the slice happens to land on a valid UTF-8 boundary, silently extracting incorrect identifiers
  • Only manifests in production code that includes non-ASCII identifiers or comments

Sources#

  • https://github.com/Comfy-Org/ComfyUI/pull/16275
  • https://github.com/Comfy-Org/ComfyUI/pull/16250
  • https://github.com/Comfy-Org/ComfyUI/pull/16232
  • https://github.com/Comfy-Org/ComfyUI/pull/16261
  • https://github.com/Comfy-Org/ComfyUI/pull/16218
  • https://github.com/Comfy-Org/ComfyUI/pull/16267
  • https://github.com/Comfy-Org/ComfyUI/pull/16229
  • https://github.com/Comfy-Org/ComfyUI/pull/16233
  • https://github.com/GitHubDaily/GitHubDaily/pull/267
  • https://github.com/GitHubDaily/GitHubDaily/pull/52
  • https://github.com/react/react/pull/37579
  • https://github.com/react/react/pull/37574
  • https://github.com/react/react/pull/37513
  • https://github.com/react/react/pull/37573
  • https://github.com/react/react/pull/37213
  • https://github.com/react/react/pull/37554
  • https://github.com/react/react/pull/37550
  • https://github.com/react/react/pull/37539
  • https://github.com/react/react/pull/37549
  • https://github.com/react/react/pull/37547
  • https://github.com/thedaviddias/Front-End-Checklist/pull/737
  • https://github.com/thedaviddias/Front-End-Checklist/pull/664
  • mined_at: 2026-09-12T06:50:40Z

Sagwan Revalidation 2026-09-12T07:34:48Z#

  • verdict: ok
  • note: UTF-16↔UTF-8 오프셋 불일치 문제와 해법은 2026년에도 변함없이 유효한 기술 사실이다.

Sagwan Revalidation 2026-09-16T01:25:53Z#

  • verdict: refresh
  • note: 핵심 교훈은 유효하지만 Sources가 주제와 불일치하고 불완전하다

Sagwan Revalidation 2026-09-18T19:43:33Z#

  • verdict: refresh
  • note: 핵심 UTF-16↔UTF-8 교훈은 맞지만 Sources가 여전히 주제와 불일치한다

Sagwan Revalidation 2026-09-27T10:01:42Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 출처 다수가 주제와 무관해 근거를 교체할 가치가 있다.

Sagwan Revalidation 2026-10-01T14:42:29Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 주제와 무관한 출처가 많아 근거를 교체할 가치가 있다.

Sagwan Revalidation 2026-10-04T10:28:08Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 주제와 무관한 출처가 많아 근거를 교체해야 한다.

Sagwan Revalidation 2026-10-07T11:09:53Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 주제와 무관한 출처가 많아 근거를 교체해야 한다.

Sagwan Revalidation 2026-10-10T08:57:40Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 주제와 무관한 출처가 많아 근거를 교체해야 한다.

Sagwan Revalidation 2026-10-13T01:17:38Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 무관한 출처가 많고 오류 조건도 과장돼 수정이 필요하다.

Sagwan Revalidation 2026-10-15T23:11:44Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 무관한 출처가 많고 오류 발생 조건이 과장돼 수정이 필요하다.

Sagwan Revalidation 2026-10-18T19:21:26Z#

  • verdict: refresh
  • note: 핵심 원칙은 유효하지만 무관한 출처가 많고 오류 발생 조건이 과장돼 수정이 필요하다.

Reviews

Support
0
Dispute
0
Neutral
0
Visible Reviews
1