Skip to content

chore: sync tests from upstream.#651

Merged
stephenamar-db merged 1 commit intodatabricks:masterfrom
He-Pin:syncTests
Mar 16, 2026
Merged

chore: sync tests from upstream.#651
stephenamar-db merged 1 commit intodatabricks:masterfrom
He-Pin:syncTests

Conversation

@He-Pin
Copy link
Contributor

@He-Pin He-Pin commented Mar 16, 2026

Fix Summary

Root Cause: The JS/Native/WASM platforms utilize the scala-yaml library; specifically, its parseYaml function does not support two features found in YAML multi-document streams:

  1. --- <content> — Content immediately following the document marker (e.g., --- > for a folded scalar, or --- 3 for a scalar value).
  2. Empty documents (where no content appears after the --- marker until the next ---).

Fix Strategy (3 Changes):

  1. src-native/Platform.scala + src-js/Platform.scala: Replaced the manual splitYamlDocuments implementation with parseManyYamls to correctly handle inline content scenarios such as --- > and --- 3. Added a addExplicitNullsForEmptyDocs pre-processing step to insert explicit null text for empty documents, thereby bypassing scala-yaml's limitation regarding the handling of empty documents.
  2. src-js/Platform.scala (nodeToJson function): Added a match case for None (specifically case null | None => ujson.Null) to ensure consistency with the Native platform. scala-yaml returns None when decoding null scalars, a scenario that was previously unhandled on the JS platform.
  3. test/src/ParseYamlTests.scala: Added regression tests to cover scenarios involving --- 3, --- >, empty documents, bare document markers, and similar cases.

@stephenamar-db stephenamar-db merged commit da1dd8d into databricks:master Mar 16, 2026
15 of 16 checks passed
@He-Pin He-Pin deleted the syncTests branch March 16, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants