Currently, it is not clear whether there is a documented standard for what should be included in metadata.yaml / sessionMetadata.yaml. For downstream tools, this makes it difficult to know which fields are required, which fields are optional, and what data types or formats should be expected. Without a documented schema or validation layer, downstream tools may parse metadata inconsistently, which can lead to corrupted derived datasets, metadata loss, or incorrect interpretation of OpenCap data.
I recommend defining the expected metadata structure using JSON Schema: https://json-schema.org/. A JSON Schema could document required and optional fields, expected data types, allowed values where appropriate, and version-specific changes. The schema could then be used to validate metadata.yaml files, since YAML can be loaded into the same data structure as JSON for validation.
An example of how this can be done in Python is here: https://devops-db.com/python-yaml-validation-with-json-schema/
I would be willing to help create an initial schema if the OpenCap team can provide guidance on what fields should be included in metadata.yaml / sessionMetadata.yaml, which fields are stable across exports, and which fields are optional or version-dependent.
Currently, it is not clear whether there is a documented standard for what should be included in
metadata.yaml/sessionMetadata.yaml. For downstream tools, this makes it difficult to know which fields are required, which fields are optional, and what data types or formats should be expected. Without a documented schema or validation layer, downstream tools may parse metadata inconsistently, which can lead to corrupted derived datasets, metadata loss, or incorrect interpretation of OpenCap data.I recommend defining the expected metadata structure using JSON Schema: https://json-schema.org/. A JSON Schema could document required and optional fields, expected data types, allowed values where appropriate, and version-specific changes. The schema could then be used to validate
metadata.yamlfiles, since YAML can be loaded into the same data structure as JSON for validation.An example of how this can be done in Python is here: https://devops-db.com/python-yaml-validation-with-json-schema/
I would be willing to help create an initial schema if the OpenCap team can provide guidance on what fields should be included in
metadata.yaml/sessionMetadata.yaml, which fields are stable across exports, and which fields are optional or version-dependent.