It happens a lot that from SaaS connectors, we have fields that end up as null under specific circumstances.
We are considering marking all properties as nullable by adding null to their type if they are not a key. For keys we don't want to make them nullable.
This can be done using traverse_jsonschema and getting a list of primary keys, and then adding null to type of every non-primary-key field. A simple example usage of traverse_jsonschema: https://github.com/estuary/airbyte/blob/master/airbyte-to-flow/src/interceptors/fix_document_schema.rs#L197-L210
It happens a lot that from SaaS connectors, we have fields that end up as
nullunder specific circumstances.We are considering marking all properties as nullable by adding
nullto their type if they are not a key. For keys we don't want to make them nullable.This can be done using
traverse_jsonschemaand getting a list of primary keys, and then addingnullto type of every non-primary-key field. A simple example usage oftraverse_jsonschema: https://github.com/estuary/airbyte/blob/master/airbyte-to-flow/src/interceptors/fix_document_schema.rs#L197-L210