fromJSON + toJSON are not Idempotent

[J-Moravec]

The operation fromJSON followed by toJSON should semantically modify the json. I.e., these two chained should be content indempotent / symetric.

This is currently not the case, rjson doesn't distinguishes between scalars and single element arrays, it also converts empty objects into arrays.

library("rjson")

# json as a multiline string
# validated with https://jsonlint.com/
json = '{
    "object": {
        "name": "value"
    },
    "object_empty": {},
    "single_element": 1,
    "array_one": [
        1
    ],
    "array_two": [
        1,
        2
    ],
    "array_empty": []
}'

obj = rjson::fromJSON(json) |> rjson::toJSON(4)

# We can't compare with identical due to changes in formatting:
cat(obj)

{
    "object":{
        "name":"value"
    },
    "object_empty":[

    ],
    "single_element":    1,
    "array_one":    1,
    "array_two":[
        1,
        2
    ],
    "array_empty":[

    ]
}

The above example demonstrate two big issues resulting in change of type. This is very problematic with any typed system that does distinguish between scalar and vectors/arrays (unlike R :/)

1. "object_empty" -- previously empty object {} is converted to empty array -> TYPE CHANGE
2. "array_one" -- previously a single element array is turned into a scalar -> TYPE CHANGE

Note that jsonlite can with some work be forced to follow the rules:

jsonlite::fromJSON(json, simplifyVector = FALSE) |>
  jsonlite::toJSON(pretty = TRUE, auto_unbox = TRUE) |>
  cat()

Its stupid that one has to go to such length to do so but at least, it works.

[alexcb]

The issue related to single element arrays was reported in #7, does using the simplify=TRUE option help you?

As to your comment related to differentiating {} from [], currently both of these are converted into a list() in R. What would you propose the new behaviour should be?

[J-Moravec]

Hi Alex,

I will investigate the "simplify = FALSE" option, looks like it helps solve the issue 2. in exchange for leaving all arrays as list instead of type-specific vectors

Still, the issue 1. remains. jsonlite introduces a subtype of lists for this, named lists, which IMHO fits well since objects needs to be named (i.e., json being formed by name-value pairs according to specification).

It might introduce a new non-native R type, but IMHO would help with type-safety without drastic changes.

[alexcb]

If a change were to be made, it's crucial that it doesn't affect backwards compatibility in any way, since many packages depend on rjson.

I don't have much spare time to dedicate to rjson currently, but would be open to looking at a PR as long as backwards compatibility is kept.

[mmyrte]

I think this could be accomplished in a backward-compatible fashion by adding R attributes (not the class attribute like jsonlite does with as.scalar since as you say, that could reduce backward-compatibility). Setting these should probably be opt-in, since it would incur an overhead and the established expectation is of non-idempotency.

So the parser would set an rjson_type attribute indicating one of {dict, scalar}, or leave it unset to revert to the default assumption of "it's all an array". There should probably also be some user-exposed function to mark objects dict or scalar, respectively, like jsonlite does with unbox.