Add an `alloc` feature & a generic `Value` type

[zeenix]

Creating a separate issue for this from the points I made here and here. To summarize, a generic Value type would be super useful to have but it will require allocations and hence I understand why it's currently a non-goal. However, to make the most out of Rust enums (after that is added), people will have to enable alloc feature of serde and IMO that sort of beats the point of resisting addition of an alloc feature to this crate.

So I'm suggesting addition of an optional alloc feature along with a generic Value type. I would most likely be able to provide a PR for this, if maintainers show interest.

[kayabaNerve]

This can be done without an alloc feature. I recently wrote a core-only crate for a binary self-describing format which is dynamically-typed to a bounded depth (which the format itself specifies, so the solution remains complete. For JSON, one would require alloc for completeness or one could presumably skip fields with constant memory if one assumes the JSON is valid and doesn't further validate it).

https://github.com/kayabaNerve/monero-oxide/blob/epee/monero-oxide/epee/src/lib.rs

There are distinctions, of course. This format has the privilege of being typed, while JSON is untyped until you actually start reading the value. The same premise can be applied to JSON however, and it's something I've been prototyping.

[zeenix]

This can be done without an alloc feature. I recently wrote a core-only crate for a binary self-describing format which is dynamically-typed to a bounded depth (which the format itself specifies, so the solution remains complete. For JSON, one would require alloc for completeness or one could presumably skip fields with constant memory if one assumes the JSON is valid and doesn't further validate it).

https://github.com/kayabaNerve/monero-oxide/blob/epee/monero-oxide/epee/src/lib.rs

There are distinctions, of course. This format has the privilege of being typed, while JSON is untyped until you actually start reading the value. The same premise can be applied to JSON however, and it's something I've been prototyping.

TBH with you, I am having a hard time seeing the relevance here. We could have avoided allocations (and make use of the heapless crate) with something like this:

use heapless::{Vec, String, IndexMap};

pub enum Value<N> {
    Null,
    Bool(bool),
    Number(Number),
    String(String<N>),
    Array(Vec<Value<N>, N>),
    Object(IndexMap<String<N>, Value<N>, N>),
}

but that can't work because of it being a recursively-sized type and can't even compile.

[kayabaNerve]

I cited code which works to deserialize arbitrary types without prior definition (of bounded depth yet unlimited width (the amount of value with an array, fields within an object)). It's comparable to Value in that as you deserialize, you can dynamically check the type of field, its length, and handle the names of fields as 'strings'. It also does not require any allocations.

[zeenix]

I cited code which works to deserialize arbitrary types

That's the thing. I found it hard to follow that very specific code. I wasn't even sure which type(s) exactly to look at there.

If you mean that Value can contain references instead of owned types, then I understand what you mean and yes, that doesn't require allocation but that's not a good generic API and can't be as useful as serde_json::Value. We need to support owned types.

It's comparable to Value

It would be useful if you could show how a Value type would look like (like I did in my previous comment).

[kayabaNerve]

I did follow through and apply the methodology to JSON: https://github.com/kayabaNerve/core-json

It offers a Value type which is internally a mutable borrow of a deserializer. The Value can have its type checked (object, array, string, number, bool, null) before being interpreted as one of them. It isn't a good, generic API and isn't as useful as serde_json::Value. What it is is non-allocating and usable on core alone. I'm not here to insist this should be implemented in serde-json-core. I just wanted to note that JSON serializations can be deserialized and interpreted at runtime, without allocating, even when they have unknown fields.

[zeenix]

It offers a Value type which is internally a mutable borrow of a deserializer.

Where? Can you please point to that type directly?

It isn't a good, generic API and isn't as useful as serde_json::Value. What it is is non-allocating and usable on core alone.

The entire point of this issue is to add a serde_json::Value equivalent. Addition of alloc is only to make that possible. If it's possible to do so without allocation, please do share the specifics of how that can be achieved. Otherwise, I don't see any value in this discussion.

[kayabaNerve]

https://github.com/kayabaNerve/core-json/blob/da0e29b33c28cb4da257078f3a380882309c492a/core/src/lib.rs#L544-L672 for the API which is possible, and why I brought this idea up in the first place.

https://github.com/kayabaNerve/core-json/blob/da0e29b33c28cb4da257078f3a380882309c492a/core/src/lib.rs#L298-L302 for the literal struct definition.

If the goal is explicitly a direct, API-compatible equivalent, then sure, add alloc. My comment was solely a comparable item (runtime-typed arbitrary JSON value) can be deserialized with core alone.