Strategies for reducing resource utilization by NFTs

[christophersanborn]

NFTs impose a RAM burden on witness and API nodes by virtue of the fact that asset objects, including description field in which the NFT payload is embedded, are kept in RAM. Each NFT will consume a larger than ordinary amount of RAM for this for all nodes, including block-producing witness nodes, which can otherwise run on lean hardware since they don't serve an API to clients. Token creators should do their utmost to reduce the storage footprints of their artwork, however, a storage burden in some form is inescapable.

As a quick calculation, it would take 2,621 NFT's utilizing the entire 400KB max_transaction_size allotment to consume 1 GB of witness node RAM. This is not actually a lot of NFT's, if this becomes popular. At present, BP-only witness node consumes around 4GB of RAM, so this would be a 25% increase for for BP's for a mere 2,621 NFTs.

Aside from offloading storage requirement by storing links to content instead of actual content data, which has a disadvantage of making the NFT reliant on hosting through, e.g., ipfs, one option to reduce the RAM hit might be to publish the NFT with full artwork, but then in a subsequent block to replace the nft_object field contents and replace with an object that refers to the block number in which the original publication occurred. This means that witness and API nodes would no longer need to keep the full NFT object in RAM (they would instead keep the truncated form), and yet the full data would still be in the blockchain. A viewer wishing to retrieve the data would no longer find it in the asset description, but would be directed to retrieve the relevant block instead, and to extract the data from there. In this scheme, the NFT signature would still be in the asset description, but the NFT object would not. However, it would be retrievable without having to rely on off-chain storage.

[christophersanborn]

Some questions to pursue:

1. Are asset descriptions in fact stored in RAM?

2. Are asset description important to consensus? My suspicion is they are irrelevant to consensus.

3. If they are not important to consensus, this would mean that they could be dropped from RAM for non-API (e.g. BP) nodes, thus reducing the RAM burden without having to implement a scheme of replacing the description with a truncated form. This would be good for BP nodes. However API nodes, including end-user personal nodes, would still need to store this data in RAM, although these nodes are already expected to have higher requirements due to storing history.

4. Would implementing disk cacheing for "large" description fields be a solution? (As in, a simpler or more elegant solution than establishing a convention of removing payload from NFT objects and replacing with ref to block:tx:op.)

[christophersanborn]

@abitmore — would be particularly interested in your commentary here.

[abitmore]

Thank you @christophersanborn. My comments are below.

Some questions to pursue:

1. Are asset descriptions in fact stored in RAM?

Yes.

2. Are asset description important to consensus? My suspicion is they are irrelevant to consensus.

Not very important IMHO.

3. If they are not important to consensus, this would mean that they could be dropped from RAM for non-API (e.g. BP) nodes, thus reducing the RAM burden without having to implement a scheme of replacing the description with a truncated form. This would be good for BP nodes. However API nodes, including end-user personal nodes, would still need to store this data in RAM, although these nodes are already expected to have higher requirements due to storing history.

Correct.

4. Would implementing disk cacheing for "large" description fields be a solution? (As in, a simpler or more elegant solution than establishing a convention of removing payload from NFT objects and replacing with ref to block:tx:op.)

Maybe. I think perhaps it's too early to think about this. Hardware is cheap nowadays. I think we will have time to react when it's getting serious.

[iamredbar]

Thank you for the information @abitmore. Instead of replacing the description after creation, we could keep the NFT data in the description as long as we can drop it from RAM on the BPs. I am guessing this would require a core change?

[abitmore]

Yes, a change in core because the witnesses are running the core software. But it's not a hard fork. So it can be developed and deployed easily and quickly when needed.

[dls-cipher]

Few remarks:

• Adding asset ownership and details to witnesses endorse violation of GDPR and other privacy laws.
• Adding asset details to a witness directly impose legal responsibility towards witness and the data of someone who is 1:1 owner of an item.
• Why not just store .json on the website.hosting for all Different implementations of NFT. That case witness has no responsibility and KYC and rest that will come to that ownership part from regulators was judged already yesterday :)

Chee®s

[grctest]

What about encouraging uploading JSON to IPFS which describe the images/multimedia, to reduce the fees and the compute load? Rather than include 50+ strings in media_png_multihashes and potentially reaching the max blocksize, I could do media_json_multihash & the contents of the json at the multihash are then processed.

I believe this is what a lot of ETH projects do to reduce costs