Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
synapse-dev | fece11f | Commit Preview URL Branch Preview URL |
Mar 31 2026, 04:16 PM |
|
@rvagg or @hugomrdias : can you please review so we get this in for M4.1? |
| :::tip[SDK Usage] | ||
| This cookbook explains the underlying mechanics. For practical usage: | ||
|
|
||
| - **Code examples**: [Storage Costs](/developer-guides/storage/storage-costs/) |
There was a problem hiding this comment.
should we be using / anchored links in these docs? wouldn't it be safer to use relative links? I don't really know Astro but this seems brittle to me
There was a problem hiding this comment.
this https://github.com/HiDeoo/starlight-links extension helps a lot with links and we have https://github.com/HiDeoo/starlight-links-validator in the build
There was a problem hiding this comment.
Good job pulling it together, the old doc was one big hand-wave so important to improve on it. But this new doc needs a framing adjustment: it reads like Shashank's design doc for getCosts() turned into a user-facing doc. That doc was internal detail for building an API that helps us hide all this complexity from the user, but then presenting it here reintroduces it. "Account state", "rate precision", "the buffer problem", these in particular are far far too in-depth for user-facing docs, even docs we expect AI to read.
What an SDK developer needs from a "Payments & Storage" cookbook:
- How much does storage cost (pricing table, floor, CDN)
- What happens when I call
prepare()/getUploadCosts()(already in storage-costs.mdx) - What happens if I run out of funds (high-level: 30-day grace, then data at risk)
- Most importantly, the mental model that they need to function with this system: you deposit, FWSS streams payments to SPs, done
I think there is scope to go into some detail about Filecoin Pay's dynamics, but strictly if it's framed as "advanced", and a lot of what's in here goes beyond that. Account struct fields are not relevant. actualLockup is too much detail when FWSS is a 30-day fixed lockup, conceptually it's much simpler. Solidity source for _calculateStorageRate is way too deep, solidity modifier behaviour like settleAccountLockupBeforeAndAfter, the whole buffer problem and the pseudocode, all the edge case details about terminations and even most of the debt stuff is too deep. These kinds of things are useful for us to understand and to frame how we build an interaction layer with the system for the user, but if we're needing to give them to the user then we may as well not have an SDK.
The old cookbook was one big hand-wave, so we need to find the middle ground. I think we could trim it to about 1/3rd of this this length: keep pricing model (rates, floor, CDN costs), the three-rail-per-dataset mental model, "what underfunded means for you" (keep it brief), operator approval (brief, for storage we just have the one to approve but knowing what that means is relatively important), and point to storage-costs.mdx for the code. Cut everything about internal formulas, Solidity source, buffer pseudocode, and settlement mechanics.
Even though you've got solid context here and good accuracy, when you're working on this further, I'd suggest you load the filecoin-pay README.md and SPEC.md into your agent's context to get full accuracy, just don't be lured by your AI to go into deep detail, it's for accuracy and this needs to be external developer facing.
Also, you need to account for #666; the data set creation fee is back, so it's now 0.16 USDFC (no CDN) and 1.16 USDFC (with CDN), not 0.06/1.06.
The guiding principle: the less a developer has to know about internals, the better. Storage should be something you bolt onto an app, not something that requires understanding contract settlement to use.
There was a problem hiding this comment.
A few things have gone awry in here I think.
I rewrote this doc as part of the multi-copy upload flow and intentionally tried to downplay use of StorageContext directly except inasmuch as it gives you the ability to do the split operations to take control of the upload operation because you have opinions or needs that upload() doesn't cover. In a sense, StorageContext becomes just a minimally stateful mechanism to do store() / pull() and commit() across multiple providers and coordinate the dance yourself. Aside from that there's not a great reason to use it directly. I even debated removing StorageContext#upload() entirely. But you've restored a lot of the stuff that I intentionally removed, in revamped form. I would like StorageContext to become more of an implementation detail that you need to know about for split operations and deeper control than something that needs a dedicated page.
The whole section on "Using synapse-core Directly" has been removed, was this a mistake? The original framing of the docs for storage were: (1) golden path with synapse.storage.upload(), (2) split operations with manual StorageContext creation and coordinating the store() / pull() and commit() dance yourself and then (3) using synapse-core directly to do the same thing, which removes state and adds some complexity due to provider selectio nconcerns, but is not too much more complex than plain split operations.
I don't mind if you delete or rename this file, but I wouldn't mind keeping the layered framing while also gently downplaying StorageContext as a special concern. There is a future where we just make StorageContext go away because as we've trimmed it down we've reduced the difference between using it and using synapse-core directly.
There was a problem hiding this comment.
The main reason I split StorageContext out was practical. The original page was too long and the flow was hard to follow, it opened with split ops + diagram, then detoured into context creation and matching logic before getting to the split examples.
Also StorageContext is not just an upload concept, it's used as input across uploads, storage costs, dataset management and other SDK apis. So I felt it can have its own page.
I even debated removing
StorageContext#upload()entirely.
Agreed, can remove and point to split operations.
The whole section on "Using synapse-core Directly" has been removed, was this a mistake?
Intentional. My plan was to move it to the synapse-core page, but I can bring it back into split-operations to keep the layered framing (golden path → split ops → synapse-core direct). What's your preference?
There is a future where we just make
StorageContextgo away
I get that and I can see the SDK moving internally from context to core, but I feel like we are not there yet. This is something we can improve incrementally as the SDK evolves. Let me know how you'd like to proceed.
There was a problem hiding this comment.
- move synapse-core into the synapse-core page
- we already explain "upload" in multiple places! We should have just one place!
- Golden path is here https://4bd803e7-synapse-dev.filoz.workers.dev/getting-started/#3-store-and-download-data from here we should link to a "Upload Pipeline" page
- Upload Pipeline goes through
- https://4bd803e7-synapse-dev.filoz.workers.dev/developer-guides/storage/storage-operations/#storage-approaches
- https://4bd803e7-synapse-dev.filoz.workers.dev/developer-guides/storage/storage-operations/#upload-with-callbacks
- https://4bd803e7-synapse-dev.filoz.workers.dev/developer-guides/storage/storage-context/#uploading-data
- https://4bd803e7-synapse-dev.filoz.workers.dev/developer-guides/storage/split-operations/
- at the end links to https://4bd803e7-synapse-dev.filoz.workers.dev/developer-guides/synapse-core/#storage
- Merge https://4bd803e7-synapse-dev.filoz.workers.dev/developer-guides/storage/storage-operations/ and https://4bd803e7-synapse-dev.filoz.workers.dev/developer-guides/storage/storage-context/ into one with a CTA saying "For upload stuff go to Upload Pipeline page". Call it Storage Operations and downplay "context" as much as you can.
| ## Upload Preparation | ||
|
|
||
| --- | ||
| Before uploading, the **WarmStorage operator** must be approved and your account must be funded. FWSS requires a 30-day prepayment buffer — when your balance drops below 30 days, the provider may remove your data. |
There was a problem hiding this comment.
This was already in here but it's misleading and we should take the change to tweak it, it's more like "FWSS uses a 30-day lockup (prepayment) period. If your account becomes underfunded, the provider may terminate the storage agreement, after which you have 30 days before data removal."
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Rod Vagg <rod@vagg.org>
|
@nijoe1 Apart from getting this branch up to date with the base branch - is this PR ready for a re-review? |
|
@rjan90 I am waiting for one answer from Rod. Then I can work on the refinements. Most of the other comments I have resolved them just haven't pushed as they might require some small changes based on Rod's resp! |
|
@nijoe1 ping me in slack if you need to chat about this |
6 participants
filecoin-paycookbook with a comprehensive Payments & Storage cookbook covering the account model, payment rails, rate mechanics, and deposit calculations. Incorporates content from the HackMD writeup. Closes Update Filecoin Pay cookbook #651