Experiments: Update experiment limits page #2074
Conversation
🔗 Link Checker Results✅ All links are valid! No broken links were detected. Checked against: https://wb-21fd5541-experiments-limits-page.mintlify.app |
📚 Mintlify Preview Links📝 Changed (1 total)📄 Pages (1)
🤖 Generated automatically when Mintlify deployment succeeds |
|
@ngrayluna I think the clarity is improved here, thanks, You may want to check with @dawita66 on the best guidance to give on the numbers right now. For example as we roll out v2, it's no longer so cut and dried between SaaS and Dedicated/Customer Managed. Some Dedicated do have v2 and with the mysql size (not a factor mentioned on this page, but relevant), can do well above 10,000 runs performantly. Meanwhile non-paid SaaS do not have v2 on and can't do 100,000 runs. etc. |
| W&B organizes logged data along three dimensions: | ||
|
|
||
| Use `wandb.Run.log()` to track experiment metrics. | ||
| * **Steps**: The number of time steps in a run that you finalize by committing logged data. Each step represents a single time index (such as a training step or epoch) and is finalized when you call `wandb.Run.log()` with `commit=True`, or implicitly when `commit` and `step` are not specified. |
There was a problem hiding this comment.
"The number of time steps in a run" or just "The number of steps in a run"
| | Dimension | Guidance at scale | | ||
| |-----------------------|--------------------------------------------| | ||
| | Steps per run | Millions of steps per run are common | | ||
| | Distinct metrics | Fewer than 100,000 per run | |
There was a problem hiding this comment.
if a project has 10 runs, each run has 100K non-overlapping distinct metrics, then the project would have a total of 10 x 100K = 1M distinct metrics. The project page would not be able to load that because the per project limit is 100,000 distinct metrics.
however, if all the runs log to the same 100K metrics, then it would be okay to say it's "100,000 per run".
would it be more accurate to say "Fewer than 100,000 per project"?
There was a problem hiding this comment.
I am assuming this is for MT SaaS.
For dedicated instances, the distinct metric limits vary by instance size from our internal sizing benchmark. We probably don't want to publish them may be worthwhile to point out they are different from this MT SaaS number.
Description
Clarify how steps, metrics, and logged points relate to performance when logging to W&B. This PR also:
Related issues