Curious about Roadmap/Upcoming Plans

[JeffWang987]

Hi @lucidrains!

I'm very excited to see the progress and innovation here. I wanted to reach out to ask if there are any public plans, roadmap items, or upcoming features you're looking to work on for Value Network?

[lucidrains]

@JeffWang987 hey Jeff, glad to hear about your interest

i was going to start looking at some distributional value networks with gating by uncertainty, because of some thread i saw online

how about you?

[JeffWang987]

@lucidrains Thanks for sharing your direction, On my end, I've been following value modeling advances in embodied AI, particularly methods like Pi*0.6 and Cosmos Policy.

Happy to exchange thoughts if useful!

[lucidrains]

sure! yea happy to chat over Signal if that suits you

did they use a value network in Cosmos Policy? i must have missed that

[JeffWang987]

Ah, sorry, I can’t use Signal, but let’s keep this conversation going here instead!
To answer your question: Cosmos Policy skips the standalone value network. Instead, they use a generative network that predicts visual outputs, actions, and value all together in one go. Super neat.

[lucidrains]

@JeffWang987

yes i agree it is neat!

[lucidrains]

@JeffWang987 does something like this already exist out there? feels like you could know. some collaborative platform to annotate robotic trajectories for training the value network

[lucidrains]

@JeffWang987 this was the online thread that made me realize this could be an area of research to pay closer attention to. there's a few interesting works in there

[JeffWang987]

@JeffWang987 does something like this already exist out there? feels like you could know. some collaborative platform to annotate robotic trajectories for training the value network

From what I’ve seen, to annotate real robotic trajectories, we don’t need to label every single frame. just wait till the trajectory finishes and tag it as success or failure. Exactly like how they do it in π*0.6

We actually built our own in-house data collection and annotation platform for this.

Simulation envs like Libero auto-generate rewards (like collision checks, target distance, grip success), no manual work needed.

[lucidrains]

@JeffWang987 very informative! thank you!