Interface documentation

[hwiedPro]

Hey,
I'm trying to understand your code, especially how and where you are interfacing with bullet and caffe.
I'd be glad, if you could answer me some questions:
Where do you evaluate the current net in the simulation?
Where do you get the value used for reward out of the simulation?
Where and how do you hand the reward back to the solver?
Thanks for the great work and your help.
Best wishes

[hwiedPro]

Or may be you could draw a graph of the modules which explains in which module what happens, please? This would make understanding way easier, as your code is quiet interlaced.

[Neo-X]

There is some documentation in the Wiki and in https://github.com/xbpeng/DeepLoco/blob/master/documentation/ but it is incomplete.

[hwiedPro]

Thanks for your reply. I already found the LearningDiagram.dia but when opening it with Dia there are only 6 nodes but no connections or anything else. With which program did you create this? Is there anything missing?

My goal is to use your code with a different simulation tool. Therefore I was especially searching for the following things:

• Where do you start the simulation?
• Where do you write the torques to the simulation model?
• Where do you get the data from the simulation to calculate the reward?

I guess that procedural there should happen the following:

• (Initiating the controller network)
• Test the current controller/network outputs in simulation
• Calculate rewards from the simulation results
• Step the solver with those rewards
• Repeat
  But up to now it's hard to track where this is happening.

[Neo-X]

True, this library can be very complicated. It might help to check out our new version where many of the simulations are provided in python code https://github.com/UBCMOCCA/TerrainRLSim. The code can be used very similar to the way the openAIGym is constructed.
Here is a simple example.
https://github.com/UBCMOCCA/TerrainRLSim/blob/master/simAdapter/terrainRLTestMultiChar.py
If you look into how the C++ code is wrapped in python code you can figure out how and where the actions of the characters are updated.

[hwiedPro]

Thank you, I'll have a look in the next days.
I started to document your code in my fork.
Up to now I've only commented very basically the headers in "anim" in general and I'm going for the "learning" directory now. I would be happy if you help completing it. If you want I can add you as collaborator.

You implemented Trainers and Learners, but it doesn't look to me like this is already the AC approach. As this seems to be implemented later with Trainer and Learner as well. Do you have a hint for me? ;)
Thanks in advance!