Improving CM script automation and CM scripts

[gfursin]

Aggregating tasks to improve CM script automation and CM scripts based on user feedback (this ticket is being gradually updated and tasks resolved based on our bandwidth and engineering resources):

Automation

• Add --silent / -s mode to avoiding printing CM workflow execution
• Add cm_min_version check for automation and scripts
• Check current dumping of all versions of all dependencies Commit
• Check readme generation with all deps Commit
  • Add --version when available Commit
• Prepare tmp_run_final_script.sh and stop to let user run it manually without CM if needed
• Generate sample Docker container during --repro
• Add cfg to load default env, state and keys for all CM scripts
  • cm set cfg default --key.cm-script.silent --key.cm-script.env.CM_SUDO="no" ...
  • Add silent mode as default
  • Support add/update cfg keys ...
• Add cm status to show version, paths to current repositories and if the version is up-to-date (useful for virtual environments)
• Add to meta where to report errors if not default repository (for MLPerf inference -> report https://github.com/mlcommons/inference)
• Add proper version detection for cuDNN

Documentation

• Check individual documentation and input description for all main CM scripts
• Add individual tests for all main CM scripts

Tutorials

Need to explain how to extend CM scripts, add new ones, unify inputs, etc)

• Prepare tutorial about CM basics and CM scripts
• Prepare tutorial about CM automation for basic inference
• Prepare tutorial about CM automation for MLPerf loadgen
• Prepare tutorial about CM automation for MLPerf inference
• Prepare tutorial about CM automation for ABTF
• Prepare tutorial about CM automation for SCC'24

Longer term

• Use logging instead of print
• Refactor and simplify CM script automation - it was heavily prototyped and it is stable but the implementation can be considerably improved and simplified
• Refactor docker container generation and execution - the prototype implementation is very complex and can be dramatically simplified