Zephyr AI projects using CMSIS Stream

Introduction

Example of use of CMSIS Stream to implement AI + audio + video apps on Alif E7 AppKit board (it should work on E8 with some adaptation).

Several demos are implemented in this application and you can read more details about them here.

It is possible to context switch between the demos by using the Zephyr shell command switch, or by pressing the button, or by touching the screen.

Technical details about the demo are here.

This version of the demo uses the CMSIS Stream Zephyr module.

You can add it to your west file with:

- name: cmsisstream
  url: https://github.com/ARM-software/CMSIS-Stream
  revision: main
  path: modules/lib/cmsisstream

How to change the demo

The demo is built with CMSIS Stream. Each application contained in the demo is described with a CMSIS Stream graph.

A Python script is used to create such a graph.

By changing the graph, and connecting different nodes, the application can be modified.

To regenerate all the applications after changing a Python description you can do:

python aidemo.py gen --size --all

The --size option is a code size optimization explained below in the README.

If you are working on a specific application and want to update it just do:

python aidemo.py gen --size appa

There are 3 applications configured in this demo : appa, appb, appc.

It is also possible to use directly the application scripts without going through the aidemo script.

If you want to regenerate the kws demo (corresponding to the appa), you can do:

python -m python.kws --size

And if code size optimization is enabled, you need to regenerate the files common to all the apps:

python -m python.generate appa appb appc

It is important to list all the apps in this command.

You need version at least 3.1.0 of the CMSIS Stream Python package. It was recently updated so do a:

pip install cmsis-stream --upgrade

And to be sure the latest CMSIS Stream runtime is used, do a west update.

Code size optimizations

When using several graphs for several applications, the same C++ template may be instantiated several times (in each different app). Code size optimization is a feature of this demo implemented in python/generate.py. It is independent of CMSIS Stream but uses information generated by CMSIS Stream.

Code size optimization is enabled with the --size option. When enabled, and if the aidemo script is not used, you should not forget to regenerate the common parts shared between all apps in the demo.

This common part is src/streamgraph/common/template_instantiations.cpp containing instantiations of all templates used in all graphs. And the templates are no more instantiated in each graph.

You'll also need CONFIG_TEMPLATE_INSTANTIATIONS=y in your prj.conf file to build this additional file.

If code size optimization is not enabled, you rely on the linker to remove duplicate C++ template instantiations (and link time optimization should be enabled. It is not enabled in this demo).

Context switching

This demo allows to switch between several applications by using command switch in the Zephyr shell or by pressing the button on the board or touching the screen.

Context switching between different graphs is not a feature of CMSIS Stream. It is something built on top of it. But to make this use case easier, the Zephyr module for CMSIS Stream is exposing a tentative API.

Assumptions for context switching are:

• Several apps in memory
  • We not not destroy / create graphs to context switch.
  • The graphs remain in memory
  • It is not as bad as it looks from memory usage point of view since big buffers like display framebuffer of tensor arena are shared between graphs
• Threads are reused
  • Threads are not linked to a given graph
  • Instead a graph is deployed on existing threads
• Context switch is cooperative
  • Easier to implement with no constraint on how the drivers (audio, camera ...) have to behave when shared between graphs
  • It should not be a problem : for a real-time demo, node / event processing should not take too much time

Build issues

Issues

Zephyr cmakefile for cmsis-dsp module must be changed : zephyr/modules/cmsis-dsp/CMakeLists.txt

since Alif sdk is not yet using the latest Zephyr version.

Add this line:

zephyr_library_compile_definitions(ZEPHYR_INCLUDE_TOOLCHAIN_STDINT_H_)

It must be added before the zephyr_library_compile_definitions_ifdef to be able to compile with Helium.

You can also directly use the CMSIS-DSP repository to get the CMSIS-DSP Zephyr module instead of getting it through Zephyr.

Add this to the west file before the zephyr definition. Remove cmsis-dsp from the Zephyr name-allowlist.

 - name: cmsis-dsp
      url: https://github.com/ARM-software/CMSIS-DSP
      revision: main
      path: modules/lib/cmsis-dsp

Remove the modules/lib/cmsis-dsp before doing a west update to get the library from the CMSIS-DSP repository and the right remote.

Note that the CMSIS-DSP Zephyr module on the repo is not tested. CMSIS-DSP is tested but independently of Zephyr. If you want a tested module, get it from Zephyr.

Flash

It is possible to put the networks and other assets (like pictures) in the external flash.

The demo provides a container format that you can use from the C source to get access to objects written in flash.

To generate the container you can do:

python -m python.tools.create_bin ./src/networks/kws_micronet_m_vela_H128.tflite.cpp

You can also pass additional .bin files and the binary files do not have to be networks.

The script will generate a md5 hash. This hash must be copied into main.cpp since a check is done at startup to validate the content of the flash.

To generate binaries for pictures, you can do:

python -m python.tools.gen_img -b

You can package all bins with:

python -m python.tools.create_bin ./src/networks/kws_micronet_m_vela_H128.tflite.cpp assets/down.bin assets/go.bin assets/left.bin  assets/no.bin  assets/off.bin  assets/on.bin  assets/right.bin  assets/stop.bin  assets/up.bin  assets/yes.bin 

The order is important since in current version of the demo the indexes for pictures are hardcoded.

Flash with vscode and CMSIS debugger

You can use the vscode run task : Update flash container.

The flash must not be enabled in the Zephyr prj.conf when using this task.

Enable the flash in the configuration file only after the flash has been programmed with your content.

The vscode task will flash the file assets/container.bin generated by the script create_bin.