|
1 | | -# Running task inside a docker container |
| 1 | +# Running a Task Inside a Docker Container |
2 | 2 |
|
3 | | -> **WARNING**: This method is not meant for the production use and is meant only as a POC to test docker images as a part of existing pipeline. Right now it was tested with outdated alma8-flp-node image running readout |
| 3 | +> ⚠️ **Warning** |
| 4 | +> This method is **not intended for production use**. |
| 5 | +> It serves only as a **proof of concept (POC)** for testing Docker images as part of an existing pipeline. |
| 6 | +> |
| 7 | +> Currently, it has been tested with the `alma9-flp-node` image running the *readout* component. |
4 | 8 |
|
5 | | -## How to |
| 9 | +--- |
6 | 10 |
|
7 | | -As a first step we need to be sure that required host computer has installed `docker`. At the time of writing this document `docker` needs to be installed manually. |
| 11 | +## How To |
8 | 12 |
|
9 | | -In order to run task inside a docker image on the executor we can simply wrap the binary call into the docker image inside the [ControlWorkflow](https://github.com/AliceO2Group/ControlWorkflows) repository. For example in order to run readout we can modify `_plain_command` part of [`readout.yaml`](https://github.com/AliceO2Group/ControlWorkflows/blob/master/tasks/readout.yaml) by adding `docker run image command`. Obviously we need to have docker image that contains required binary with proper settings (creating one is outside of this document). A bit tricky part is that we need to manually (for now) specify all ENV variables with `-e` option in `docker run` call. Moreover we might need to add `--network=host` and `--ipc=host` to the call itself. Michal Tichak was able to run readout inside the alma8-flp-node as a part of workflow using following: |
| 13 | +### 1. Manual Setup |
10 | 14 |
|
11 | | -``` |
12 | | -"/usr/bin/docker run --network=host --ipc=host -v /tmp:/tmp -e O2_DETECTOR={{ detector }} -e O2_PARTITION={{ environment_id }} -e OCC_CONTROL_PORT=31000 -e O2_SYSTEM=FLP -e O2_ROLE=mtichak-ostack gitlab-registry.cern.ch/aliceo2group/dockerfiles/alma8-flp-node:latest /opt/o2/bin/o2-readout-exe" |
| 15 | +Before running tasks in Docker, ensure that the host machine has **Docker** installed. |
| 16 | +At the time of writing, Docker must be installed **manually**. |
| 17 | + |
| 18 | +> ⚠️ **Security Note** |
| 19 | +> The `flp` user must be able to run `sudo` **without a password**, because Docker requires root privileges. |
| 20 | +> |
| 21 | +> This setup is **not safe for production systems**. |
| 22 | +
|
| 23 | +Run the following commands as `root`: |
| 24 | + |
| 25 | +```bash |
| 26 | +usermod -aG wheel flp |
| 27 | +echo '%wheel ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/90-wheel-nopasswd |
13 | 28 | ``` |
14 | 29 |
|
15 | | -In order to figure out all of the ENV variables required one can take a look into the ECS gui environment details page and find task in question where all of the env variables are defined while running the binary outside of docker. |
| 30 | +--- |
16 | 31 |
|
17 | | -## Tips and tricks |
| 32 | +### 2. Modifying ControlWorkflows |
18 | 33 |
|
19 | | -Production systems run RHEL which doesn't install native `docker` by running `dnf install docker` but they are emulating the functionality by using `podman` which might behave different. |
| 34 | +To run a task inside a Docker container on the executor, wrap the binary call in a `docker run` command within the [ControlWorkflows](https://github.com/AliceO2Group/ControlWorkflows) repository. |
20 | 35 |
|
21 | | -In order to debug whether ECS is even starting the container we can use |
| 36 | +For example, to run **readout**, modify the `_plain_command` section of [`readout.yaml`](https://github.com/AliceO2Group/ControlWorkflows/blob/master/tasks/readout.yaml) by adding a Docker command. |
22 | 37 |
|
23 | | -``` |
24 | | -docker ps -a |
25 | | -``` |
| 38 | +> 🧩 **Note** |
| 39 | +> You must already have a Docker image that includes the required binary and configuration. |
| 40 | +> (Creating such an image is **outside the scope** of this document.) |
26 | 41 |
|
27 | | -This will show all of the containers which were run/are runinng on the system under current user. However there is a catch: ECS is using user `flp` so in order to figure out which container was running under this user, we need to switch the user by |
| 42 | +#### Example Command |
28 | 43 |
|
29 | | -``` |
30 | | -su - flp |
| 44 | +When running *readout*, **Michal Tichak** successfully used the following command inside the `alma9-flp-node` image: |
| 45 | + |
| 46 | +```bash |
| 47 | +sudo /usr/bin/docker run --name readout --replace \ |
| 48 | + --user "$(id -u flp):$(id -u flp)" \ |
| 49 | + --network=host --ipc=host \ |
| 50 | + -e O2_DETECTOR -e O2_PARTITION -e OCC_CONTROL_PORT \ |
| 51 | + -e O2_SYSTEM -e O2_ROLE \ |
| 52 | + gitlab-registry.cern.ch/aliceo2group/dockerfiles/alma9-flp-node:2 \ |
| 53 | + /opt/o2/bin/o2-readout-exe |
31 | 54 | ``` |
32 | 55 |
|
33 | | -You can show logs directly from docker itself by using: |
| 56 | +#### Environment Variables |
34 | 57 |
|
| 58 | +To identify all required environment variables: |
| 59 | + |
| 60 | +1. Open the **ECS GUI**. |
| 61 | +2. Go to the **Environment Details** page for the relevant task. |
| 62 | +3. Review the variables defined there — these match those used when running the binary outside Docker. |
| 63 | + |
| 64 | +#### Shared Memory Communication |
| 65 | + |
| 66 | +To enable shared memory communication between processes, add the `--ipc=host` flag when running the container. |
| 67 | +However, doing so requires **elevated privileges**. |
| 68 | + |
| 69 | +While **Podman** can run without root privileges, it pauses other Podman processes for the same user. |
| 70 | +This means commands like `podman ps -a` or starting multiple containers in parallel will not work. |
| 71 | + |
| 72 | +Therefore, you should run containers using the same user as the rest of the pipeline: |
| 73 | + |
| 74 | +```bash |
| 75 | +--user "$(id -u flp):$(id -u flp)" |
35 | 76 | ``` |
36 | | -docker logs [container-id|name] |
37 | | -``` |
| 77 | + |
| 78 | +This ensures shared memory segments are created under the same user context. |
| 79 | + |
| 80 | +--- |
| 81 | + |
| 82 | +## Tips and Tricks |
| 83 | + |
| 84 | +* Production systems running **RHEL** do not install native Docker via: |
| 85 | + |
| 86 | + ```bash |
| 87 | + dnf install docker |
| 88 | + ``` |
| 89 | + |
| 90 | + Instead, they use **Podman**, which emulates Docker’s behavior but may differ in certain aspects. |
| 91 | + |
| 92 | +* To check whether ECS has started a container, run: |
| 93 | + |
| 94 | + ```bash |
| 95 | + docker ps -a |
| 96 | + ``` |
| 97 | + |
| 98 | + This lists all containers that have run (or are currently running) under the current user. |
| 99 | + |
| 100 | + > ECS typically runs as the `flp` user, so to inspect its containers, switch users first: |
| 101 | + > |
| 102 | + > ```bash |
| 103 | + > su - flp |
| 104 | + > ``` |
| 105 | +
|
| 106 | +* To view container logs directly from Docker: |
| 107 | +
|
| 108 | + ```bash |
| 109 | + docker logs <container-id|name> |
| 110 | + ``` |
| 111 | +
|
| 112 | +--- |
0 commit comments