Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/verify_and_publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
python-version: ['3.9', '3.10', '3.11', '3.12']
python-version: ['3.10', '3.11', '3.12', '3.13']

steps:
- name: Grab source
Expand Down Expand Up @@ -159,7 +159,7 @@ jobs:
coverage run \
$(which cijoe) --monitor -l \
--config cijoe-example-${{ matrix.usage_example }}/cijoe-config.toml \
cijoe-example-${{ matrix.usage_example }}/cijoe-workflow.yaml
cijoe-example-${{ matrix.usage_example }}/cijoe-task.yaml

- name: Coverage report
run: |
Expand Down Expand Up @@ -292,11 +292,11 @@ jobs:
matrix:
cli:
- name: "integrity-check"
cmd: "$(which cijoe) example/cijoe-workflow.yaml -c example/cijoe-config.toml --integrity-check"
cmd: "$(which cijoe) example/cijoe-task.yaml -c example/cijoe-config.toml --integrity-check"
- name: "script-target"
cmd: "$(which cijoe) core.example_script_default -c example/cijoe-config.toml"
- name: "workflow-steps"
cmd: "$(which cijoe) example/cijoe-workflow.yaml -c example/cijoe-config.toml script_with_args"
- name: "task-steps"
cmd: "$(which cijoe) example/cijoe-task.yaml -c example/cijoe-config.toml script_with_args"
- name: "emit-example"
cmd: "$(which cijoe) --example core.default"
- name: "archive"
Expand Down Expand Up @@ -358,7 +358,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5.2.0
with:
python-version: '3.9'
python-version: '3.10'

- name: Python dependencies and their versions
run: |
Expand Down
2 changes: 1 addition & 1 deletion docs/emit.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
{{ "=" * pkg_name_len }}

These are the scripts provided in the package, they are listed by the **full**
name that you can use to refer to them in a workflow.
name that you can use to refer to them in a task.

Scripts
-------
Expand Down
16 changes: 8 additions & 8 deletions docs/source/050_usage_help.out
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ options:
-h, --help show this help message and exit

run:
Options for running a workflow script.
Options for running a task script.

target A cijoe workflow or script to run.
step Given a workflow, the steps of the workflow it
target A cijoe task or script to run.
step Given a task, the steps of the task it
should run. If none are given, all steps are
run.
--config CONFIG, -c CONFIG
Expand All @@ -25,22 +25,22 @@ run:
'-ll' for debug. (default: None)
--monitor, -m Dump command output to stdout (default: False)
--no-report, -n Skip the producing, and opening, a report at the
end of the workflow-run (default: False)
end of the task-run (default: False)
--skip-report, -s Skip the report opening at the end of the
workflow-run (default: True)
--tag TAG, -t TAG Tags to identify a workflow-run. This will be
task-run (default: True)
--tag TAG, -t TAG Tags to identify a task-run. This will be
prefixed while storing in archive (default:
None)

utilities:
Workflow, and workflow-related utilities
Task, and task-related utilities

--archive, -a Move the output at '-o / --output' to 'cijoe-
archive/YYYY-MM-DD_HH:MM:SS (default: False)
--produce-report, -p Produce report, and open it, for output at '-o /
--output' and exit. (default: None)
--integrity-check, -i
Check integrity of workflow given as positional
Check integrity of task given as positional
argument and exit. (default: False)
--resources, -r List collected resources and exit. (default:
False)
Expand Down
6 changes: 3 additions & 3 deletions docs/source/200_quickstart.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ pipx install cijoe
# Print a list of bundled usage examples
cijoe --example

# Produce example script, config, and workflow
# Produce example script, config, and task
cijoe --example core.default

# Execute the workflow
cijoe cijoe-example-core.default/cijoe-workflow.yaml \
# Execute the task
cijoe cijoe-example-core.default/cijoe-task.yaml \
--config cijoe-example-core.default/cijoe-config.toml

4 changes: 2 additions & 2 deletions docs/source/300_lint.cmd
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Check format of workflow and verify existance of the scripts used
# Check format of task and verify existance of the scripts used
cijoe --integrity-check \
--config cijoe-example-core.default/cijoe-config.toml \
cijoe-example-core.default/cijoe-workflow.yaml
cijoe-example-core.default/cijoe-task.yaml
3 changes: 3 additions & 0 deletions docs/source/420_usage_task_all.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Run the task
cijoe cijoe-example-core.default/cijoe-task.yaml \
--config cijoe-example-core.default/cijoe-config.toml
3 changes: 0 additions & 3 deletions docs/source/420_usage_workflow_all.cmd

This file was deleted.

4 changes: 4 additions & 0 deletions docs/source/450_usage_task_subset.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Run a subset of the steps in the task
cijoe cijoe-example-core.default/cijoe-task.yaml \
--config cijoe-example-core.default/cijoe-config.toml \
inline_commands
4 changes: 0 additions & 4 deletions docs/source/450_usage_workflow_subset.cmd

This file was deleted.

37 changes: 21 additions & 16 deletions docs/source/configs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ provided, the reason being that with **cijoe**, the core interface of
configured to do so, ``run()`` executes commands on a **target** machine that
is not the **initiator**.

Often, tasks performed by **cijoe** can be, if not destructive, modifying
the **target** system in one way of the other. The default is for the
**initator** and the **target** to be the same system, and therefore, to avoid
accidents like invoking a destructive workflow and forgetting to supply a
Often, tasks performed by **cijoe** can be, if not destructive, modifying
the **target** system in one way of the other. The default is for the
**initator** and the **target** to be the same system, and therefore, to avoid
accidents like invoking a destructive task and forgetting to supply a
configuration file, providing a configuration file in the command line tool
is required.
is required.

**cijoe** specific keys
=======================
Expand All @@ -32,7 +32,7 @@ you should avoid using for other things than the intended purpose:

* SSH Transport Configuration
* Shell Configuration
* Workflow Configuration
* Task Configuration

**cijoe** abides by the convention that configuration values are grouped under
a relevant key. Thus, the those for **cijoe** itself are all stored under the
Expand All @@ -46,7 +46,7 @@ a relevant key. Thus, the those for **cijoe** itself are all stored under the
[cijoe.run]
...

[cijoe.workflow]
[cijoe.task]
...


Expand Down Expand Up @@ -154,27 +154,32 @@ In the table below, you can see the keys associated with different shells in
- ``csh``


.. _sec-resources-configs-workflow:
.. _sec-resources-configs-task:

Workflow Configuration
----------------------
Task Configuration
------------------

When a workflow is processed, each step is executed in sequence. It can be
advantageous to *fail fast*, meaning the workflow stops processing further steps
When a task is processed, each step is executed in sequence. It can be
advantageous to *fail fast*, meaning the task stops processing further steps
once a failure occurs in any step.

This is possible via the **cijoe** configuration option:
``cijoe.workflow.fail_fast`` which you can set in your
``cijoe.task.fail_fast`` which you can set in your
:ref:`configuration-file <sec-resources-configs>`:

.. literalinclude:: ../../../src/cijoe/core/configs/example_config_default.toml

By default **cijoe** functionality, then this can also be controlled environment
variable:
As with any **cijoe** configuration option, this can also be controlled via an
environment variable:

.. code-block:: bash

CIJOE_WORKFLOW_FAIL_FAST=true
CIJOE_TASK_FAIL_FAST=true

.. note::
The legacy ``[cijoe.workflow]`` section, including
``cijoe.workflow.fail_fast`` (and ``CIJOE_WORKFLOW_FAIL_FAST``), is still
honored when ``[cijoe.task]`` is not set, but emits a deprecation warning.


.. _sec-resources-configs-evar-override:
Expand Down
34 changes: 17 additions & 17 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,28 @@
==================================================

**cijoe** is a tool designed to collect and formalize the common elements used
during systems development, enabling an **automated** and **reproducible**
workflow.
during systems development, enabling **automated** and **reproducible**
execution.

When using **cijoe**, the first step involves creating :ref:`scripts
<sec-resources-scripts>` for automating repetitive tasks. As these :ref:`scripts
<sec-resources-scripts>` to automate repetitive work. As these :ref:`scripts
<sec-resources-scripts>` are developed, all input values are stored in
and passed through :ref:`configuration files <sec-resources-configs>` and
environment variables.
By keeping input values separate from the script itself, tasks can be easily
By keeping input values separate from the script itself, scripts can be easily
replicated across different environments, ensuring flexibility and consistency
in execution.

As your script collection grows, **cijoe** allows you to organize them
into :ref:`workflows <sec-resources-workflows>`.
A :ref:`workflow <sec-resources-workflows>` consolidates the
into :ref:`tasks <sec-resources-tasks>`.
A :ref:`task <sec-resources-tasks>` consolidates the
:ref:`script <sec-resources-scripts>` execution sequence, documents their
combined purpose, and provides usage instructions. This clarity makes it easier
for others to understand and consistently execute the workflow in their own
for others to understand and consistently execute the task in their own
environments.

After execution, **cijoe** generates a report that includes command output,
script documentation, auxiliary collected artifacts, and a workflow summary.
script documentation, auxiliary collected artifacts, and a task summary.
This report facilitates collaboration by providing results transparently, making
them easy to **review** and **reproduce**.

Expand All @@ -43,10 +43,10 @@ errors occur. Options for customizing the behavior include:

- ``-n`` / ``--no-report``: Disable the generation and display of the HTML report.
- ``-l`` / ``--log-level``: Increase verbosity by setting a higher log level.
- ``-m`` / ``--monitor``: Enable real-time monitoring of the workflow.
- ``-m`` / ``--monitor``: Enable real-time monitoring of the task.

For a detailed description of usage see ``cijoe --help`` and refer to the rest
of the documentation, and good place to continue is with :ref:`sec-usage`.
of the documentation; a good place to continue is with :ref:`sec-usage`.

.. _sec-introduction:

Expand All @@ -73,13 +73,13 @@ Unlike configuration management tools, **cijoe** is a minimal, open-ended
scripting tool that emphasizes maintainability, reusability, and built-in
reporting for sharing results, including command output and artifacts.

**cijoe** is designed to execute commands, scripts, or workflows
**cijoe** is designed to execute commands, scripts, or tasks
within continuous integration (CI) environments such as
:github:`GitHub <>`, :gitlab:`GitLab <>`, :travis:`Travis CI <>`,
and :jenkins:`Jenkins <>`.
It also allows for seamless execution of the same scripts on local systems,
enabling developers to switch between CI providers while maintaining the ability
to run automated tasks locally.
to run the same automation locally.

.. figure:: _static/cijoe-networked.drawio.png
:alt: Development Environment
Expand Down Expand Up @@ -128,10 +128,10 @@ Key Features

- **Realtime Output**:

- Whether **cijoe** is executing scripts of workflows, then it **can**
- Whether **cijoe** is executing scripts or tasks, then it **can**
provide you with realtime command output, directly in your console.
- When running in cloud environments e.g. GitHub Actions, GitLab, Azure,
Travis, etc. then it is very convient to immediately observe execution
Travis, etc. then it is very convenient to immediately observe execution
progress

- **Postprocessing**:
Expand All @@ -142,7 +142,7 @@ Key Features

- **Reporting**:

- All data from **runs**, script and workflow executions, is collected in a
- All data from **runs**, script and task executions, is collected in a
**HTML** report. Conveniently viewable even when running on remote systems

In summary, **cijoe** aims to be a simple yet powerful tool that integrates
Expand All @@ -151,7 +151,7 @@ without adding complexity.

Once you have ensured that the system prerequisites (:ref:`sec-prerequisites`)
are met, proceed to the :ref:`sec-usage` section to run an example
script and workflow. For documentation on how to create your own scripts,
script and task. For documentation on how to create your own scripts,
see :ref:`sec-resources`. Finally, refer to :ref:`sec-packages` for descriptions
of existing script collections and related packages.

Expand All @@ -162,7 +162,7 @@ of existing script collections and related packages.

usage/index.rst
scripts/index.rst
workflows/index.rst
tasks/index.rst
configs/index.rst
testrunner/index.rst
resources/index.rst
Expand Down
2 changes: 1 addition & 1 deletion docs/source/packages/core/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ core
====

These are the scripts provided in the package, they are listed by the **full**
name that you can use to refer to them in a workflow.
name that you can use to refer to them in a task.

Scripts
-------
Expand Down
2 changes: 1 addition & 1 deletion docs/source/packages/linux/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ linux
=====

These are the scripts provided in the package, they are listed by the **full**
name that you can use to refer to them in a workflow.
name that you can use to refer to them in a task.

Scripts
-------
Expand Down
2 changes: 1 addition & 1 deletion docs/source/packages/qemu/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ qemu
====

These are the scripts provided in the package, they are listed by the **full**
name that you can use to refer to them in a workflow.
name that you can use to refer to them in a task.

Scripts
-------
Expand Down
2 changes: 1 addition & 1 deletion docs/source/packages/system_imaging/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ system_imaging
==============

These are the scripts provided in the package, they are listed by the **full**
name that you can use to refer to them in a workflow.
name that you can use to refer to them in a task.

Scripts
-------
Expand Down
2 changes: 1 addition & 1 deletion docs/source/resources/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
===========

In **cijoe** the most essential **resources** are :ref:`sec-resources-scripts`,
:ref:`sec-resources-workflows`, and :ref:`sec-resources-configs`.
:ref:`sec-resources-tasks`, and :ref:`sec-resources-configs`.
In addition to these are :ref:`Auxiliary files <sec-resources-auxiliary>`, and
:ref:`sec-resources-templates`.

Expand Down
Loading
Loading