From ec51c456fbe2b5e05ee14ac9f5d5b32a51347f75 Mon Sep 17 00:00:00 2001 From: ckunki Date: Sun, 1 Feb 2026 14:32:36 +0100 Subject: [PATCH 1/5] #157: Added instructions for (Re-)using an external or local database to the README --- pytest-backend/README.md | 79 +++++++++++++++++------- pytest-backend/doc/changes/unreleased.md | 3 +- 2 files changed, 60 insertions(+), 22 deletions(-) diff --git a/pytest-backend/README.md b/pytest-backend/README.md index 1e3f290..a4648f9 100644 --- a/pytest-backend/README.md +++ b/pytest-backend/README.md @@ -1,8 +1,10 @@ # pytest-exasol-backend Plugin -The `pytest-exasol-backend` plugin is a collection of pytest fixtures commonly used for testing -projects related to Exasol. In particular, it provides unified access to both Exasol On-Prem and -SaaS backends. This eliminates the need to build different sets of tests for different backends. +Pytest plugin `pytest-exasol-backend` provides pytest fixtures for using various variants of Exasol database instances in integration tests. + +In particular, the plugin enables accessing both Exasol On-Prem and SaaS backends in a unified way. + +This enables your integration tests to use either backend variant without modification, incl. iterating the test for each backend variant. ## Features @@ -21,8 +23,9 @@ pip install pytest-exasol-backend ## Usage in Tests -Below is an example of a test that requires access to the database. Note, that by default -this test will run twice - once for each backend. +### Pyexasol Connection + +This test accesses the database via [Pyexasol](github.com/exasol/pyexasol). Note, that by default this test will run **twice** - once for each backend. ```python import pyexasol @@ -33,8 +36,9 @@ def test_number_of_rows_in_my_table(backend_aware_database_params): assert num_of_rows == 5 ``` -Here is an example of a test that requires access to the BucketFS. Again, this test will -run for each backend, unless one of them is disabled in the CLI. +### BucketFS + +This test accesses the [BucketFS](https://docs.exasol.com/db/latest/database_concepts/bucketfs/bucketfs.htm). Again, this test will run for each backend, unless one of them is disabled in the CLI. ```python import exasol.bucketfs as bfs @@ -45,8 +49,9 @@ def test_my_file_exists(backend_aware_bucketfs_params): assert my_bfs_file.exists() ``` -Sometimes it may be necessary to know which backend the test is running with. In such -a case the `backend` fixture can be used, as in the example below. +### Inspect the Selected Backend Variant + +If a tests wants to know, which backend it is running with, it can use the `backend` fixture, as shown below. ```python def test_something_backend_sensitive(backend): @@ -60,35 +65,67 @@ def test_something_backend_sensitive(backend): raise RuntimeError(f'Unknown backend {backend}') ``` -## Selecting Backends in CLI +## Selecting Backends on the Command Line + +By default, none of the backends is selected for testing. + +Please use the `--backend` option to specify the target backend with either `onprem`, `saas`, or `all`. + +The plugin automatically starts the selected backends and shuts them down after the test session has finished. +* A SaaS backend is started via [saas-api-python](https://github.com/exasol/saas-api-python/). +* An on-prem backend via the [ITDE](https://github.com/exasol/integration-test-docker-environment). +* Additionally you can [use an external or local database](#re-using-an-external-or-local-database). + +Please noe that all selected backends are started preemptively, regardless of their _actual usage_ in tests. -By default, none of the backends is selected for testing. Please use the `--backend` option to specify the target backend. -The command below runs the tests on an on-prem database. +Therefore, it is important to make sure the backends are not selected where they are not needed, for instance when running unit tests only. + +### Example Command Lines + +Run the tests on an on-prem database: ```shell pytest --backend=onprem my_test_suite.py ``` -This following command runs the test on two backends. +Run the tests on two backends: ```shell pytest --backend=onprem --backend=saas my_test_suite.py ``` -The next command runs the test on all backends, which currently is equivalent to the previous command since there -are only two backends available. +Run the test on all backends—equivalent to the previous command: ```shell pytest --backend=all my_test_suite.py ``` -Please note that all selected backends starts preemptively, regardless of their actual usage in tests. -Therefore, it is important to make sure the backends are not selected where they are not needed, -for instance when running unit tests only. +### (Re-)Using an External or Local Database + +During development you can shorten the time between code changes and running the tests by (re-)using a backend that is already running. + +To save 2-20 minutes each cycle, simply add CLI parameter `--itde-db-version=external`. + +Alternatively, you can export environment variable `PYTEST_ADDOPTS`, e.g. +```shell +export PYTEST_ADDOPTS="--backend=onprem --itde-db-version=external" +``` + +More parameters may be required if your setup deviates from the default values: + +| Option | Default value | +|-----------------------|------------------| +| `--exasol-host` | `localhost` | +| `--exasol-port` | `8563` | +| `--exasol-username` | `sys` | +| `--exasol-password` | `exasol` | +| `--bucketfs-url` | `127.0.0.1:2580` | +| `--bucketfs-username` | `w` | +| `--bucketfs-password` | (none) | -## Setting ITDE parameters in CLI +### Setting ITDE Parameters via CLI Options -Sometimes the default ITDE parameters cannot satisfy the test requirements. The plugin allows setting +Sometimes, the default ITDE parameters cannot satisfy the test requirements. The plugin allows setting some of the parameters of the [api.spawn_test_environment(...)](https://github.com/exasol/integration-test-docker-environment/blob/92cc67b8f9ab78c52106c1c4ba19fe64811bcb2c/exasol_integration_test_docker_environment/lib/api/spawn_test_environment.py#L35) function. The parameter values can be provided in the CLI options. Currently, it is possible to set values of the following parameters: - `--itde-db-mem-size` @@ -97,7 +134,7 @@ function. The parameter values can be provided in the CLI options. Currently, it - `--itde-additional-db-parameter` - `--itde-db-version` -In the example below the tests are run using an instance of the DockerDB with increased memory. +This example runs the tests using an instance of the DockerDB with increased memory. ```shell pytest --backend=onprem --itde-db-mem-size "8 GiB" my_test_suite.py diff --git a/pytest-backend/doc/changes/unreleased.md b/pytest-backend/doc/changes/unreleased.md index ecb975a..64bddcf 100644 --- a/pytest-backend/doc/changes/unreleased.md +++ b/pytest-backend/doc/changes/unreleased.md @@ -1,7 +1,8 @@ # Unreleased -## Internal +## Refactorings * #140: Ensured that a proper project-short-tag is used in SaaS tests. * #141: Added a Merge Gate to the CI Workflow. * #146: Relocked transitive dependency filelock +* #157: Added instructions for (Re-)using an external or local database to the README From 240efcd2ea59c7882abf53a6882e9d72a8b9b436 Mon Sep 17 00:00:00 2001 From: ckunki Date: Mon, 2 Feb 2026 09:42:36 +0100 Subject: [PATCH 2/5] Added proposals from ticket #150 --- pytest-backend/README.md | 23 ++++++++++++++++++++++- pytest-backend/doc/changes/unreleased.md | 2 +- 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/pytest-backend/README.md b/pytest-backend/README.md index a4648f9..cfc5de3 100644 --- a/pytest-backend/README.md +++ b/pytest-backend/README.md @@ -21,11 +21,32 @@ The pytest-exasol-backend plugin can be installed using pip: pip install pytest-exasol-backend ``` +Alternatively via poetry, recommended as `dev` dependency: + +```shell +poetry add pytest-exasol-backend --group dev +``` + ## Usage in Tests ### Pyexasol Connection -This test accesses the database via [Pyexasol](github.com/exasol/pyexasol). Note, that by default this test will run **twice** - once for each backend. +This test accesses the database via [Pyexasol](github.com/exasol/pyexasol) and requires an additional (`dev`) dependency to `pyexasol` to be added to your project. + +Note: If pytest option `--backend all` is specified, then this test will run **twice** - once for each backend. + +```python +import pyexasol + +def test_number_of_rows_in_my_table(backend_aware_database_params): + with pyexasol.connect(**backend_aware_database_params) as conn: + value = conn.execute('SELECT 1 FROM DUAL').fetchval() + assert value == 1 +``` + +The following test accesses a specific table within a database schema, and requires +* Database schema `MY_SCHEMA` and table `MY_TABLE` to exist +* Database table `MY_TABLE` to contain 5 rows ```python import pyexasol diff --git a/pytest-backend/doc/changes/unreleased.md b/pytest-backend/doc/changes/unreleased.md index 64bddcf..d113204 100644 --- a/pytest-backend/doc/changes/unreleased.md +++ b/pytest-backend/doc/changes/unreleased.md @@ -5,4 +5,4 @@ * #140: Ensured that a proper project-short-tag is used in SaaS tests. * #141: Added a Merge Gate to the CI Workflow. * #146: Relocked transitive dependency filelock -* #157: Added instructions for (Re-)using an external or local database to the README +* #150: Updated user guide and added instructions for (Re-)using an external or local database to the `README` From 5c19e7109a2abad62607b435ec4cd36ab4b399ad Mon Sep 17 00:00:00 2001 From: ckunki Date: Mon, 2 Feb 2026 09:43:42 +0100 Subject: [PATCH 3/5] Renamed sample test --- pytest-backend/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pytest-backend/README.md b/pytest-backend/README.md index cfc5de3..dd7d5f2 100644 --- a/pytest-backend/README.md +++ b/pytest-backend/README.md @@ -38,7 +38,7 @@ Note: If pytest option `--backend all` is specified, then this test will run **t ```python import pyexasol -def test_number_of_rows_in_my_table(backend_aware_database_params): +def test_simple_sql(backend_aware_database_params): with pyexasol.connect(**backend_aware_database_params) as conn: value = conn.execute('SELECT 1 FROM DUAL').fetchval() assert value == 1 From 8c1f8af8fbe6a6445a95935f54240747390625be Mon Sep 17 00:00:00 2001 From: Christoph Kuhnke Date: Mon, 2 Feb 2026 12:53:02 +0100 Subject: [PATCH 4/5] Apply suggestions from code review Co-authored-by: Ariel Schulz <43442541+ArBridgeman@users.noreply.github.com> --- pytest-backend/README.md | 14 +++++++------- pytest-backend/doc/changes/unreleased.md | 2 +- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/pytest-backend/README.md b/pytest-backend/README.md index dd7d5f2..d8bfafc 100644 --- a/pytest-backend/README.md +++ b/pytest-backend/README.md @@ -1,6 +1,6 @@ # pytest-exasol-backend Plugin -Pytest plugin `pytest-exasol-backend` provides pytest fixtures for using various variants of Exasol database instances in integration tests. +The pytest plugin `pytest-exasol-backend` provides pytest fixtures for using various Exasol database instances in integration tests. In particular, the plugin enables accessing both Exasol On-Prem and SaaS backends in a unified way. @@ -29,11 +29,11 @@ poetry add pytest-exasol-backend --group dev ## Usage in Tests -### Pyexasol Connection +### PyExasol Connection -This test accesses the database via [Pyexasol](github.com/exasol/pyexasol) and requires an additional (`dev`) dependency to `pyexasol` to be added to your project. +This test accesses the database via [PyExasol](github.com/exasol/pyexasol) and requires an additional (`dev`) dependency to `pyexasol` to be added to your project. -Note: If pytest option `--backend all` is specified, then this test will run **twice** - once for each backend. +Note: If the pytest option `--backend all` is specified, then this test will run **twice** - once for each backend. ```python import pyexasol @@ -88,13 +88,13 @@ def test_something_backend_sensitive(backend): ## Selecting Backends on the Command Line -By default, none of the backends is selected for testing. +There is no default backend specified for testing. Please use the `--backend` option to specify the target backend with either `onprem`, `saas`, or `all`. The plugin automatically starts the selected backends and shuts them down after the test session has finished. * A SaaS backend is started via [saas-api-python](https://github.com/exasol/saas-api-python/). -* An on-prem backend via the [ITDE](https://github.com/exasol/integration-test-docker-environment). +* An On-Prem backend via the [ITDE](https://github.com/exasol/integration-test-docker-environment). * Additionally you can [use an external or local database](#re-using-an-external-or-local-database). Please noe that all selected backends are started preemptively, regardless of their _actual usage_ in tests. @@ -103,7 +103,7 @@ Therefore, it is important to make sure the backends are not selected where they ### Example Command Lines -Run the tests on an on-prem database: +Run the tests on an On-Prem database: ```shell pytest --backend=onprem my_test_suite.py diff --git a/pytest-backend/doc/changes/unreleased.md b/pytest-backend/doc/changes/unreleased.md index d113204..c982f3e 100644 --- a/pytest-backend/doc/changes/unreleased.md +++ b/pytest-backend/doc/changes/unreleased.md @@ -5,4 +5,4 @@ * #140: Ensured that a proper project-short-tag is used in SaaS tests. * #141: Added a Merge Gate to the CI Workflow. * #146: Relocked transitive dependency filelock -* #150: Updated user guide and added instructions for (Re-)using an external or local database to the `README` +* #150: Updated user guide and added instructions for (re-)using an external or local database to the `README` From 08280ff67ad24a0192d7c2b99f63c81ad2f68e6b Mon Sep 17 00:00:00 2001 From: ckunki Date: Mon, 2 Feb 2026 13:01:28 +0100 Subject: [PATCH 5/5] Fixed review finding --- pytest-backend/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pytest-backend/README.md b/pytest-backend/README.md index d8bfafc..0ec3cac 100644 --- a/pytest-backend/README.md +++ b/pytest-backend/README.md @@ -72,7 +72,7 @@ def test_my_file_exists(backend_aware_bucketfs_params): ### Inspect the Selected Backend Variant -If a tests wants to know, which backend it is running with, it can use the `backend` fixture, as shown below. +For inquiring the currenty selected backend variant in a test case, you can use the `backend` fixture, as shown below. ```python def test_something_backend_sensitive(backend):