Boost pyoptex optimal-design test coverage + reliance assessment

.github/workflows/run-tests.yml

@@ -114,6 +114,36 @@ jobs:
         token: ${{ secrets.CODECOV_TOKEN }}
         fail_ci_if_error: false
 
+  # Optional-dependency coverage: exercise the pyoptex-backed optimal-design
+  # paths (D/I/A-optimal coordinate exchange, split-plot). pyoptex is an
+  # optional, undeclared dependency, so the main `test` job skips every
+  # pyoptex-gated test. We install pyoptex separately here to get real signal.
+  #
+  # This job is NON-BLOCKING (`continue-on-error: true`) on purpose: the
+  # plotly>=6 compatibility fix (mborn1/pyoptex#49) is merged upstream but not
+  # yet on PyPI, so the latest release conflicts with our `plotly>=6.5.2`. We
+  # therefore install pyoptex from git `main` on top of the normal sync; if
+  # upstream main breaks or stops building, this job goes yellow, never red.
+  # Promote it to a blocking check (and move pyoptex into the `expt`/`all`
+  # extras) once a plotly>=6-compatible pyoptex is published to PyPI.
+  test-with-pyoptex:
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        fetch-depth: 0
+    - name: Install uv and set the python version
+      uses: astral-sh/setup-uv@v7
+      with:
+        python-version: "3.13"
+    - name: Install the project (editable)
+      run: uv sync --dev --all-extras
+    - name: Install pyoptex from git main (PR #49, unreleased)
+      run: uv pip install "pyoptex @ git+https://github.com/mborn1/pyoptex.git@main"
+    - name: Run the optimal-design tests with pyoptex present
+      run: uv run pytest tests/test_design_generation.py tests/test_designs_screening_optimal.py -o "addopts=" -v
+
   # ENG-15 sub-track: catch validation `assert`s stripped by `python -O`.
   # SEC-08 swept ~105 sites; SEC-17 (#266) swept the multivariate / bivariate /
   # batch / experiments / monitoring sites that survived. This job now runs as

CHANGELOG.md

@@ -11,6 +11,25 @@ those changes.
 
 ## [Unreleased]
 
+## [1.40.3] - 2026-06-12
+
+### Added
+
+- Expanded test coverage for the optimal-design generators
+  (`experiments/designs_optimal.py`): the D-optimal point-exchange fallback
+  edge cases (SEC-19 candidate-set cap, `n_points` floor, budget capping) and,
+  on the `pyoptex` path, categorical-factor conversion, multiple
+  hard-to-change factors, model-type variants, and D/I/A criterion
+  differentiation.
+
+### Changed
+
+- CI now runs a dedicated, non-blocking `test-with-pyoptex` job that installs
+  `pyoptex` from git `main` and exercises the optimal-design tests, so the
+  `pyoptex`-gated paths get real coverage. `pyoptex` remains an optional,
+  undeclared dependency; see the "Reliance on pyoptex" note in the DOE
+  coverage docs for the rationale.
+
 ## [1.40.2] - 2026-06-12
 
 ### Fixed
@@ -2010,7 +2029,8 @@ this entry records them together.
 - Reworked the README with a sharper value proposition and a
   "Why not scikit-learn?" comparison table.
 
-[Unreleased]: https://github.com/kgdunn/process-improve/compare/v1.40.2...HEAD
+[Unreleased]: https://github.com/kgdunn/process-improve/compare/v1.40.3...HEAD
+[1.40.3]: https://github.com/kgdunn/process-improve/compare/v1.40.2...v1.40.3
 [1.40.2]: https://github.com/kgdunn/process-improve/compare/v1.40.1...v1.40.2
 [1.40.1]: https://github.com/kgdunn/process-improve/compare/v1.40.0...v1.40.1
 [1.40.0]: https://github.com/kgdunn/process-improve/compare/v1.39.0...v1.40.0

CITATION.cff

@@ -12,7 +12,7 @@ authors:
 repository-code: "https://github.com/kgdunn/process-improve"
 url: "https://kgdunn.github.io/process-improve/"
 license: MIT
-version: 1.40.2
+version: 1.40.3
 date-released: "2026-06-12"
 keywords:
   - chemometrics

docs/doe/coverage.md

@@ -58,6 +58,48 @@ All 14 metrics live in `experiments/evaluate.py` behind the `_METRIC_REGISTRY`:
 - **Optimal designs without `pyoptex`.** D-optimal has a point-exchange fallback; I/A-optimal raise `ImportError` instead. Hard-to-change factors (split-plot structure) are currently ignored with a `logger.warning` when `pyoptex` is not available.
 - **Taguchi OA auto-selection** picks the smallest standard array that covers the requested factor count and level counts, but the underlying `pyDOE3.taguchi_design` requires the number of `levels_per_factor` entries to match the OA column count exactly, so requesting a Taguchi design with fewer factors than any available OA will fail. Users typically pick *k* to match one of the standard arrays (3, 4, 7, 11, 15, …).
 
+## Reliance on `pyoptex`
+
+`pyoptex` (github.com/mborn1/pyoptex) powers the high-quality optimal designs
+(D/I/A-optimal coordinate exchange and split-plot structures) in
+`designs_optimal.py`. It is an **optional, undeclared** dependency: the core
+install never pulls it in, and the integration degrades cleanly when it is
+absent (D-optimal falls back to point exchange; I/A-optimal raise a clear
+`ImportError`; hard-to-change factors are ignored with a warning). This keeps
+our coupling to a single, well-isolated adapter module.
+
+We deliberately keep that coupling loose, for two reasons.
+
+- **Release cadence.** `pyoptex` is updated infrequently. The compatibility fix
+  that lets it coexist with our `plotly>=6.5.2` (loosened `plotly`/`pandas`/
+  `scikit-learn` pins, upstream PR #49) is merged but not yet released to PyPI;
+  the latest release (`pyoptex==1.2.1`) still pins `plotly~=5.24` and therefore
+  cannot be resolved alongside our stack. Until a compatible release ships,
+  `pyoptex` cannot be added to the `expt`/`all` extras without breaking the
+  `uv sync --all-extras` resolution that CI relies on.
+- **Where the value lives.** The slice we use (`doe/fixed_structure`) is
+  Cython-compiled (the coordinate-exchange optimizer and split-plot formula
+  evaluation ship as C extensions). Vendoring it is permitted (`pyoptex` is
+  BSD-3-Clause) but would add a C build toolchain to an otherwise pure-Python
+  package and hand us the maintenance of numerically delicate optimizer code.
+  The current friction is a packaging release lag, not a code problem, so
+  vendoring is the wrong trade.
+
+**How the gated tests still get exercised.** Because the main CI job never
+installs `pyoptex`, the `@_skip_no_pyoptex` tests would skip everywhere. A
+dedicated, **non-blocking** `test-with-pyoptex` job in `run-tests.yml` installs
+`pyoptex` from git `main` (PR #49) on top of the normal sync and runs the
+optimal-design tests, so the real `pyoptex` paths get coverage without coupling
+core resolution to an unreleased upstream. If upstream main breaks, that job
+goes yellow, never red.
+
+**Exit criterion.** Once `pyoptex` publishes a `plotly>=6`-compatible release,
+move it into the `expt`/`all` extras as a normal pinned dependency and promote
+the `test-with-pyoptex` job to a blocking check. If `pyoptex` instead goes
+unmaintained, the clean replacement is to write a minimal coordinate-exchange
+for I/A-optimal in numpy (reusing the existing `point_exchange` pattern in
+`optimal.py`), not to transplant the upstream Cython.
+
 ## No Silent Fallbacks
 
 `generate_design` does **not** silently substitute a different design type when the requested one is infeasible. Unknown `design_type` values raise `ValueError` in `designs.py:328-331`; individual dispatchers validate their own inputs (e.g. BBD and DSD require `k ≥ 3`). Errors surface through the tool wrapper as `{"error": ...}` and reach agent callers as HTTP 422.

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "process-improve"
-version = "1.40.2"
+version = "1.40.3"
 description = 'Designed Experiments; Latent Variables (PCA, PLS, multivariate methods with missing data); Process Monitoring; Batch data analysis.'
 readme = "README.md"
 license = "MIT"

tests/test_design_generation.py

@@ -6,6 +6,12 @@
 import pytest
 
 from process_improve.experiments.designs import _auto_select, generate_design
+from process_improve.experiments.designs_optimal import (
+    _convert_factors_to_pyoptex,
+    dispatch_a_optimal,
+    dispatch_d_optimal,
+    dispatch_i_optimal,
+)
 from process_improve.experiments.factor import Constraint, Factor, FactorType
 
 _HAS_PYOPTEX = False
@@ -619,6 +625,105 @@ def test_split_plot_metadata(self) -> None:
         assert result.metadata.get("backend") == "pyoptex"
 
 
+# ---------------------------------------------------------------------------
+# pyoptex adapter and dispatch coverage (categorical, multi split-plot,
+# model-type variants, criterion differentiation)
+# ---------------------------------------------------------------------------
+
+
+@_skip_no_pyoptex
+class TestConvertFactorsToPyoptex:
+    """Unit tests for the ``_convert_factors_to_pyoptex`` adapter."""
+
+    def test_hard_to_change_gets_random_effect(self) -> None:
+        """Only the hard-to-change factor carries a RandomEffect."""
+        factors = _continuous_factors(3, "ABC")
+        pyoptex_factors = _convert_factors_to_pyoptex(factors, hard_to_change=["A"], n_runs=12)
+        by_name = {f.name: f for f in pyoptex_factors}
+        assert by_name["A"].re is not None
+        assert by_name["B"].re is None
+        assert by_name["C"].re is None
+
+    def test_whole_plots_are_balanced(self) -> None:
+        """The whole-plot assignment groups runs into equal consecutive blocks."""
+        factors = _continuous_factors(2, "AB")
+        pyoptex_factors = _convert_factors_to_pyoptex(
+            factors, hard_to_change=["A"], n_runs=12, n_whole_plots=4
+        )
+        z = np.asarray({f.name: f for f in pyoptex_factors}["A"].re.Z)
+        # 12 runs / 4 plots -> [0,0,0, 1,1,1, 2,2,2, 3,3,3]
+        assert z.tolist() == [0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3]
+
+    def test_categorical_factor_conversion(self) -> None:
+        """Categorical factors carry their type and levels into pyoptex."""
+        factors = [
+            Factor(name="Cat", type="categorical", levels=["x", "y", "z"]),
+            Factor(name="B", low=0, high=10),
+        ]
+        pyoptex_factors = _convert_factors_to_pyoptex(factors)
+        cat = {f.name: f for f in pyoptex_factors}["Cat"]
+        assert cat.type == "categorical"
+        assert cat.levels == ["x", "y", "z"]
+
+
+@_skip_no_pyoptex
+class TestOptimalCategorical:
+    """Optimal designs with a mix of categorical and continuous factors."""
+
+    def test_d_optimal_with_categorical(self) -> None:
+        factors = [
+            Factor(name="Cat", type="categorical", levels=["x", "y", "z"]),
+            Factor(name="B", low=0, high=10),
+        ]
+        design, meta = dispatch_d_optimal(factors, budget=9)
+        assert design.shape == (9, 2)
+        assert meta["backend"] == "pyoptex"
+
+
+@_skip_no_pyoptex
+class TestMultipleHardToChange:
+    """Split-plot designs with more than one hard-to-change factor."""
+
+    def test_two_hard_to_change_factors(self) -> None:
+        factors = _continuous_factors(3, "ABC")
+        design, meta = dispatch_d_optimal(factors, budget=12, hard_to_change=["A", "B"])
+        assert design.shape[0] == 12
+        assert meta["hard_to_change"] == ["A", "B"]
+        assert meta["backend"] == "pyoptex"
+
+
+@_skip_no_pyoptex
+class TestModelTypeVariants:
+    """Each model_type maps to a valid pyoptex RSM and produces a design."""
+
+    @pytest.mark.parametrize("model_type", ["main_effects", "interactions", "quadratic"])
+    def test_model_type_produces_finite_metric(self, model_type: str) -> None:
+        factors = _continuous_factors(3, "ABC")
+        _design, meta = dispatch_d_optimal(factors, budget=12, model_type=model_type)
+        assert meta["model_type"] == model_type
+        assert np.isfinite(meta["metric_value"])
+
+
+@_skip_no_pyoptex
+class TestCriterionDifferentiation:
+    """D/I/A-optimal each report their own criterion and a finite metric."""
+
+    @pytest.mark.parametrize(
+        ("dispatch", "criterion"),
+        [
+            (dispatch_d_optimal, "d_optimal"),
+            (dispatch_i_optimal, "i_optimal"),
+            (dispatch_a_optimal, "a_optimal"),
+        ],
+    )
+    def test_metadata_reports_criterion(self, dispatch, criterion: str) -> None:
+        factors = _continuous_factors(3, "ABC")
+        _design, meta = dispatch(factors, budget=10)
+        assert meta["optimality_criterion"] == criterion
+        # I/A-optimal metric values are routinely negative, so check finiteness.
+        assert np.isfinite(meta["metric_value"])
+
+
 # ---------------------------------------------------------------------------
 # Mixture
 # ---------------------------------------------------------------------------

tests/test_designs_screening_optimal.py

@@ -11,6 +11,7 @@
 import pytest
 
 from process_improve.experiments.designs_optimal import (
+    _run_point_exchange_fallback,
     dispatch_a_optimal,
     dispatch_d_optimal,
     dispatch_i_optimal,
@@ -101,6 +102,66 @@ def test_hard_to_change_without_pyoptex_warns(self, caplog: pytest.LogCaptureFix
         assert any("pyoptex is not installed" in rec.message for rec in caplog.records)
 
 
+class TestPointExchangeFallback:
+    """Direct tests of the no-pyoptex D-optimal fallback.
+
+    These call ``_run_point_exchange_fallback`` directly, so they exercise the
+    fallback regardless of whether pyoptex is installed (the function does not
+    consult ``_PYOPTEX_AVAILABLE``).
+    """
+
+    def test_returns_design_and_d_optimality(self) -> None:
+        """Fallback returns a k-column matrix and a finite d_optimality score."""
+        import numpy as np
+
+        k = 3
+        design, meta = _run_point_exchange_fallback(_continuous(k), budget=8)
+        assert design.shape[1] == k
+        # point_exchange is a heuristic that grows from k rows up to the cap,
+        # so the row count lands in [k, budget] rather than being exact.
+        assert k <= design.shape[0] <= 8
+        assert meta["backend"] == "point_exchange_fallback"
+        assert np.isfinite(meta["d_optimality"])
+
+    def test_n_points_floor_avoids_singular_request(self) -> None:
+        """A budget below k+1 is raised so point_exchange does not raise.
+
+        Without the ``max(n_points, k + 1)`` floor, a budget of 2 with 4
+        factors would ask point_exchange for fewer points than columns and
+        raise a ValueError. The floor keeps the request estimable.
+        """
+        design, _meta = _run_point_exchange_fallback(_continuous(4), budget=2)
+        assert design.shape[1] == 4
+        assert design.shape[0] >= 4  # point_exchange always starts with k rows
+
+    def test_budget_capped_to_candidate_set(self) -> None:
+        """A budget larger than the 3**k candidate set is capped to its size."""
+        # 2 factors -> 3**2 = 9 candidate rows; an over-large budget cannot
+        # select more than the 9 available candidates.
+        design, _meta = _run_point_exchange_fallback(_continuous(2), budget=50)
+        assert design.shape[1] == 2
+        assert design.shape[0] <= 9
+
+    def test_sec19_cap_rejects_too_many_factors(self, monkeypatch: pytest.MonkeyPatch) -> None:
+        """The 3**k candidate set is refused past the SEC-19 factor cap."""
+        from process_improve.config import settings
+
+        monkeypatch.setattr(settings, "max_factors_combinatorial", 3)
+        with pytest.raises(ValueError, match="SEC-19 cap"):
+            _run_point_exchange_fallback(_continuous(5), budget=8)
+
+
+@_skip_with_pyoptex
+class TestDOptimalFallbackModelType:
+    """model_type is accepted on the fallback dispatch path (no pyoptex)."""
+
+    @pytest.mark.parametrize("model_type", ["main_effects", "interactions", "quadratic"])
+    def test_model_type_accepted(self, model_type: str) -> None:
+        design, meta = dispatch_d_optimal(_continuous(3), budget=8, model_type=model_type)
+        assert design.shape[1] == 3
+        assert meta["backend"] == "point_exchange_fallback"
+
+
 @_skip_with_pyoptex
 class TestOptimalRequiresPyoptex:
     """I-optimal and A-optimal raise a clear error without pyoptex."""