Ax 1.2.1

Bug fixes

• Improved error messaging for client.compute_analyses when certain analyses are
  not yet available (#4441)
• Fix tooltip mismatch bug in ArmEffectsPlot (#4479)

Other changes

• Bumped pinned botorch version to 0.16.1 (#4570)
• Removed deprecated robust optimization functionality (#4493)
• Allow HierarchicalSearchSpace to be constructed with multiple root nodes (#4560)

Ax 1.2.0

New features

• DerivedParameterConfig allows users to specify parameters which are not tuned,
  instead taking the value of some expression of other tunable parameters (#4454)
• New argument simplify_parameter_changes in client.configure_generation_strategy
  (defaulted to False) which when True informs Ax to change as few parameters as
  possible relative to the status quo without degrading optimization performance. Has
  a near-zero effect on generation time (#4409)
• Default to qLogNParEgo acquisition function for multi-objective optimization in
  multi-objective optimization when the number of objectives is > 4, leading to
  improved walltime performance (#4347)

Bug fixes

• Fix issue during candidate generation involving MapMetrics providing progressions
  at different scales i.e. one progression goes up to 10^9 and the other goes up to
  10^6 by normalizing to [0, 1] (#4458)

Other changes

• Improve visual clarity in ArmEffectsPlot by removing certain elements including
  red "infeasibility" halos and optional cumulative best line (#4397, #4398)
• Instructions on citing Ax included in README.md and ax.dev (#4317, #4357)
• New "Using external methods for candidate generation in Ax" tutorial on website (#4298)

Ax 1.1.2

Bug fixes

• Fixed rendering issue in ArmEffectsPlot when the number of arms displayed is greater than 20 (#4273)

Other changes

• Enabled Winsorization transform by default, improving surrogate performance in the presence of outliers (#4277)

Ax 1.1.1

Bug fixes

• Correctly filter out observations from Abandoned trials/arms during candidate
  generation (#4155)
• Handle scalarized objectives in ResultsAnalysis (#4193)
• Fix bug in polytope sampler when generating more than one candidate in a batch (#4244)

Other changes

• Transition from setup.py to pyproject.toml for builds, modernizing Ax's build
  configuration and bringing it in compliance with PEP 518 and PEP 621 (#4100)
• Add py.typed file, which allows typecheckers like Pyre, mypy, etc. to see Ax's types
  and avoid a TypeStub error when importing Ax (#4139)
• Improve legibility of ArmEffectPlot by modifying legend and x-axis labels (#4220,
  #4243)
• Address logspew in OneHotEncoder transform (#4232)

Ax 1.1.0

New Features

• New option for the method parameter in client.configure_generation_strategy:
  quality -- allows uers to indicate they would like Ax to generate the highest
  quality candidates it is able to at the expense of slower runtime (#4042)
• New logic for deciding which analyses to produce by default in
  client.compute_analyses (#4013)
• New parameters in client.summarize allow users to filter their summary by trial
  index and/or trial status (#4012, #4118)

Bug Fixes

• Allow client.summarize to be called without a GenerationStrategy being set
  (i.e. before client.configure_generation_strategy or client.get_next_trails
  has been called.) (#3801)
• Fixed incorrect grouping in TopSurfacesAnalysis (#4095)
• Fixed ContourPlot failing to compute in certain search spaces with parameter
  constraints (#4124)
• Misc. plotting fixes and improvements

Other changes

• Bumped pinned botorch version to 0.15.1
• Performance improvements in SensitivityAnalysis (#3891)
• Improved optimization performance in constrained optimization settings (#3585)
• Augmented logging in Client, early stopping module (#4044, #4108)

Ax 1.0.0

We released Ax in 2019 along with Botorch, and both libraries have evolved significantly since their initial release. Today, we are excited to release Ax version 1.0.0.

Along with typical new features, improvements, and bug fixes this release introduces a new public API (found in ax.api) which allows users to access all fully supported Ax features via a singular entry point. Classes and functions from Ax entry points (“Loop API, “Service API”, and “Developer API”) remain present in the library, but are considered deprecated and may be changed backwards incompatibly (or removed) in subsequent major or minor releases of Ax.

This release also comes with a full rewrite of our documentation which can be found at ax.dev

Highlights

• New unified API can be found in ax.api module. The classes and functions here constitute the Ax public API
  • The Client class manages state throughout the optimization process
  • Various Config classes allow the user to configure their search space, the optimization’s goals, etc.
• Complete documentation overhaul
  • An entirely rewritten https://ax.dev/ introduces users to important concepts, demonstrates usage on synthetic and real-world optimization problems, and thoroughly documents the API

Breaking changes

Now that we have defined a singular Ax API, users should make efforts to use this as an entry point into the library. This will give users assurance that the method signatures they use will remain stable between major versions of Ax, and allow our team to provide better support via Github Issues.

This will be straightforward for those previously using the “Service API”, which closely resembles the current API – users of the “Loop API” should look towards the Quickstart Guide (https://ax.dev/docs/next/tutorials/quickstart/) for an example which demonstrates using the new API for the simple optimizations the Loop API previously served, and “Developer API” users should also use the full documentation to understand how the new API can serve their needs.

Maintenance Release, New Website, Improved Performance in Mixed Search Spaces

Compatibility

• Require BoTorch 0.13.0

Highlights

• AX website has been upgraded to utilize Docusaurus v3, with the API reference being hosted by ReadTheDocs. The tutorials now expose an option to open with Colab, for easy access to a runtime with modifiable tutorials. The old versions of the website can be found at archive.ax.dev.
• Updated the default transforms and optimizers used in MBM-based models, to offer improved optimization performance in mixed discrete search spaces.
• Implemented a preview version of Ax 1.0 API (ax/preview). The new API is available for use & testing, but remains subject to change until the 1.0 release.
• Introduced new Analysis cards (ax/analysis) for analyzing experiment progress, results and model quality; which will replace various plotting and reporting utilities with a unified API.
• Deprecated a number of legacy models, including GPEI, in favor of the modular BoTorch model (MBM) API.
• Refactored Ax benchmark APIs to simplify problem setup and to support additional use cases such as noisy & asynchronous benchmarking.
• Added EBAshr model, which is used for repeating arms in online experiments as well as regression analysis in the health checks (#3276).
• New advanced capabilities:
  • MBM now supports per-metric model specification and model selection.
  • Support for new transition criteria and input constructors in GenerationNodes.

V0.4.3 - Maintenance Release

Key things:

• Update to BoTorch 0.12.0 – which included some major updates to underlying modeling, please see BoTorch release for detailed description
• Fix for a sql storage bug related to an import trying to import a deleted file

Compatibility

• BoTorch 0.12.0 -- which included some major updates to underlying modeling, please see BoTorch release for detailed description
• pin numpy to < 2.0 until tensorboard cuts a release (#2641)
• Make SQLAlchemy==1.4.17 a required dependency (#2690)

New Features

• AnalysisCard refactor (#2589)
• New parallel coordinates plot (#2590)
• Add an optimal_value to mixed-integer problems (#2596)
• Add AutoTransitionAfterGenCriterion to storage (#2614)
• Storage support for RBFKernel & LogNormalPrior (#2616)
• Add storage support for SobolQMCNormalSampler (#2622)
• AuxiliaryExperiment (#2632)
• Add auxiliary_experiments to Experiment (#2634)
• Create Scheduler.generate_candidates() function (#2640)
• AnalysisCard load/save methods (#2645)
• Healthcheck analysis class (#2646)
• add auxiliary experiments to SQAExperiment (#2658)
• Add Scheduler.compute_analyses method (#2660)
• Change time limit for unit tests to 60 seconds and introduce a decorator for long running tests (#2661)
• Add utility methods to Analysis (#2663)
• Introduce ParamBasedTestProblem for benchmarking (#2675)
• Run ModelLauncher with a BenchmarkRunner and BenchmarkMetric (#2681)
• Add init_position argument to UniformGenerator (#2686)
• Run Torchvision problems with Benchmark[Problem/Runner/Metric]; consolidate PyTorchCNN problems (#2688)
• allow more candidate trials than max_trials in Scheduler (#2689)
• create utility for trying to load GS (#2694)
• Implement Data/MapData.clone and use it in Experiment.clone_with (#2703)
• Compute ground-truth optimization trace on BenchmarkProblem (#2704)
• Add evaluate_oracle method to BenchmarkRunner (#2705)
• Merge create_single_objective_problem_from_botorch and create_multi_objective_problem_from_botorch; support constrained MOO (#2722)
• Add an option to override ModelSpec.model_key (#2726)
• Update test_generation_strategy to utilize MBM models (#2734)
• Refactor computation of optimization trace (#2747)
• Update the remaining models to use new default covar & likelihood modules (#2742)
• Add qLogNParEGO to ACQUISITION_FUNCTION_REGISTRY (#2749)
• Transform batch to new sq (#2755)

Bug Fixes

• Disable long test timeouts when using a debugger (#2745)
• Require all problems to have an optimal_value; simplify Problem inheritance (#2601)
• Update the default SingleTaskGP prior (#2610)
• Remove reference_point from multi-objective benchmark problems (#2619)
• More precise type annotations for MultiObjectiveOptimizationConfig (#2620)
• Change blob type to str (#2624)
• Revert D60472381 (#2627)
• Check for Nans and Infs in TensorboardMetric (#2628)
• Add warning to SerializationMixin, to prevent usage that will result in circular deps (#2629)
• Use Standardize by default for SingleTaskGP (#2630)
• Modify get_data to error out on nan/inf (#2633)
• fix bug in UnitX when transforming new search space (#2639)
• Add _maybe_clear_multiplexer method to deal with memory leaks (#2642)
• Fix Surrogate.best_out_of_sample_point (#2652)
• Fix clear_multiplexer (#2656)
• Fix Pyre errors in sqa_classes (#2659)
• Relax type hints on compute() (#2662)
• Fix type error in validate_sphinx script (#2665)
• Make sure random seed persists beyond storage (#2671)
• Log an exception rather than erroring out when data is found for metrics that are not attached to the experiment (#2680)
• fix input args for target-aware model in TLMB (#2685)
• Stop calling float() on single element Series (#2691)
• Serialize data classes based on their fields only (#2697)
• [Website] Use mathjax svg renderer (#2707)
• Take out error about HSS not being supported with batch trials (#2709)
• Make sure is_ax_equal returns a boolean (#2714)
• Remove internal type shed for plotly (#2715)
• Bring back ax/benchmark/init.py (#2719)
• BUG FIX: Only construct surrogate once in SurrogateRunner (#2720)
• Mock out downloading data from internet in torchvision unit test (#2725)
• robustify PL test (#2729)
• Require unique model_key in GenerationNode, clean up model selection errors (#2730)
• Add longtest decorator to pairwise model tests (#2731)
• Ensure new arm names do not match a different name on the experiment (#2732)
• Ignore fixed parameters for search space dimensionality computation (#2765)
• In ‘best_point’ don’t require GeneratorRun to have best_arm_predictions to predict from model (#2767)
• Fix Scheduler test_run_trials_and_yield_results_with_early_stopper that was timing out the CI (#2770)
• Make Acquisition.optimize work with discrete optimizer regardless of whether raw_samples is in optimizer_options (#2772)

Other Changes

• Document setup_import_mocks (#2736)
• Rename maximum_hypervolume to optimal_value in MOO Benchmark Problems (#2598)
• Make all benchmark problems take an observe_noise_stds argument (#2600)
• Move lazy construction of a surrogate from problem to runner (#2603)
• Update BestModelSelector to return a ModelSpec rather than the index (#2609)
• Add STATUSES_EXPECTING_DATA to trials consts (#2621)
• Drop critertion from AutoTransitionAfterGenCriterion name (#2625)
• Expanded testing get_pbo_experiment testing stub (#2631)
• Use reduced state for axolotl (#2637)
• AnalysisCard encoder/decoder refactor (#2643)
• SQAAnalysisCard refactor (#2644)
• codemod pep585_imports (#2648)
• Add Sphinx doc to Healthcheck analysis (#2649)
• Pass botorch_model_class to Surrogate._set_formatted_inputs (#2653)
• [Move model input constructor helper functions off of Surrogate, to standalone functions](htt...

V0.4.2 - Maintenance Release

Key things:

• Update to BoTorch 0.12.0 – which included some major updates to underlying modeling, please see BoTorch release for detailed description
• Fix for a sql storage bug related to an import trying to import a deleted file

Compatibility

• BoTorch 0.12.0 -- which included some major updates to underlying modeling, please see BoTorch release for detailed description
• pin numpy to < 2.0 until tensorboard cuts a release (#2641)
• Make SQLAlchemy==1.4.17 a required dependency (#2690)

New Features

• AnalysisCard refactor (#2589)
• New parallel coordinates plot (#2590)
• Add an optimal_value to mixed-integer problems (#2596)
• Add AutoTransitionAfterGenCriterion to storage (#2614)
• Storage support for RBFKernel & LogNormalPrior (#2616)
• Add storage support for SobolQMCNormalSampler (#2622)
• AuxiliaryExperiment (#2632)
• Add auxiliary_experiments to Experiment (#2634)
• Create Scheduler.generate_candidates() function (#2640)
• AnalysisCard load/save methods (#2645)
• Healthcheck analysis class (#2646)
• add auxiliary experiments to SQAExperiment (#2658)
• Add Scheduler.compute_analyses method (#2660)
• Change time limit for unit tests to 60 seconds and introduce a decorator for long running tests (#2661)
• Add utility methods to Analysis (#2663)
• Introduce ParamBasedTestProblem for benchmarking (#2675)
• Run ModelLauncher with a BenchmarkRunner and BenchmarkMetric (#2681)
• Add init_position argument to UniformGenerator (#2686)
• Run Torchvision problems with Benchmark[Problem/Runner/Metric]; consolidate PyTorchCNN problems (#2688)
• allow more candidate trials than max_trials in Scheduler (#2689)
• create utility for trying to load GS (#2694)
• Implement Data/MapData.clone and use it in Experiment.clone_with (#2703)
• Compute ground-truth optimization trace on BenchmarkProblem (#2704)
• Add evaluate_oracle method to BenchmarkRunner (#2705)
• Merge create_single_objective_problem_from_botorch and create_multi_objective_problem_from_botorch; support constrained MOO (#2722)
• Add an option to override ModelSpec.model_key (#2726)
• Update test_generation_strategy to utilize MBM models (#2734)
• Refactor computation of optimization trace (#2747)
• Update the remaining models to use new default covar & likelihood modules (#2742)
• Add qLogNParEGO to ACQUISITION_FUNCTION_REGISTRY (#2749)
• Transform batch to new sq (#2755)

Bug Fixes

• Disable long test timeouts when using a debugger (#2745)
• Require all problems to have an optimal_value; simplify Problem inheritance (#2601)
• Update the default SingleTaskGP prior (#2610)
• Remove reference_point from multi-objective benchmark problems (#2619)
• More precise type annotations for MultiObjectiveOptimizationConfig (#2620)
• Change blob type to str (#2624)
• Revert D60472381 (#2627)
• Check for Nans and Infs in TensorboardMetric (#2628)
• Add warning to SerializationMixin, to prevent usage that will result in circular deps (#2629)
• Use Standardize by default for SingleTaskGP (#2630)
• Modify get_data to error out on nan/inf (#2633)
• fix bug in UnitX when transforming new search space (#2639)
• Add _maybe_clear_multiplexer method to deal with memory leaks (#2642)
• Fix Surrogate.best_out_of_sample_point (#2652)
• Fix clear_multiplexer (#2656)
• Fix Pyre errors in sqa_classes (#2659)
• Relax type hints on compute() (#2662)
• Fix type error in validate_sphinx script (#2665)
• Make sure random seed persists beyond storage (#2671)
• Log an exception rather than erroring out when data is found for metrics that are not attached to the experiment (#2680)
• fix input args for target-aware model in TLMB (#2685)
• Stop calling float() on single element Series (#2691)
• Serialize data classes based on their fields only (#2697)
• [Website] Use mathjax svg renderer (#2707)
• Take out error about HSS not being supported with batch trials (#2709)
• Make sure is_ax_equal returns a boolean (#2714)
• Remove internal type shed for plotly (#2715)
• Bring back ax/benchmark/init.py (#2719)
• BUG FIX: Only construct surrogate once in SurrogateRunner (#2720)
• Mock out downloading data from internet in torchvision unit test (#2725)
• robustify PL test (#2729)
• Require unique model_key in GenerationNode, clean up model selection errors (#2730)
• Add longtest decorator to pairwise model tests (#2731)
• Ensure new arm names do not match a different name on the experiment (#2732)
• Ignore fixed parameters for search space dimensionality computation (#2765)
• In ‘best_point’ don’t require GeneratorRun to have best_arm_predictions to predict from model (#2767)
• Fix Scheduler test_run_trials_and_yield_results_with_early_stopper that was timing out the CI (#2770)
• Make Acquisition.optimize work with discrete optimizer regardless of whether raw_samples is in optimizer_options (#2772)

Other Changes

• Document setup_import_mocks (#2736)
• Rename maximum_hypervolume to optimal_value in MOO Benchmark Problems (#2598)
• Make all benchmark problems take an observe_noise_stds argument (#2600)
• Move lazy construction of a surrogate from problem to runner (#2603)
• Update BestModelSelector to return a ModelSpec rather than the index (#2609)
• Add STATUSES_EXPECTING_DATA to trials consts (#2621)
• Drop critertion from AutoTransitionAfterGenCriterion name (#2625)
• Expanded testing get_pbo_experiment testing stub (#2631)
• Use reduced state for axolotl (#2637)
• AnalysisCard encoder/decoder refactor (#2643)
• SQAAnalysisCard refactor (#2644)
• codemod pep585_imports (#2648)
• Add Sphinx doc to Healthcheck analysis (#2649)
• Pass botorch_model_class to Surrogate._set_formatted_inputs (#2653)
• [Move model input constructor helper functions off of Surrogate, to standalone functions](h...

V0.4.1 - Maintenance Release

Compatibility

• Require BoTorch version v0.11.3 (#2594)

New Features

• Add TimeAsFeature transform (#2438)
• Add tutorial to mimic parameter constraint functionality for nonlinear constraints (#2500)
• Add option for using posterior predictive in cross-validation (#2517)
• Refactor BestModelSelctor to operate on ModelSpecs (#2557)
• Templates for questions on github (#2575)

Bug Fixes

• Update tests so that data contains metric_names that are on the corresponding experiments (#2422)
• Use name of enum instead of casting to string (#2423)
• Fill missing values with NaN in observation_data_to_array (#2430)
• Prevent unused NaNs from marking constraints as infeasible in pareto_frontier_evaluator (#2431)
• Add copyright comment (#2445)
• Fix various spelling errors in GS (#2448)
• Add warning when metric is skipped because it’s not on the experiment (#2449)
• Stop ignoring unused keyword arguments (#2451)
• Improve error clarity in TensorboardMetric (#2456)
• Save status quo name and feature when multiple status quo present (#2457)
• Remove unused import from MOO tutorial (#2462)
• Update MBM tutorial with modern, non-deprecated functionality and accurate description of defaults (#2466)
• Include CANDIDATE trials in pending_features (#2482)
• Test that we’re not referencing same parameter dict in ObservationFeatures.from_arm (#2483)
• Do not deduplicate against FAILED in GenerationStrategy/Node (#2488)
• Use updated BoTorch HitAndRunPolytopeSampler for Sobol fallback (#2492)
• Fix test_optimize_l0_homotopy test numerics (#2493)
• Remove unused imports from ax/ (#2501)
• Remove unused experiment arg from compute_model_fit_metrics_from_modelbridge (#2504)
• Fix equality check for floats (#2507)
• Fix MOO tutorial (#2509)
• Remove unnecessary pyre-fixme (#2513)
• Disallow modeling metrics not attached to the experiment (#2518)
• Fix pending point bug in ExternalGenerationNode (#2521)
• Fix num_bayesopt_trials logic to support ensembling (#2522)
• Update np.Inf -> np.inf per numpy 2.0 release (#2527)
• Reinforce testenv in Ax test case (#2530)
• Avoid errors in telemetry due to node-based GenerationStrategy (#2554)
• Use fast_botorch_optimize to fix test timeouts (#2556)
• Make MBM acquisition pass the correct argos to each BoTorch optimizer (#2571)
• Remove unnecessary special-casing for qEHVI and qMES from optimizer argparse (#2572)
• Remove link to the Raytune Tutorial (#2578)
• Expand trials_as_df to support multiple GenerationNodes in a single trial (#2577)
• Fixing typo in docstring of get_sobol_botorch_modular_acquisition (#2583)
• Fix incorrect status quo update warning (#2586)

Other Changes

• Update PairwiseModelBridge to support non-preference outcomes (#2424)
• Allow more flexible definition of which trial statuses to fit (#2432)
• Support multiple node generation for single trial (#2428)
• Add TimeAsFeature to transform registery (#2439)
• Handle pending observations in TimeAsFeature (#2440)
• Handle ObservationFeatures without trial_index in Realativize (#2441)
• Pyre update (#2442)
• Creating preference testing stub (#2443)
• Add qExpectedUtilityOfBestOption to test_PairwiseModelBridge (#2444)
• Add transition edges property to GenerationNode (https://github.com/facebook/Ax/pull/24460)
• Update should_transition_to_next_node to handle multiple transition edges (#2447)
• Rename TaskEncode transform to TaskChoiceToIntTaskChoice (#2450)
• Explicit mention that AxService API is recommended throughout docs (#2453)
• Add validation of transition edges to GS (#2454)
• Another pyre update (#2463)
• Always consider choice parameters with 2 values as ordered (#2464)
• Fix forward (#2465 )
• Change flag name from supress_generation_errors to raise_generation_errors (#2468)
• Improve typing in SyntheticFunctions (#2470)
• Clean up unnecessary todos in _get_model_state_from_last_generator_run (#2476)
• Update docstring to signal in-place_modification (#2477)
• Test against python 3.12 (#2478)
• Have MBM only pass acqf input constructor kwargs to BoTorch when they are not None (#2480)
• Enable evaluating Branin test experiments with None status quo (#2489)
• Remove unused suppress_all_errors kwarg (#2491)
• Adding analysis_batch class to ax.analysis (#2498)
• Adding schema for AnalysisBatch and Analysis (#2499)
• Passing current_node_name when gs is node based (#2505)
• Put model fit data in gen_metadata (#2511)
• Save full[er] GeneratorRuns ([https://github....