Skip to content

Merge upstream#105

Draft
AlekSi wants to merge 112 commits intoferretdbfrom
merge-into-ferretdb
Draft

Merge upstream#105
AlekSi wants to merge 112 commits intoferretdbfrom
merge-into-ferretdb

Conversation

@AlekSi
Copy link
Member

@AlekSi AlekSi commented Feb 8, 2026

No description provided.

CosmosDB Build Service (msdata) and others added 30 commits August 21, 2025 20:14
…ase 1.108

This PR updates the extension version for the new release 1.108.

Changes included:
- Control files updated to version 1.107-0
- SQL files generated for version upgrades
- Release notes file created for version 1.108
- TOC.yml updated with new release notes entry
- CHANGELOG.md updated with new version entry
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Updating code to meet contributing guidelines
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

For $lookup we were ignoring the joinquals for point read pushdown - this happened to work before if the planner chose a point read plan before. But now that we're more explicit in looking for this we need to handle this explicitly.

Also ensure point read plan generation considers outer_relids (this wasn't done before).

----
#### AI description  (iteration 1)
#### PR Classification
Bugfix: This PR corrects issues in the point read planner and lookup functionality.

#### PR Summary
The changes fix bugs related to index selection and query planning for multi-layer lookups by improving how join and restriction information is handled, and by extending test coverage for point read scenarios.
- **`oss/pg_documentdb/src/opclass/index_support.c`**: Updated handling of outerRelids and join restrictions to ensure proper index path identification.
- **`oss/pg_documentdb/src/planner/documents_planner.c` and `oss/pg_documentdb/include/opclass/bson_index_support.h`**: Modified calls to pass join info into restriction path processing for accurate planning.
- Test files under **`oss/internal/pg_documentdb_distributed`** and **`pgmongo`**: Added new multi-layer lookup queries and expected outputs to verify that lookups consistently pick the _id index.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
* Revise README for DocumentDB details and instructions
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Debugging

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes. `documentdb.enableDebugQueryText` to enable query text for debugging

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

This PR adds a test configuration `documentdb.enableDebugQueryText` that starts deparsing the query tree and set it as source text either in QueryDesc if run via executor or source text in the PlannedStmp if run via Portal. This will enhance our ability to have deeper insights on the query that results in an unexpected behavior or dump.

NOTE: This should be used **only and only** for debugging purposes, as it starts deparsing all the queries which will take a performance hit.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new debugging feature to conditionally capture and display PG query source text during query execution.

#### PR Summary
The changes update multiple query execution functions to extract the query definition via pg_get_querydef when the EnableQueryText flag is enabled, and pass this source text for debugging purposes.
- `oss/pg_documentdb/src/commands/cursors.c`: Modified functions to conditionally retrieve the query source text and pass it to executors and portal definitions, including updating the DrainStatementViaExecutor signature.
- `oss/pg_documentdb/src/configs/system_configs.c`: Added a new custom Boolean configuration variable (EnableQueryText) with a default value and appropriate documentation.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #4578637
…2, RolesInfo Parsing)

### Does this PR have any customer impact?
Yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
1.107
### Description
We are implementing the RolesInfo command in Engine with three separate PRs. The first one will include the parsing of the input, the second one will be the core implementation of the command, and the third one will be the implementation of showBuiltInRoles.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new feature by adding support for the rolesInfo command in the engine.

#### PR Summary
This update introduces functions to parse and handle the rolesInfo command, supporting various input formats and validations while updating error messages and tests.
- **`roles.c`**: Added new parsing functions (`ParseRolesInfoSpec`, `ParseSingleRoleFromArray`, and `ParseRoleDocument`) and integrated distributed query logic in `roles_info`.
- **`command_roles_info` & `create_role`**: Updated command wrappers and role validation to use the new logic and the `IS_SUPPORTED_BUILTIN_ROLE` macro.
- **`roles.h`**: Introduced the `RolesInfoSpec` structure and defined the `IS_SUPPORTED_BUILTIN_ROLE` macro.
- **Test Files**: Extended regression tests and expected outputs in both DocumentDB and Helio API test suites to cover multiple rolesInfo scenarios and error cases.
- **`users.c` & Feature Counters**: Removed the redundant built-in role support function and added a new feature counter for rolesInfo on built-in roles.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #4333805
…ommands

### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
[Insert] Feature Usage Insert

----
#### AI description  (iteration 1)
#### PR Classification
New feature addition that implements feature usage tracking for insert and update commands.

#### PR Summary
This pull request adds inline functions to report feature usage based on batch sizes for insert and update operations and updates the associated feature enum definitions.
- `/oss/pg_documentdb/src/commands/insert.c`: Added `ReportInsertFeatureUsage` to report usage metrics based on batch size and integrated its call during batch insertion processing.
- `/oss/pg_documentdb/src/commands/update.c`: Introduced `ReportUpdateFeatureUsage` with similar threshold-based reporting, integrated into the batch update process.
- `/oss/pg_documentdb/include/utils/feature_counter.h`: Extended the feature counter enums with new constants for detailed insert and update reporting, and increased `MAX_FEATURE_COUNT`.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Testing
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Fix issues  identified by similarity checker

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a bug fix and code cleanup that standardizes and clarifies error messages across aggregation, query, and index creation operators.

#### PR Summary
The changes update error text throughout the codebase to be more explicit and consistent regarding type validations and operator requirements. Notable updates include:
- Updates to error messages in aggregation operator files (e.g. in `bson_expression_bitwise_operators.c`, `bson_expression_conditional_operators.c`, and similar files) to clearly specify expected input types and constant requirements.
- Rewording of messages for operators such as `$regexFindAll`, `$merge`, `$switch`, `$ifNull`, and trigonometric functions to use precise language (e.g. “requires” or “must be provided” instead of “only supports”).
- Improved clarity in messages for view cycle detection (e.g. in `create_indexes.c` and view creation tests) and hashed index validations.
- Consistent rephrasing for messages in densify and percentile operators to better indicate numeric or date constraints and limits.
- Refinements in error reporting for operators in type conversion and array handling (e.g. in `$arrayElemAt`, `$slice`, `$getField`, and `$unsetField`), ensuring that expected value types (such as string, numeric, or object) are explicitly stated.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Features

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

1) Port a few more changes into the public documentdb_rum to reduce diff to internal RUM
2) Add more inlining of computation of RumItem from the pages (behind a GUC defaulted on)
3) Add option for bson_build_document and fix bson_get_value_text to not quote strings unnecessarily to be compatible with `::text` casts

----
#### AI description  (iteration 1)
#### PR Classification
This PR implements API updates and performance improvements by introducing a new item pointer decoding mechanism in RUM indexes and extending BSON UDF functionalities.

#### PR Summary
The pull request refactors core RUM index scan routines and enhances BSON API functions to improve performance and API consistency. Key changes include:
- **`oss/internal/pg_documentdb_extended_rum` and `pgmongo_rum`**: Introduced new macros and inline functions (e.g., `decode_varbyte_blocknumber`, `rumDataPageLeafReadWithBlockNumberIncr`) along with the boolean flag `RumUseNewItemPtrDecoding` to optimize item pointer decoding.
- **`rumget.c`, `rumentrypage.c`, and `rumdatapage.c`**: Updated the posting tree scan logic to conditionally apply the new decoding approach, streamlining data page processing.
- **`pg_documentdb_core/sql/udfs/bson_io`**: Added new BSON UDFs (such as `bson_build_document`) and overloads (e.g., a three-argument version of `bson_get_value_text`) to expand API capabilities.
- **`pgbson.c` and `bson_io.c`**: Enhanced BSON logging functions with optional quoting support for JSON conversion, improving logging clarity and consistency.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Testing
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Update to match coding standards

----
### Does this PR have any customer impact?
Yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
similarity issue
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a code cleanup aimed at standardizing and clarifying error and diagnostic messages throughout the codebase.

#### PR Summary
The changes update error messages and logging across many modules (aggregation, query, update, index creation, collation, etc.) to use clearer, more precise, and consistent language.
- **Aggregation Operators:** Many files in the aggregation code (e.g. in `bson_aggregation_*` and related operator implementations) now report errors such as division by zero, invalid inputs, and argument count in a formal and uniform manner.
- **Index & Projection Validation
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

update errmsg

----
#### AI description  (iteration 1)
#### PR Classification
This pull request focuses on updating and standardizing error messages across various operators, aggregation stages, and test expectation files.

#### PR Summary
The changes reword error messages throughout the codebase and test outputs to provide clearer, more consistent, and more informative feedback for users. Key updates include:
- **`/oss/internal/pg_documentdb_distributed/src/test/regress/expected/*.out`** – Revised expected error outputs for aggregation, update, and geospatial tests to use clearer phrasing (e.g., for `$merge`, `$pop`, `$indexOfArray`, and `$geoNear`).
- **Core operator files (e.g., `bson_aggregation_output_pipeline.c`, `bson_expression_arithmetic_operators.c`, `bson_expression_conditional_operators.c`)** – Updated error message texts to explicitly state operator requirements and types.
- **Geospatial and vector modules (e.g., files under `bson_geojson_utils.c`, `bson_geospatial_*`, and vector search/index configuration files)** – Standardized validations and error reporting for GeoJSON, CRS, and vector search parameters.
- **Additional modules (e.g., collection view creation and user management files)** – Improved error handling messages to better guide users on input misconfigurations.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

draft

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix: This PR revises error messages and inline comments to improve clarity and consistency across the code and test outputs.

#### PR Summary
The changes refactor and update a large number of error messages across various modules—ranging from aggregation and update operators to geospatial, schema validation, and command handling—to provide more precise diagnostic feedback.
- **Tests (e.g. in `tests/expected/...
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
draft

update error msg

----
#### AI description  (iteration 1)
#### PR Classification
This is a code cleanup and bug‑fix pull request that standardizes and clarifies error messages and log details across multiple modules.

#### PR Summary
The changes improve the consistency, clarity, and accuracy of user‑facing messages and logging throughout the code base. Key changes include:
- In files under directories such as `oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and `pgmongo/src/test/docdb_compat/expected/`, error messages for geospatial query operators (e.g. geoNear, $center, $within) and aggregation stage operators have been rewritten for clarity (e.g. replacing vague “Too many geoNear expressions” with “Excessive number of geoNear query expressions” and clarifying numeric parameter validations).
- Several operators’ messages (e.g. `$cond`, `$not`, `$abs`, `$floor`, `$divide`, `$log`, etc.) have been updated to state exactly how many arguments are required, using
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Description
1. Fixing issues identified by similarity checker
2. Fix the corresponding sql unit tests
3. Fix JSTests failure due to above changes
4. Fix corresponding C# test failure

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a code cleanup that standardizes and improves error message wording across the codebase.

#### PR Summary
The changes update error messages throughout the repository to provide clearer, more consistent, and more detailed feedback when operations fail.
- Updates in geospatial operator files (e.g. in `$geoNear`, `$near`, `$nearSphere`) replace informal phrasing with precise messages.
- Numerous error messages in aggregation and projection test files have been reworded for consistent phrasing
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
* Update CODEOWNERS to match maintainers list
Additional docs needed for:

Governance
Project Maintainers
Update CONTRIBUTING guidelines
Updated Security
Update Code of Conduct
Co-authored-by: Yang Qiao <yangqiao@microsoft.com>
visridha and others added 24 commits September 24, 2025 00:54
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Move order by support into 106.1 schema as a hotfix option

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new feature by adding reverse order support for the 106.1 schema.

#### PR Summary
The changes refine reverse sorting by updating conditional checks in the C code and aligning SQL migration scripts with the new version.
- `oss/pg_documentdb/src/commands/current_op.c`: Added a null check for `detail` before allocating and returning the index specification, ensuring proper handling.
- `oss/pg_documentdb/src/opclass/index_support.c`: Modified reverse order conditions to include the 106.1 patch check along with the existing 107.0 version check.
- `pgmongo/sql/`: Moved the SQL migration file (renamed from `pgmongo--1.106-0--1.107-0.sql` to `pgmongo--1.106-0--1.106-1.sql`) and updated the corresponding sha1 checksum files.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
…ue support

### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

This new option for create indexes creates an index as if it was a UNIQUE composite term index, but without actually creating the constraint on the table, so it is a regular index.
This is the first phase of 3 needed.

We introduce a three-step workflow that lets customers build the index structure in the background (non-blocking), then progressively enable uniqueness enforcement:
1. buildAsUnique (createIndex) — build the index as if unique but without registering any table constraint (non-blocking)
2. prepareUnique (collMod) — register the uniqueness constraint in the system catalogs but mark it unvalidated so it starts enforcing uniqueness for new writes only
3. convertUnique (collMod) — validate/enforce uniqueness for existing rows (walk the table or short-circuit if an existing validated unique index already proves uniqueness)
## Goals
- Allow background construction of unique-capable indexes without blocking writes
- Provide a fast metadata-only step to enable uniqueness for new writes
- Provide a validation step that enforces uniqueness for pre-existing rows only when it is safe
- Preserve data integrity and PostgreSQL metadata correctness
- Expose these steps through familiar MongoDB commands (`createIndex` and `collMod`)
- Migrating non ordered unique indexes to ordered indexes without blocking production writes.
## High-level Workflow
Step 1 — buildAsUnique (createIndex):
- Customer requests an index with storageEngine.buildAsUnique=true.
- The system creates a regular index physically (non-blocking) but records metadata indicating the index is intended to become unique
Step 2 — prepareUnique (collMod):
- Customer requests convert-to-unique preparation via `collMod` (prepareUnique)
- The system registers a constraint in the PostgreSQL catalogs but does not validate it against existing data; it enforces uniqueness on new writes going forward
- This is a fast, mostly metadata-only operation (convalidated = false)
Step 3 — convertUnique (collMod):
- Customer requests final conversion via `collMod` (convertUnique)
- The system validates uniqueness for existing rows by walking the table (or aborts if duplicates are found), or it can skip the full walk when an existing validated unique index already proves uniqueness

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new feature that adds support for the "prepareForUnique" option in index creation.

#### PR Summary
The changes extend the index creation mechanism to allow a non-table unique index to be prepared via the "prepareForUnique" flag, with appropriate error handling and ...

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
…tdb#301)

* added documentation on how to compile and install the binary

Signed-off-by: Vamsi Konakanchi <contact@vamsi-k.com>

* added seperate document for info on building from scratch

Signed-off-by: Vamsi Konakanchi <contact@vamsi-k.com>

* changed release to release-with-symbols for better debugging

Signed-off-by: Vamsi Konakanchi <contact@vamsi-k.com>

* updated the port and added the proper build command for gw

Signed-off-by: Vamsi Konakanchi <contact@vamsi-k.com>

---------

Signed-off-by: Vamsi Konakanchi <contact@vamsi-k.com>
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfixes

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

1) Fix bugs around post-recovery cleanup and incomplete splits in some cases causing Lost path
2) The retry around the failure was also not working due to case differences in errors
3) Ensure tracking vs fixing are tracked by different GUCs.

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix addressing lost index paths in RUM indexes by ensuring incomplete page splits are properly handled.

#### PR Summary
This pull request fixes lost path errors in RUM indexes by enhancing the insertion and page split logic to detect, complete, and clear incomplete splits.
- **`pgmongo_rum/src/rumbtree.c`**: Introduced new functions (e.g., `rumPlaceToPage`, `rumFinishSplit`, `rumFinishOldSplit`) to complete pending page splits during insertions and update parent links appropriately.
- **`pgmongo_rum/src/rum.h`, `rumdatapage.c`, and `rumentrypage.c`**: Updated to track and clear the `RUM_INCOMPLETE_SPLIT` flag and to integrate new callbacks for incomplete split scenarios.
- **`pgmongo_rum/src/rumutil.c` and related configuration files**: Added custom GUC variables (`track_incomplete_split`, `fix_incomplete_split`, `enable_inject_page_split_incomplete`) to control the behavior of incomplete split handling.
- **Test files in `/pgmongo/src/test/regress/`**: Added new regression tests to validate the correct handling of incomplete splits and the associated lost path conditions.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
… Builds & a few index fixes

### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature/Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

- Add tracking of parallel workers in currentOp.
- Ensure index build progress works for blocking index builds
- Fix crash in parallel rum index builds
- Allow RUM parallel index build path even for serial build (so we can unify to one codepath)
- Turn on amcanparalllelbuild for PG >= 17 based on the GUC

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix addressing logging inaccuracies for parallel build operations.

#### PR Summary
This pull request improves logging for current operations and index build progress by incorporating additional metadata and refining SQL queries to better handle parallel workers.
- In `oss/pg_documentdb/src/commands/current_op.c`, new attributes (backend type and leader PID) are extracted from query results and appended to BSON output for accurate identification of parallel workers.
- In `oss/pg_documentdb/src/commands/current_op.c`, progress logging is enhanced by including a "concurrent" flag and updating phase-to-message mappings for index build initialization.
- In `pgmongo/src/infrastructure/index_stats_logging.c`, the index progress query now uses left joins with COALESCE to reliably capture index and table names, improving stats logging for parallel builds.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
…ution

### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Add support for hidden indexes with distributed tables.

- Add function that supports per node distribution with shards: Select all shards for the table and pick one shard per node.
- Add a planner worker function that executes on every shard
- On the shard skip execution if it's not the selected shard
- Run the local function on the worker function.

- Plumb in hooks to execute the index functions remotely.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new feature to support hidden indexes in a distributed environment.

#### PR Summary
The changes integrate hidden index support into distributed index update operations by introducing new per-node functions, updating API hooks, and modifying planner and UDF logic for proper shard routing and logging.
- **`node_distribution_operations.c`**: New file that adds functions (e.g., `UpdateDistributedPostgresIndex` and its worker) to execute index update commands per node in a distributed setting.
- **`diagnostic_commands_common.c`**: Introduces `command_node_worker` to ensure that update commands target the appropriate shard based on table identity.
- **`coll_mod.c` and `coll_mod.h`**: Updated to use `UpdatePostgresIndexWithOverride` and correctly incorporate hidden index modifications with distributed operations.
- **API hooks and metadata files** (e.g., `api_hooks.c`, `api_hooks_def.h`, `api_hooks.h`, and `metadata_cache.c`): Enhanced to support new function pointers and caching mechanisms for hidden index updates via the new command worker.
- **SQL UDF scripts**: New UDF definitions for `command_node_worker` and `update_postgres_index_worker` enable proper registration and execution of hidden index operations in the distributed framework.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
Adding a warmpath aggregate 5M flow for the Rust gateway by following the original design and approach. In summary, we store the information we need in memory and flush every 5 minutes. The data is aggregated based on a key hash of static fields and the values like count, max and min per dimension are upserted.

Related work items: #3761267, #4673400

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
…tended_rum

### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Testing
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Currently documentdb tests only run with default rum and non-composite op-class.

While with PG18 they run with ex_rum as a binary they still don't leverage the extended_rum index am and op-class.

Add an option to add backend tests that test the combo until we make this the default path.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request introduces a new feature by adding support for running backend tests that leverage extended_rum in DocumentDB.

#### PR Summary
The changes integrate an extended_rum test suite into the DocumentDB backend testing infrastructure, ensuring that extended index handlers are properly configured and validated.
- **`src/test/extended_rum_tests`**: Added new SQL test files, expected output files, scheduling files, a Makefile, a shell script (`mutate_schedule.sh`), a custom `postgresql.conf`, and a `.gitignore` to support and isolate extended_rum tests.
- **`src/test/regress/validate_test_output.sh`**: Modified to correctly resolve directories for validating test outputs.
- **Top-level and project Makefiles**: Updated (`Makefile`, `oss/pg_documentdb/Makefile`, and `src/test/Makefile`) with new targets to run extended_rum tests alongside the existing regression tests.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
bugfix
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
no
### Are you introducing any new config? If yes, do you have tests with and without them being set?
no
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
- enable building both locally & in docker
- fix the hardcoded paths

----
#### AI description  (iteration 1)
#### PR Classification
Enhancement: Updates to build scripts and Docker configuration to enable building the gateway both locally and in Docker.

#### PR Summary
This pull request revises several scripts and configuration files to integrate local and containerized build processes for the gateway.
- **`/oss/scripts/emulator_entrypoint.sh`**: Refactors the configuration file handling by defining a variable and updating file permissions and copy paths.
- **`/oss/scripts/build_and_start_gateway.sh`**: Introduces a local build process using Cargo (clean, build, and binary copy) and adjusts the binary execution path.
- **`/oss/.github/containers/Build-Ubuntu/Dockerfile_gateway`**: Updates copy paths to place the gateway binary and configuration into the new subfolder.
- **`/oss/rust-toolchain.toml` & `/oss/pg_documentdb_gw/.gitignore`**: Adds the Cargo component to the toolchain and ensures the built binary is ignored.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
* update CONTRIBUTING.md doc

Signed-off-by: Zhe Li <zhli2@microsoft.com>

added dco signoff guidance.
…ified

### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Have the OSS PGRX install script build release by default for install

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix to correctly set build profiles in the pgrx installation script.

#### PR Summary
This pull request adjusts the pgrx installation script to default to a release build when no profile is specified and to separate profile arguments for installation and packaging.
- In `/oss/scripts/build_and_install_with_pgrx.sh`, the script now introduces `packageProfileArg` and `installProfileArg`, using `--release` for installation when no profile is provided.
- The cargo commands in the script are updated to use the new profile arguments for both the install and package operations.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Turn on some Vacuum cleanup GUCs and Parallel build GUCs by default for RUM

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a configuration update that enables key RUM features by adjusting default GUC settings.

#### PR Summary
The changes update default settings in RUM header files to enable parallel index building and vacuuming of entry items.
- `pgmongo_rum/src/rum.h`: Changed `RUM_DEFAULT_ENABLE_PARALLEL_INDEX_BUILD` from false to true and `RUM_DEFAULT_VACUUM_ENTRY_ITEMS` from false to true.
- `oss/internal/pg_documentdb_extended_rum/src/pg_documentdb_rum.h`: Changed `RUM_DEFAULT_ENABLE_PARALLEL_INDEX_BUILD` from false to true.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Continous TTL was not taking into consideration the max continous timeout but instead the single ttl task so it was always capped at 20 seconds.

TTL descending needed to use the order by function and fullrange so that it goes through the order by push down logic rather than PGs logic.

Also, add an entry to the log to indicate if it is descending or not.

----
#### AI description  (iteration 1)
#### PR Classification
This PR fixes TTL purging issues by correcting the descending sort behavior and ensuring continuous purging.

#### PR Summary
The changes update TTL deletion logic to use descending order when enabled and adjust associated logging and configuration options. This enhances purging consistency and aligns test expectations.
- `pgmongo/src/test/regress/expected/pgmongo_internal_ttl_tests.out` and related test outputs now include the `use_desc_sort` flag in log messages.
- `oss/pg_documentdb_distributed/src/test/regress/expected/commands_create_ttl_indexes.out` and SQL test files have been modified to set descending sort and disable sequential scans.
- `oss/pg_documentdb/src/ttl/ttl_index.c` has been updated to conditionally append descending order queries using index orderby pushdown.
- Configuration files such as `oss/pg_documentdb/src/configs/background_job_configs.c` now define and enable TTL descending sort options (e.g., `EnableTTLDescSort` and `EnableIndexOrderbyPushdown`).
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
no

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
no

### Are you introducing any new config? If yes, do you have tests with and without them being set?
no
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Use GA version of PG18

----
#### AI description  (iteration 1)
#### PR Classification
This pull request updates dependency references to use the GA version of PG18 and adjusts related version settings.

#### PR Summary
The changes update version references to switch from a release candidate to the GA version of PG18 while also bumping the CITUS version. Key modifications include:
- `oss/scripts/setup_versions.sh`: Changed `POSTGRES_18_REF` from "REL_18_RC1" to "REL_18_0" with the corresponding new commit hash and updated `CITUS_13_VERSION` from v13.1.0 to v13.2.0.
- `cgmanifest.json`: Updated the postgres repository commit hash to align with the GA version.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
…he top level folder

### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Move pg_documentdb_extended_rum to the top level folder from the internal folder since documentdb (tests and product) now has a dependency on it.

pg_documentdb_extended_rum is a fork of  the RUM Extension that provides extensions that were needed for documentdb. This includes the ability to push down order by and group by to the index; The ability to do index only scans; improvements on vacuum and handling of page splits; index scan improvements while retaining the original inverted index for text search capabilities that the RUM index had.

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
DevOps

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Move the PG18 build container to the latest image

----
#### AI description  (iteration 1)
#### PR Classification
Configuration update for build container images.

#### PR Summary
This pull request updates the PG18 build container image to use the latest normalized tag, ensuring consistency with the PG17 configuration.
- Updated ``/.pipelines/templates/common_variables.yml`` by replacing the hard-coded tag for PG18 with ``variables.NormalizedContainerTag``.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Order by expressions have a filter for full scan that makes it so it can push down to the index. However the default selectivity of 1% kicks in at the time.  Instead ensure that It has a selectivity of 100%

Also fix list collections with distribution failing on views

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix addressing errors in BSON range operator selectivity and index pushdown behavior.

#### PR Summary
This PR fixes issues with full scan detection and selectivity handling in BSON range queries while enhancing index pushdown and metadata listing tests. The changes improve the correctness of query planning and operator behavior in DocumentDB integrations.
- **`src/query/bson_dollar_selectivity.c` & `src/opclass/index_support.c`**: Added functions to detect full scans and correctly set selectivity (e.g., returning 1.0) for `$range` queries.
- **Test Files (`bson_composite_order_by_index_tests.out`, `list_metadata_cursor_tests.out`, and SQL scripts)**: Extended tests for composite order-by indexes and distributed metadata, ensuring proper index selection and query plans.
- **Configuration and Startup Scripts (`feature_flag_configs.c`, `start_oss_server.sh`)**: Introduced the `setSelectivityForFullScan` flag and updated extended RUM extension settings.
- **SQL Migration Scripts**: Altered operator restrict definitions to use updated selectivity functions for consistent operator handling.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
…umentdb#336)

* Refactor CI workflows to conditionally set OS and PostgreSQL version for pull requests

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* exclude specific internal directories from Debian and RPM packages

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* fix

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* skip check-extended-rum

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* Include check-extended-rum target and tests in building

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* fix

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* update sql tests

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* fix

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* use exclude

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* more

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* use template

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* feedback

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

---------

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Improve pushdown of $elemMatch for composite indexes.

When there's an elemMatch of the form `{ "a": { "$elemMatch": { "$gt": 10, "$lt": 15 }}`
This can scan the index only from 10 -> 15

However given the multi-key index requirement we were doing 2 scans:
- One from 10 to infinity and the other from -infinity to 15.

Optimize this path for $elemMatch.

----
#### AI description  (iteration 1)
#### PR Classification
This PR introduces a new feature that optimizes the pushdown of `$elemMatch` queries for composite indexes.

#### PR Summary
The pull request implements optimized handling of `$elemMatch` predicates on composite indexes by converting them into specialized index pushdown expressions and updating the corresponding test expectations.
- **`oss/pg_documentdb/src/opclass/bson_gin_composite_core.c` & `index_support.c`**: Added logic to detect `$elemMatch` in range queries and convert them into optimized pushdown expressions (via functions like `GetElemMatchIndexPushdownOperator`).
- **Configuration Changes**: Introduced a new GUC (`documentdb.useNewElemMatchIndexOperatorOnPushdown`) in `pg_documentdb/src/configs/feature_flag_configs.c` to control the new behavior.
- **Test Updates**: Modified expected explain outputs in both `/oss/internal/pg_documentdb_distributed/src/test/regress/` and `/pgmongo/src/test/docdb_compat/expected/` to reflect the revised index condition format using `elemMatchIndexOp`.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
# Conflicts:
#	.github/workflows/regress_tests.yml
#	.gitignore
#	CHANGELOG.md
#	CODEOWNERS
#	packaging/build_packages.sh
#	packaging/debian_files/changelog
#	packaging/rpm/spec/documentdb.spec
@mergify mergify bot assigned AlekSi Feb 8, 2026
Comment on lines +26 to +47
if: github.event_name == 'pull_request' && github.event.pull_request.draft == false
strategy:
fail-fast: false
matrix:
include:
- pg_version: 15
arch: amd64
runner: ubuntu-22.04
- pg_version: 16
arch: arm64
runner: ubuntu-22.04-arm
- pg_version: 17
arch: amd64
runner: ubuntu-22.04
uses: ./.github/workflows/regress_tests_template.yml
with:
pg_version: ${{ matrix.pg_version }}
arch: ${{ matrix.arch }}
runner: ${{ matrix.runner }}
citus_version: 12

test-full:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}

Copilot Autofix

AI 11 days ago

To fix the problem, add an explicit permissions block that grants the minimal required scopes to GITHUB_TOKEN. Since this workflow appears to orchestrate tests via a reusable workflow and there is no indication that it needs to write to the repository, pull requests, or other resources, a conservative and safe default is permissions: contents: read at the workflow root. This documents that the workflow is intended to be read-only and prevents it from gaining broader rights if repository/org defaults change later.

Concretely, edit .github/workflows/regress_tests.yml and insert a permissions section near the top-level keys (e.g., after run-name or after on:). Because we cannot see the inner steps and must avoid breaking existing functionality, we should choose a minimal but generally safe set of permissions. The minimal recommended starting point per GitHub’s guidance is contents: read; if the reusable workflow needs more, those can be defined there without changing existing behavior here. No imports or external libraries are involved, only YAML changes.

Suggested changeset 1
.github/workflows/regress_tests.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/regress_tests.yml b/.github/workflows/regress_tests.yml
--- a/.github/workflows/regress_tests.yml
+++ b/.github/workflows/regress_tests.yml
@@ -4,6 +4,9 @@
   group: sql-tests-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
+permissions:
+  contents: read
+
 on:
   workflow_dispatch:
   push:
EOF
@@ -4,6 +4,9 @@
group: sql-tests-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: read

on:
workflow_dispatch:
push:
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines 48 to 69
@@ -40,128 +61,10 @@ jobs:
runner: ubuntu-22.04
- arch: arm64
runner: ubuntu-22.04-arm
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Install CMake 3.22
run: |
sudo apt-get update
sudo apt-get install -y cmake=3.22.*
cmake --version

- name: Install package dependencies
run: |
export PG_VERSION=${{ matrix.pg_version }}
export CITUS_VERSION=12

export LC_ALL=en_US.UTF-8
export LANGUAGE=en_US
export LC_COLLATE=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8

# Import the repository signing key:
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
wget \
curl \
sudo \
gnupg2 \
lsb-release \
tzdata \
build-essential \
pkg-config \
cmake \
git \
locales \
gcc \
gdb \
libipc-run-perl \
unzip \
apt-transport-https \
bison \
flex \
libreadline-dev \
zlib1g-dev \
libkrb5-dev \
software-properties-common \
libtool \
libicu-dev \
libssl-dev

export CLEAN_SETUP=1
export INSTALL_DEPENDENCIES_ROOT=/tmp/install_setup
mkdir -p /tmp/install_setup

# Install dependencies
cp ./scripts/setup_versions.sh /tmp/install_setup

cp ./scripts/install_setup_libbson.sh /tmp/install_setup
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT MAKE_PROGRAM=cmake /tmp/install_setup/install_setup_libbson.sh
cp ./scripts/utils.sh /tmp/install_setup
cp ./scripts/install_setup_postgres.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_setup_postgres.sh -d /usr/lib/postgresql/${PG_VERSION} $POSTGRES_INSTALL_ARG -v ${PG_VERSION}
cp ./scripts/install_setup_rum_oss.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_rum_oss.sh
cp ./scripts/install_setup_citus_core_oss.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_citus_core_oss.sh ${CITUS_VERSION}
cp ./scripts/install_citus_indent.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_citus_indent.sh
cp ./scripts/install_setup_system_rows.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_system_rows.sh
cp ./scripts/install_setup_pg_cron.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_pg_cron.sh
cp ./scripts/install_setup_intel_decimal_math_lib.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_setup_intel_decimal_math_lib.sh
cp ./scripts/install_setup_pcre2.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_setup_pcre2.sh
cp ./scripts/install_setup_pgvector.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_pgvector.sh

sudo apt-get update
sudo apt-get install -qy \
libproj-dev \
libxml2-dev \
libjson-c-dev \
libgdal-dev \
libgeos++-dev \
libgeos-dev \

cp ./scripts/install_setup_postgis.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_postgis.sh

- name: Export pg_config PATH
run: |
echo "/usr/lib/postgresql/${{ matrix.pg_version }}/bin" >> $GITHUB_PATH

- name: Configure, Build and Install DocumentDB
run: |
which pg_config
make
sudo PATH=$PATH make install

- name: Run Tests
run: |
which pg_config
export LC_ALL=en_US.UTF-8
export LANGUAGE=en_US
export LC_COLLATE=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8
make check

- name: Citus Indent
run: |
citus_indent --check || (echo ""; echo "citus_indent failed, refer to CONTRIBUTING.md on how to use citus_indent"; exit 1)

- name: Validate sanity of files
run: |
./scripts/validate_extension_file_state.sh
uses: ./.github/workflows/regress_tests_template.yml
with:
pg_version: ${{ matrix.pg_version }}
arch: ${{ matrix.arch }}
runner: ${{ matrix.runner }}
citus_version: 12

- name: Upload Log Files
if: always()
uses: actions/upload-artifact@v4
with:
name: logs_${{ matrix.runner }}_PG${{ matrix.pg_version }}
overwrite: true
path: "**/*.log"

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}

Copilot Autofix

AI 11 days ago

In general, to fix this class of issue you add an explicit permissions section either at the workflow root (so it applies to all jobs) or per job, and set it to the least privileges required by the workflow steps. For a typical test workflow that only needs to read the repository contents and possibly read packages, contents: read (and optionally packages: read) is an appropriate baseline.

For this specific workflow, the simplest and safest fix without changing existing functionality is to add a top-level permissions block right after the run-name (or before on:) that restricts the GITHUB_TOKEN to repository read access. Since the code we see is only dispatching to a reusable workflow, and there is no evidence here that write access is required, using contents: read is a reasonable minimal starting point and matches GitHub’s own recommendations. This block will apply to both test-pr and test-full, satisfying CodeQL’s requirement and documenting the intended permissions.

Concretely:

  • Edit .github/workflows/regress_tests.yml.

  • Insert:

    permissions:
      contents: read

    after line 2 (run-name: ...) and before the existing concurrency: key on line 3.

  • No imports or other code constructs are required; this is a pure YAML configuration change.

Suggested changeset 1
.github/workflows/regress_tests.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/regress_tests.yml b/.github/workflows/regress_tests.yml
--- a/.github/workflows/regress_tests.yml
+++ b/.github/workflows/regress_tests.yml
@@ -1,5 +1,7 @@
 name: DocumentDB sql tests
 run-name: ${{ github.event.pull_request.title || '' }}
+permissions:
+  contents: read
 concurrency:
   group: sql-tests-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
EOF
@@ -1,5 +1,7 @@
name: DocumentDB sql tests
run-name: ${{ github.event.pull_request.title || '' }}
permissions:
contents: read
concurrency:
group: sql-tests-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines +21 to +141
runs-on: ${{ inputs.runner }}
name: Run sql tests on ${{ inputs.runner }} with pg ${{ inputs.pg_version }}
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Install package dependencies
run: |
export PG_VERSION=${{ inputs.pg_version }}
export CITUS_VERSION=${{ inputs.citus_version }}

export LC_ALL=en_US.UTF-8
export LANGUAGE=en_US
export LC_COLLATE=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8

# Import the repository signing key:
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
wget \
curl \
sudo \
gnupg2 \
lsb-release \
tzdata \
build-essential \
pkg-config \
cmake \
git \
locales \
gcc \
gdb \
libipc-run-perl \
unzip \
apt-transport-https \
bison \
flex \
libreadline-dev \
zlib1g-dev \
libkrb5-dev \
software-properties-common \
libtool \
libicu-dev \
libssl-dev

export CLEAN_SETUP=1
export INSTALL_DEPENDENCIES_ROOT=/tmp/install_setup
mkdir -p /tmp/install_setup

# Install dependencies
cp ./scripts/setup_versions.sh /tmp/install_setup

cp ./scripts/install_setup_libbson.sh /tmp/install_setup
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT MAKE_PROGRAM=cmake /tmp/install_setup/install_setup_libbson.sh
cp ./scripts/utils.sh /tmp/install_setup
cp ./scripts/install_setup_postgres.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_setup_postgres.sh -d /usr/lib/postgresql/${PG_VERSION} $POSTGRES_INSTALL_ARG -v ${PG_VERSION}
cp ./scripts/install_setup_rum_oss.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_rum_oss.sh
cp ./scripts/install_setup_citus_core_oss.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_citus_core_oss.sh ${CITUS_VERSION}
cp ./scripts/install_citus_indent.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_citus_indent.sh
cp ./scripts/install_setup_system_rows.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_system_rows.sh
cp ./scripts/install_setup_pg_cron.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_pg_cron.sh
cp ./scripts/install_setup_intel_decimal_math_lib.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_setup_intel_decimal_math_lib.sh
cp ./scripts/install_setup_pcre2.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT /tmp/install_setup/install_setup_pcre2.sh
cp ./scripts/install_setup_pgvector.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_pgvector.sh

sudo apt-get update
sudo apt-get install -qy \
libproj-dev \
libxml2-dev \
libjson-c-dev \
libgdal-dev \
libgeos++-dev \
libgeos-dev \

cp ./scripts/install_setup_postgis.sh /tmp/install_setup/
sudo INSTALL_DEPENDENCIES_ROOT=$INSTALL_DEPENDENCIES_ROOT PGVERSION=$PG_VERSION /tmp/install_setup/install_setup_postgis.sh

- name: Export pg_config PATH
run: |
echo "/usr/lib/postgresql/${{ inputs.pg_version }}/bin" >> $GITHUB_PATH

- name: Configure, Build and Install DocumentDB
run: |
which pg_config
make
sudo PATH=$PATH make install

- name: Run Tests
run: |
which pg_config
export LC_ALL=en_US.UTF-8
export LANGUAGE=en_US
export LC_COLLATE=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8
make check

- name: Citus Indent
run: |
citus_indent --check || (echo ""; echo "citus_indent failed, refer to CONTRIBUTING.md on how to use citus_indent"; exit 1)

- name: Validate sanity of files
run: |
./scripts/validate_extension_file_state.sh

- name: Upload Log Files
if: always()
uses: actions/upload-artifact@v4
with:
name: logs_${{ inputs.runner }}_PG${{ inputs.pg_version }}
overwrite: true
path: "**/*.log"

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 11 days ago

To fix the problem, explicitly declare restricted GITHUB_TOKEN permissions for this workflow or job. Since this is a reusable workflow that checks out code and uploads artifacts, it only needs read access to repository contents; no write operations to the repo or PRs are required.

The best minimal fix is to add a root-level permissions block right under the workflow name: (before on:). This will apply to all jobs in this workflow that do not override permissions. For this workflow, contents: read is sufficient; other scopes (issues, pull-requests, etc.) are not needed based on the provided steps. No imports or additional methods are required because this is just YAML configuration.

Concretely, in .github/workflows/regress_tests_template.yml, insert:

permissions:
  contents: read

on new lines after line 1 (name: Reusable SQL Tests). No other parts of the file need to change.

Suggested changeset 1
.github/workflows/regress_tests_template.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/regress_tests_template.yml b/.github/workflows/regress_tests_template.yml
--- a/.github/workflows/regress_tests_template.yml
+++ b/.github/workflows/regress_tests_template.yml
@@ -1,5 +1,8 @@
 name: Reusable SQL Tests
 
+permissions:
+  contents: read
+
 on:
   workflow_call:
     inputs:
EOF
@@ -1,5 +1,8 @@
name: Reusable SQL Tests

permissions:
contents: read

on:
workflow_call:
inputs:
Copilot is powered by AI and may make mistakes. Always verify output.
@AlekSi AlekSi added packages PRs that should build packages trust PRs that can access Actions secrets not ready Issues that are not ready to be worked on; PRs that should skip CI and removed not ready Issues that are not ready to be worked on; PRs that should skip CI labels Feb 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

packages PRs that should build packages trust PRs that can access Actions secrets

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Comments