From 7f6706ae17fe208c2e950bc84fc4586d3f086a1d Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Fri, 23 Jan 2026 11:03:43 +0000 Subject: [PATCH 1/7] [Security] Gap auto fill known issue --- .../elastic-security/known-issues.md | 79 +++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/release-notes/elastic-security/known-issues.md b/release-notes/elastic-security/known-issues.md index 041e4bc3e1..3317c664d5 100644 --- a/release-notes/elastic-security/known-issues.md +++ b/release-notes/elastic-security/known-issues.md @@ -16,6 +16,85 @@ Known issues are significant defects or limitations that may impact your impleme % ::: +:::{dropdown} Gap auto fill scheduler task might fail after upgrading from {{stack}} versions earlier than 8.9 + +Applies to: {{stack}} upgrades from versions earlier than 8.9 + +**Impact** + +After upgrading from a {{stack}} version earlier than 8.9, the gap auto fill scheduler task that queries the event log might fail with a security exception, and gap fill might not update gaps. This issue affects the following features: + +* **Gap auto fill scheduler**: The task is created, but logs show an error similar to the following: + + ``` + Error during execution: querying for Event Log by for type "alert" and ids "c2e40c59-f403-4c0a-9791-6320a2a1ae29,b0604a70-c2c5-44f1-9ca5-5878871ccdaf" failed with: security_exceptionRoot causes:security_exception: action [indices:data/read/search] is unauthorized for user [elastic/kibana] with effective roles [elastic/kibana], this action is granted by the index privileges [read,all] + ``` + +* **Gap fills**: Gap fills are scheduled but not updated. + +* **Rule deletion**: If a rule has gaps and you delete the rule, the rule is removed but the gaps are not marked as deleted. You may see incorrect numbers when viewing total rules with gaps. + +**Root cause** + +When upgrading from {{stack}} versions earlier than 8.9, the old event log index is reindexed with a new name: + +* Old index: `.reindexed-v8-kibana-event-log-{version}-000001` +* Aliases: `.kibana-event-log-{version}`, `.kibana-event-log-{version}-000001` + +Starting in {{stack}} 8.9.0, a new data stream (`.kibana-event-log-ds`) was introduced for event log storage. + +The `elastic/kibana` service account has permissions to access the new data stream but does not have permissions to access the old reindexed indices. When {{kib}} queries `.kibana-event-log-*`, it matches both the new data stream and the old reindexed index, causing Point-in-Time (PIT) operations to fail. + +**Workaround** + +Migrate data from the old reindexed index to the new data stream, then delete the old index. + +1. **Identify the old index**: + + ```console + GET .kibana-event-log-* + ``` + + Look for indices with names like `.reindexed-v8-kibana-event-log-{version}-*`. + +2. **Reindex data to the new data stream**: + + ```console + POST _reindex + { + "source": { + "index": ".reindexed-v8-kibana-event-log-7.17.29-000001" <1> + }, + "dest": { + "index": ".kibana-event-log-ds", + "op_type": "create" + } + } + ``` + + 1. Replace `7.17.29` with your version number. + +3. **Delete the old index**: + + ```console + DELETE .reindexed-v8-kibana-event-log-7.17.29-000001 + ``` + +4. **Verify**: + + ```console + GET .kibana-event-log-* + ``` + + Only the data stream (`.kibana-event-log-ds`) and its backing indices (`.ds-.kibana-event-log-ds-*`) should remain. + +:::{important} +* **Backup**: Consider backing up your data before performing these operations in production environments. +* **Event log retention**: Event log data has a default retention of 90 days. If you don't need historical data, you can skip the reindex step and simply delete the old index and its aliases. +::: + +::: + :::{dropdown} Intermittent blue screen due to conflict with Windows ODX in {{elastic-defend}} Applies to: 8.19.8, 8.19.9, 9.1.8, 9.1.9, 9.2.2, and 9.2.3 From d7d0b944167940d9fd88d9a8ba54f27553d8a91b Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 27 Jan 2026 12:25:00 -0500 Subject: [PATCH 2/7] Update release-notes/elastic-security/known-issues.md --- release-notes/elastic-security/known-issues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-notes/elastic-security/known-issues.md b/release-notes/elastic-security/known-issues.md index 3317c664d5..a89ca152fa 100644 --- a/release-notes/elastic-security/known-issues.md +++ b/release-notes/elastic-security/known-issues.md @@ -22,7 +22,7 @@ Applies to: {{stack}} upgrades from versions earlier than 8.9 **Impact** -After upgrading from a {{stack}} version earlier than 8.9, the gap auto fill scheduler task that queries the event log might fail with a security exception, and gap fill might not update gaps. This issue affects the following features: +After upgrading from a {{stack}} version earlier than 8.9 to 9.3, the gap auto fill scheduler task that queries the event log might fail with a security exception, and gap fill might not update gaps. This issue affects the following features: * **Gap auto fill scheduler**: The task is created, but logs show an error similar to the following: From bef01e55352879c1c48888a7be16332823a8ac68 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 27 Jan 2026 12:31:30 -0500 Subject: [PATCH 3/7] Update release-notes/elastic-security/known-issues.md --- release-notes/elastic-security/known-issues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-notes/elastic-security/known-issues.md b/release-notes/elastic-security/known-issues.md index a89ca152fa..9919f13e88 100644 --- a/release-notes/elastic-security/known-issues.md +++ b/release-notes/elastic-security/known-issues.md @@ -30,7 +30,7 @@ After upgrading from a {{stack}} version earlier than 8.9 to 9.3, the gap auto f Error during execution: querying for Event Log by for type "alert" and ids "c2e40c59-f403-4c0a-9791-6320a2a1ae29,b0604a70-c2c5-44f1-9ca5-5878871ccdaf" failed with: security_exceptionRoot causes:security_exception: action [indices:data/read/search] is unauthorized for user [elastic/kibana] with effective roles [elastic/kibana], this action is granted by the index privileges [read,all] ``` -* **Gap fills**: Gap fills are scheduled but not updated. +* **Gap fills**: Manual runs are scheduled to fill gaps, but gap statuses aren't updated to `Filled` after the manual runs complete. * **Rule deletion**: If a rule has gaps and you delete the rule, the rule is removed but the gaps are not marked as deleted. You may see incorrect numbers when viewing total rules with gaps. From 82a190ba2f8fff9564b94138f9dcef3206708771 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon Date: Fri, 30 Jan 2026 10:01:35 -0500 Subject: [PATCH 4/7] Removes auto gap fill issues --- release-notes/elastic-security/known-issues.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/release-notes/elastic-security/known-issues.md b/release-notes/elastic-security/known-issues.md index 9919f13e88..79ff8ec78a 100644 --- a/release-notes/elastic-security/known-issues.md +++ b/release-notes/elastic-security/known-issues.md @@ -22,13 +22,7 @@ Applies to: {{stack}} upgrades from versions earlier than 8.9 **Impact** -After upgrading from a {{stack}} version earlier than 8.9 to 9.3, the gap auto fill scheduler task that queries the event log might fail with a security exception, and gap fill might not update gaps. This issue affects the following features: - -* **Gap auto fill scheduler**: The task is created, but logs show an error similar to the following: - - ``` - Error during execution: querying for Event Log by for type "alert" and ids "c2e40c59-f403-4c0a-9791-6320a2a1ae29,b0604a70-c2c5-44f1-9ca5-5878871ccdaf" failed with: security_exceptionRoot causes:security_exception: action [indices:data/read/search] is unauthorized for user [elastic/kibana] with effective roles [elastic/kibana], this action is granted by the index privileges [read,all] - ``` +After upgrading from a {{stack}} version earlier than 8.9 to 9.3, you might encounter the following issues with gap fill functionality: * **Gap fills**: Manual runs are scheduled to fill gaps, but gap statuses aren't updated to `Filled` after the manual runs complete. From e08d287c770892aab70944e7ff37daa37cb85ded Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Fri, 30 Jan 2026 10:03:08 -0500 Subject: [PATCH 5/7] Apply suggestion from @nastasha-solomon --- release-notes/elastic-security/known-issues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-notes/elastic-security/known-issues.md b/release-notes/elastic-security/known-issues.md index 79ff8ec78a..17e1713e60 100644 --- a/release-notes/elastic-security/known-issues.md +++ b/release-notes/elastic-security/known-issues.md @@ -18,7 +18,7 @@ Known issues are significant defects or limitations that may impact your impleme :::{dropdown} Gap auto fill scheduler task might fail after upgrading from {{stack}} versions earlier than 8.9 -Applies to: {{stack}} upgrades from versions earlier than 8.9 +Applies to: 9.3 **Impact** From 32f2a3b0ca116cd15ff0f04a43af5928be601fbc Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Fri, 30 Jan 2026 10:04:59 -0500 Subject: [PATCH 6/7] Apply suggestion from @nastasha-solomon --- release-notes/elastic-security/known-issues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-notes/elastic-security/known-issues.md b/release-notes/elastic-security/known-issues.md index 17e1713e60..f58586d6d7 100644 --- a/release-notes/elastic-security/known-issues.md +++ b/release-notes/elastic-security/known-issues.md @@ -16,7 +16,7 @@ Known issues are significant defects or limitations that may impact your impleme % ::: -:::{dropdown} Gap auto fill scheduler task might fail after upgrading from {{stack}} versions earlier than 8.9 +:::{dropdown} Details about gap fills aren't properly updated Applies to: 9.3 From 2bdb1f69a6d88c34b6e316097dff3d0d7f7b9268 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Fri, 30 Jan 2026 11:24:21 -0500 Subject: [PATCH 7/7] Apply suggestion from @nastasha-solomon --- release-notes/elastic-security/known-issues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-notes/elastic-security/known-issues.md b/release-notes/elastic-security/known-issues.md index f58586d6d7..8c49149b74 100644 --- a/release-notes/elastic-security/known-issues.md +++ b/release-notes/elastic-security/known-issues.md @@ -22,7 +22,7 @@ Applies to: 9.3 **Impact** -After upgrading from a {{stack}} version earlier than 8.9 to 9.3, you might encounter the following issues with gap fill functionality: +After upgrading to 9.3 from a {{stack}} version earlier than 8.9, you might encounter the following issues with gap fill functionality: * **Gap fills**: Manual runs are scheduled to fill gaps, but gap statuses aren't updated to `Filled` after the manual runs complete.