Skip to content

Update instructions on vm.max_map_count persistence#5072

Open
yetanothertw wants to merge 5 commits intomainfrom
5047-max-count
Open

Update instructions on vm.max_map_count persistence#5072
yetanothertw wants to merge 5 commits intomainfrom
5047-max-count

Conversation

@yetanothertw
Copy link
Contributor

@yetanothertw yetanothertw commented Feb 9, 2026

Summary

Fixes #5047 and updates the instructions to explain that for systemd-based systems the Elasticsearch package may install a sysctl file (/usr/lib/sysctl.d/elasticsearch.conf) that sets vm.max_map_count to 262144 and that this can override a value set only in /etc/sysctl.conf.

Generative AI disclosure

  1. Did you use a generative AI (GenAI) tool to assist in creating this contribution?
  • Yes
  • No

Used composer-1 to validate the order in which Debian sysctl.d directories take precedence and find the corresponding manpage.

Fixes #5047 and updates the instructions to explain that for systemd-based systems the Elasticsearch package may install a sysctl file (`/usr/lib/sysctl.d/elasticsearch.conf`) that sets `vm.max_map_count` to 262144 and that this can override a value set only in `/etc/sysctl.conf`.
@github-actions
Copy link
Contributor

github-actions bot commented Feb 9, 2026

✅ Vale Linting Results

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 9, 2026

🔍 Preview links for changed docs

@yetanothertw yetanothertw marked this pull request as ready for review February 9, 2026 17:40
@yetanothertw yetanothertw requested a review from a team as a code owner February 9, 2026 17:40
@yetanothertw
Copy link
Contributor Author

Hi @kunisen, I remember you've recently worked on updating the docs around the vm.max_map_count value. Could you take a look at the associated issue and review the PR, please?

Many thanks in advance! 🙏

Copy link
Contributor

@eedugon eedugon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just a small comment added, for your consideration. Maybe unneeded!

@kunisen
Copy link
Contributor

kunisen commented Feb 10, 2026

Thanks @yetanothertw
I think that covers if there's no file having the value in /etc/sysctl.d/. But the original ticket said there's already a file (link):

/usr/lib/sysctl.d/elasticsearch.conf is created with:
vm.max_map_count=262144

Where I think if in this case we create 99-elasticsearch.conf, that will be overridden eventually by elasticsearch.conf, because alphabet e will win 99.

I think we probably could say something like: check first, if existing, then modify; if no, then create new one.
Just an idea proposal:
#5072 (comment)

@eedugon
Copy link
Contributor

eedugon commented Feb 10, 2026

@kunisen

Where I think if in this case we create 99-elasticsearch.conf, that will be overridden eventually by elasticsearch.conf, because alphabet e will win 99.

great point :( it probably does, I was expecting the opposite behavior but probably you are right!

@yetanothertw
Copy link
Contributor Author

I think we probably could say something like: check first, if existing, then modify; if no, then create new one.
Just an idea proposal:
#5072 (comment)

Thank you for the suggestion! I've made a few more changes, could you please review when you get the chance, @kunisen ?

Comment on lines 30 to 32
```sh
sysctl -w vm.max_map_count=1048576
```
Copy link
Contributor

@kunisen kunisen Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yetanothertw I don't have the permission to do suggestion. I will comment about my thought here about this.

:: Background
[1] This parameter, IIUC, if we use sysctl command to update it, it will not persist after reboot.
[2] Given this is a file under /etc/sysctl.d/ directory, and it should has an entry similar to vm.max_map_count= 262144, the action we need to do is to manually change that value from 262144 (or any value lower than 1048576) to 1048576, in that file (filename can be arbitrary).

:: My proposal
I would think we could say it like:


If the parameter already exists in a file under /etc/sysctl.d/: Update the entry to 1048576. After the update, the line should read: vm.max_map_count=1048576.


WDYT?

cc @eedugon for a pairing eyes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that makes sense! Thanks for catching that, I've updated that line



:::{note}
On systemd-based systems, the {{es}} package might install `/usr/lib/sysctl.d/elasticsearch.conf` with a lower value such as `262144`. A file with the same name under `/etc/sysctl.d/` takes precedence, so creating `/etc/sysctl.d/99-elasticsearch.conf` is a valid option to permanently set the value of `vm.max_map_count` to `1048576`. Do not edit files under `/usr/lib/sysctl.d/` directly, as they are managed by the package and may be overwritten on upgrade.
Copy link
Contributor

@kunisen kunisen Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yetanothertw @eedugon my understanding is, the file is loaded in a lexicographic order, or ascii code order, so e (elasticsearch.conf) will win 99 (99-elasticsearch.conf). (Means settings in elasticsearch.conf will override the ones in 99-elasticsearch.conf, so that vm.max_map_count won't be updated as expected, because the value in elasticsearch.conf is the old one)

Which means it's opposite in this description - Given there's already elasticsearch.conf exists (per description in #5047), then it won't take effect to override that file even if we define a new one called 99-elasticsearch.conf).

If we want to make sure a file there will win elasticsearch.conf, then how about we say zz-elasticsearch.conf which will make sure that "zz" will be loaded last?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've updated the name from 99-elasticsearch.conf to zz-elasticsearch.conf.

@kunisen
Copy link
Contributor

kunisen commented Feb 12, 2026

@yetanothertw may I have some more time to test this please?

I checked around and found /etc/sysctl.conf is supposed to be the final winner that value will take effect. So our existing doc seems to be correct:
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/vm-max-map-count

To set this value permanently, update the vm.max_map_count setting in /etc/sysctl.conf. To verify after rebooting, run sysctl vm.max_map_count.

But given this statement in #5047,

This file is loaded later by systemd-sysctl and overrides any value set in /etc/sysctl.conf

I think I need to setup a test env and have some test in reality to prove the theory.


FWIW, this is my investigation result.

:: The hierarchy

Files are loaded based on their location first, and their name second.

  1. /etc/sysctl.d/*.conf (User Overrides — Highest Priority)
  2. /etc/sysctl.conf (Traditional Local Config)
  3. /run/sysctl.d/*.conf (Runtime Overrides)
  4. /usr/lib/sysctl.d/*.conf (Vendor/Package Defaults — Lowest Priority)

Highest priority means the last one to apply the value, means the actual winner.


Reference doc: https://www.freedesktop.org/software/systemd/man/latest/sysctl.d.html

Configuration files are read from directories in /etc/, /run/, /usr/local/lib/, and /usr/lib/, in order of precedence, as listed in the SYNOPSIS section above. Files must have the ".conf" extension. Files in /etc/ override files with the same name in /run/, /usr/local/lib/, and /usr/lib/. Files in /run/ override files with the same name under /usr/.
...
Packages should install their configuration files in /usr/lib/ (distribution packages) or /usr/local/lib/ (local installs) [1]. Files in /etc/ are reserved for the local administrator, who may use this logic to override the configuration files installed by vendor packages.

You can see the Linux doc page says package use their config in /usr/lib/ is the right way to go, and also in case of overriding, we should use /etc/, where our doc is saying that /etc/sysctl.conf is the way to go.


cc @yetanothertw @eedugon

@kunisen
Copy link
Contributor

kunisen commented Feb 12, 2026

@yetanothertw @eedugon

[1] my test result

I did some detailed test

Details

steps

Spin up Debian 12

Start ES

[1]
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/install-elasticsearch-with-debian-package#deb-key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

[2]
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/install-elasticsearch-with-debian-package#step-2-install-es

sudo apt-get install apt-transport-https
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-9.x.list
sudo apt-get update && sudo apt-get install elasticsearch

[3]
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/install-elasticsearch-with-debian-package#step-3-set-up-the-node-for-connectivity

If you're running a single-node cluster, then skip to the next step.

[4]
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/install-elasticsearch-with-debian-package#deb-enable-indices
skip

[5]
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/install-elasticsearch-with-debian-package#running-systemd

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

sudo systemctl start elasticsearch.service

[6]
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/install-elasticsearch-with-debian-package#step-6-reset-the-elastic-superuser-password

[7]
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/install-elasticsearch-with-debian-package#deb-check-running

for quick

$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200
{
  "name" : "kuni-test-debian",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "b7_UNx4wScSzpGDw4lIcuA",
  "version" : {
    "number" : "9.3.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "17b451d8979a29e31935fe1eb901310350b30e62",
    "build_date" : "2026-01-29T10:05:46.708397977Z",
    "build_snapshot" : false,
    "lucene_version" : "10.3.2",
    "minimum_wire_compatibility_version" : "8.19.0",
    "minimum_index_compatibility_version" : "8.0.0"
  },
  "tagline" : "You Know, for Search"
}

Check vm.max_map_count

only in /usr/lib/sysctl.d/elasticsearch.conf

kuniyasu_sen@kuni-test-debian:~$ cat /usr/lib/sysctl.d/elasticsearch.conf
vm.max_map_count=262144

kuniyasu_sen@kuni-test-debian:~$ grep -r vm.max_map_count /etc/sysctl.d/

override the value in /etc/sysctl.conf

echo "vm.max_map_count=1048576" | sudo tee -a /etc/sysctl.conf

check

# note that we need to use `-R` but not `-r` to search symbolink - 99-sysctl.conf is a symbolink
root@kuni-test-debian:~# grep -R vm.max_map_count /etc/sysctl.d/
/etc/sysctl.d/99-sysctl.conf:vm.max_map_count=1048576

apply change

root@kuni-test-debian:~# sysctl --system | grep vm.max_map_count
vm.max_map_count = 1048576
vm.max_map_count = 262144
vm.max_map_count = 1048576

check applied status

root@kuni-test-debian:~# sysctl vm.max_map_count
vm.max_map_count = 1048576

In conclusion,

  • our doc is not wrong
  • but maybe we could improve something

:: 1-A)
https://www.elastic.co/docs/deploy-manage/deploy/self-managed/vm-max-map-count

To verify after rebooting

If we use sudo sysctl --system, then we could get this value reflected immediately.
IMHO, we could say "To verify after rebooting, or sudo sysctl --system to reflect it immediately without rebooting" for completeness.

:: 1-B)

The RPM and Debian packages will configure this setting automatically. No further configuration is required.

In reality, it's /etc/sysctl.conf will override value set in /usr/lib/sysctl.d/elasticsearch.conf.
So no need to actually do "modify /usr/lib/sysctl.d/elasticsearch.conf and set vm.max_map_count=1048576" if we do it in /etc/sysctl.conf

IMHO, we could probably make this part more clear, by saying:


While RPM and Debian packages attempt to configure this automatically, setting the value in /etc/sysctl.conf ensures it takes precedence. By following the steps below, you guarantee the correct value is applied, overriding any package-level defaults.


[2] based on that

I would think we only need to change doc with above 1-A and 1-B, based on my test.
But I want to make sure we all are on the correct page.

That said, I'd like to follow up with the user @\guidogeniox directly if @yetanothertw @eedugon you think that's reasonable.

I can make sure we are all agreeing the above proposal, and make commit to change it in that way, and then ask for your doc side follow up 🙏

@eedugon
Copy link
Contributor

eedugon commented Feb 12, 2026

thanks for the research @kunisen , I agree with your view, and it's better to tell our users to configure directly /etc/sysctl.conf, as it's the most common way. Thankfully it has the highest priority.

@yetanothertw , would you take care of it?

@yetanothertw
Copy link
Contributor Author

I would think we only need to change doc with above 1-A and 1-B, based on my test. But I want to make sure we all are on the correct page.

That said, I'd like to follow up with the user @\guidogeniox directly if @yetanothertw @eedugon you think that's reasonable.

@kunisen thanks so much for testing this! I've applied the changes to this PR, so please feel free to follow up with the issue reporter directly.

Copy link
Contributor

@eedugon eedugon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kunisen
Copy link
Contributor

kunisen commented Feb 13, 2026

Thanks @yetanothertw and @eedugon
I did a follow up in #5047 (comment).

Meanwhile, could we please keep this doc PR open for a while, in case we may need to do further wording tuning based on the sync with the user?

Thanks for your understanding and patience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

**Documentation issue: vm.max_map_count persistence on Debian 12**

3 participants