Update instructions on vm.max_map_count persistence#5072
Update instructions on vm.max_map_count persistence#5072yetanothertw wants to merge 5 commits intomainfrom
Conversation
Fixes #5047 and updates the instructions to explain that for systemd-based systems the Elasticsearch package may install a sysctl file (`/usr/lib/sysctl.d/elasticsearch.conf`) that sets `vm.max_map_count` to 262144 and that this can override a value set only in `/etc/sysctl.conf`.
✅ Vale Linting ResultsNo issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
🔍 Preview links for changed docs |
|
Hi @kunisen, I remember you've recently worked on updating the docs around the Many thanks in advance! 🙏 |
eedugon
left a comment
There was a problem hiding this comment.
LGTM! Just a small comment added, for your consideration. Maybe unneeded!
|
Thanks @yetanothertw
Where I think if in this case we create 99-elasticsearch.conf, that will be overridden eventually by I think we probably could say something like: check first, if existing, then modify; if no, then create new one. |
great point :( it probably does, I was expecting the opposite behavior but probably you are right! |
Thank you for the suggestion! I've made a few more changes, could you please review when you get the chance, @kunisen ? |
| ```sh | ||
| sysctl -w vm.max_map_count=1048576 | ||
| ``` |
There was a problem hiding this comment.
@yetanothertw I don't have the permission to do suggestion. I will comment about my thought here about this.
:: Background
[1] This parameter, IIUC, if we use sysctl command to update it, it will not persist after reboot.
[2] Given this is a file under /etc/sysctl.d/ directory, and it should has an entry similar to vm.max_map_count= 262144, the action we need to do is to manually change that value from 262144 (or any value lower than 1048576) to 1048576, in that file (filename can be arbitrary).
:: My proposal
I would think we could say it like:
If the parameter already exists in a file under /etc/sysctl.d/: Update the entry to 1048576. After the update, the line should read: vm.max_map_count=1048576.
WDYT?
cc @eedugon for a pairing eyes.
There was a problem hiding this comment.
Yes, that makes sense! Thanks for catching that, I've updated that line
|
|
||
|
|
||
| :::{note} | ||
| On systemd-based systems, the {{es}} package might install `/usr/lib/sysctl.d/elasticsearch.conf` with a lower value such as `262144`. A file with the same name under `/etc/sysctl.d/` takes precedence, so creating `/etc/sysctl.d/99-elasticsearch.conf` is a valid option to permanently set the value of `vm.max_map_count` to `1048576`. Do not edit files under `/usr/lib/sysctl.d/` directly, as they are managed by the package and may be overwritten on upgrade. |
There was a problem hiding this comment.
@yetanothertw @eedugon my understanding is, the file is loaded in a lexicographic order, or ascii code order, so e (elasticsearch.conf) will win 99 (99-elasticsearch.conf). (Means settings in elasticsearch.conf will override the ones in 99-elasticsearch.conf, so that vm.max_map_count won't be updated as expected, because the value in elasticsearch.conf is the old one)
Which means it's opposite in this description - Given there's already elasticsearch.conf exists (per description in #5047), then it won't take effect to override that file even if we define a new one called 99-elasticsearch.conf).
If we want to make sure a file there will win elasticsearch.conf, then how about we say zz-elasticsearch.conf which will make sure that "zz" will be loaded last?
There was a problem hiding this comment.
Thanks, I've updated the name from 99-elasticsearch.conf to zz-elasticsearch.conf.
|
@yetanothertw may I have some more time to test this please? I checked around and found
But given this statement in #5047,
I think I need to setup a test env and have some test in reality to prove the theory. FWIW, this is my investigation result. :: The hierarchy Files are loaded based on their location first, and their name second.
Highest priority means the last one to apply the value, means the actual winner. Reference doc: https://www.freedesktop.org/software/systemd/man/latest/sysctl.d.html
You can see the Linux doc page says package use their config in |
[1] my test resultI did some detailed test Detailssteps Spin up Debian 12Start ES
for quick Check vm.max_map_countonly in /usr/lib/sysctl.d/elasticsearch.conf override the value in /etc/sysctl.conf check apply change check applied status In conclusion,
:: 1-A)
If we use :: 1-B)
In reality, it's IMHO, we could probably make this part more clear, by saying: While RPM and Debian packages attempt to configure this automatically, setting the value in [2] based on thatI would think we only need to change doc with above 1-A and 1-B, based on my test. That said, I'd like to follow up with the user I can make sure we are all agreeing the above proposal, and make commit to change it in that way, and then ask for your doc side follow up 🙏 |
|
thanks for the research @kunisen , I agree with your view, and it's better to tell our users to configure directly @yetanothertw , would you take care of it? |
@kunisen thanks so much for testing this! I've applied the changes to this PR, so please feel free to follow up with the issue reporter directly. |
|
Thanks @yetanothertw and @eedugon Meanwhile, could we please keep this doc PR open for a while, in case we may need to do further wording tuning based on the sync with the user? Thanks for your understanding and patience! |
Summary
Fixes #5047 and updates the instructions to explain that for systemd-based systems the Elasticsearch package may install a sysctl file (
/usr/lib/sysctl.d/elasticsearch.conf) that setsvm.max_map_countto 262144 and that this can override a value set only in/etc/sysctl.conf.Generative AI disclosure
Used composer-1 to validate the order in which Debian
sysctl.ddirectories take precedence and find the corresponding manpage.