cms: Reduce stack usage in generate_digest() #126
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The existing code copies an unchecked number of bytes onto the stack, adds padding, and then digests that. Instead, digest it directly and then digest the padding separately to avoid the memset and copy onto the stack. At the same time, make the debug message more useful since previously it wrote out an arbitrary userspace pointer.
Before:
...
pesum: 1747920082.420641 cms_pe_common.c:generate_digest():265: digesting ba000 + 1c000 pesum: 1747920082.420692 cms_pe_common.c:generate_digest():265: digesting d6000 + 1000 pesum: 1747920082.421691 cms_pe_common.c:generate_digest():285: digesting 7fffdec34af0 + 1f340 pesum: 1747920082.421744 cms_pe_common.c:generate_digest():293: end of hash ee8240fefb898334933d7a34a6c0d2a9774c13f8c3e58ac0111393b3fdbffd7c shimx64-unsigned.efi
After:
...
pesum: 1747920070.894430 cms_pe_common.c:generate_digest():265: digesting ba000 + 1c000 pesum: 1747920070.894489 cms_pe_common.c:generate_digest():265: digesting d6000 + 1000 pesum: 1747920070.895417 cms_pe_common.c:generate_digest():279: digesting d7000 + 1f33d pesum: 1747920070.895439 cms_pe_common.c:generate_digest():286: padding to 1f340 pesum: 1747920070.895452 cms_pe_common.c:generate_digest():289: end of hash ee8240fefb898334933d7a34a6c0d2a9774c13f8c3e58ac0111393b3fdbffd7c shimx64-unsigned.efi