Improve attachment exporting, including making names file-system-safe #707
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR includes several fixes and improvements for exporting attachments. This includes:
Details
File-System-Safety
Share and Group names are arbitrary strings and could contain characters that are not valid on file systems such as forward slashes ('/') and theoretically things like control characters. This applies to attachment file names also, though less commonly so since they usually will have a name from the uploading user's file system.
Thus, when exporting attachments, these strings must be sanitized. This is painful to do in lpass-att-export.sh, so this PR introduces the new '_' format string modifier. This modifier works similarly to the existing '/' modifier, but causes instances of forward slashes, carriage returns, new lines, and tabs with an underscore in output.
lpass-att-export.sh has been modified to use this new format string when a new-enough version of lpass is present.
Truncated Attachment Names
lpass-att-export.sh used awk to read space separated tokens from the attachment description printed by lpass. This works fine for the attachment id, but incorrectly truncates file names containing spaces. Using the bash built-in
readnot only solves this but also prevents needing 2 external program calls and a temp variable.POSIX compliant sed
The version of sed that currently ships with MacOS is old and only supports POSIX regular expressions. Thus, it supports the
[:space:]character class but not the shorthand/sescape-like character class. So we switch to the long version and everyone is happy.Export Speed
lpass-att-export.sh used
lpass lsto iterate through every item in the vault, callinglpass showfor each in order to find any that contain attachments. This will prompt for your master password for every item with a reprompt set, whether they have an attachment or not. It also is a lot of calls to PBKDF2 and if your interation count is high enough, the computation time can add up.Instead we now use
lpass exportto find the ids of items that contain attachments via the id and attachpresent fields. This doesn't reprompt for every item. Much faster and less annoying!Shellcheck Warnings
Most of the changes here were just to add quotes to ensure that variables don't get parsed wrong, and in most cases this doesn't actually change anything. But the script currently sometimes exits with a failure exit code even when it works fine due to the
let attcount-=1at the very end. Switching to(( attcount-=1 )) || trueprevents this. Not that it's a really big deal, but might as well fix it.