feat: handle email encryption via x25519 asymmetric keys

[Carbonhell]

Closes #2245.

Note: The PR requires two changes by an infra-team admin before being ready to be merged (cc @marcoieni):

1. a new private/public key pair must be generated via the new CLI command generate-keys;
2. the public key inside email_encryption,rs must be replaced with the newly generated one;
3. the private key must be configured as a GitHub secret named EMAIL_PRIVATE_KEY, which is passed as an environment variable in the same way as the now deprecated EMAIL_ENCRYPTION_KEY secret/env var;
4. the currently encrypted email addresses must be migrated by decrypting them (via the currently existing decrypt-email command on the mai branch), encrypting them via this branch's encrypt-email utility, and adding them as changes to this PR;

There's a section at the end of the README which refers to a SSM parameter - for simplicity, I haven't changed its syntax (even though it'd be more correct to rename it to "email-private-key" instead of encryption key).

The encryption scheme uses x25519-dalek to generate a shared secret via Diffie-Hellman, with blake3 (recommended in RustCrypto) used as a key derivation function to get the key to use with XChaCha20Poly1305 (which was the library used previously for symmetric encryption/decryption).

[ubiratansoares]

Hi @Carbonhell, thanks for rising this PR! Great to have this contribution 🚀

I'll leave two considerations regarding your proposal

Encryption scheme

I'm by no means a PKI expert, but when I first evaluated what I'd propose myself to implement this feature, I definetly recollected rage, the Rust implementation for age encryption. I use age (the Go CLI) to encrypt some of my personal stuff 😄

rage builds on top of the some dependencies you mentioned, but the implementation for small recipients using the very same x25519 looks super straightforward as per available examples, and it also seems the right fit for our use case.

My initial take is that having less code attached to this feature make things simpler to grasp and maintain* for the long run. That said : would you mind giving a try on rage and bring some feedback to us, without commiting to it all?

I think the implementation will become shorter in terms of code, but we could definetly learn from a comparison on binary sizes, private/public key sizes and total dependencies we'd bring to our dep graph.

After that, perhaps we can make a decision regarding which crypto approach we want to use, perhaps gathering input from other infra folks too 🙂

Migrating existing encrypted emails and setting credentials

As soon as we have an agreement on the PKI approach - either yours, rage or something else - I'd be happy to generate the key, add it to CI, decrypt existing emails and leave a comment to additional changes you can add in a single additional commit to finish this PR 🙂

Just wanted to let you know that this particular feature is not a priority for infra right now, hence it may take some time between reviews, feedback and action from our side.

I'll leave some additional comments in the code itself, just to highlight a few things that came to mind

[ubiratansoares]

Considering the change we are proposing here, I think we could store the private key somewhere else. Ofc we'll have it as CI secret, but eventually 1password could be an option for infra admins as well, just in case we need to decrypt existing emails and get in touch with people for whatever reason.

Hence, I do think we could simplify a bit this approach. Not sure what other infra admins think about this 👀

[Carbonhell]

I'm not familiar with 1password and how you use it, but I think a relevant point is that I can't seem to find the Terraform definition of this parameter on simpleinfra - I assume it was created directly via the AWS dashboard. This might be an extra point in favor of using 1password to remove this parameter that isn't currently managed under IaC

[Carbonhell]

Hi @Carbonhell, thanks for rising this PR! Great to have this contribution 🚀

I'll leave two considerations regarding your proposal

Encryption scheme

I'm by no means a PKI expert, but when I first evaluated what I'd propose myself to implement this feature, I definetly recollected rage, the Rust implementation for age encryption. I use age (the Go CLI) to encrypt some of my personal stuff 😄

rage builds on top of the some dependencies you mentioned, but the implementation for small recipients using the very same x25519 looks super straightforward as per available examples, and it also seems the right fit for our use case.

My initial take is that having less code attached to this feature make things simpler to grasp and maitain for the long run. That said : would you mind giving a try on rage and bring some feedback to us, without commiting to it all?

I think the implementation will become shorter in terms of code, but we could definetly learn from a comparison on binary sizes, private/public key sizes and total dependencies we'd bring to our dep graph.

After that, perhaps we can make a decision regarding which crypto approach we want to use, perhaps gathering input from other infra folks too 🙂

Migrating existing encrypted emails and setting credentials

As soon as we have an agreement on the PKI approach - either yours, rage or something else - I'd be happy to generate the key, add it to CI, decrypt existing emails and leave a comment to additional changes you can add in a single additional commit to finish this PR 🙂

Just wanted to let you know that this particular feature is not a priority for infra right now, hence it may take some time between reviews, feedback and action from our side.

I'll leave some additional comments in the code itself, just to highlight a few things that came to mind

Thank you for the feedback! To be fair, I wasn't aware of rage. The rationale for using x25519-dalek and blake3 was due to them being on https://blessed.rs/crates#section-cryptography, but I agree that the steps required to actually encrypt contents aren't ideal / a bit too low level for our needs here, which are harder to maintain and with footguns (e.g., requiring a KDF instead of directly using the shared secret as key). I'll definitely give rage a try and report back about the tradeoffs between each solution! I should be able to before the end of this week :)

Edit: technically, x25519-dalek isn't on blessed.rs, but the ed25519-dalek is, and they're both based on the same core from what I gathered

[github-actions]

Dry-run check results

[WARN  sync_team] sync-team is running in dry mode, no changes will be applied.
[INFO  sync_team] synchronizing crates-io
[INFO  sync_team] synchronizing github
[INFO  sync_team] 💻 Repo Diffs:
    📝 Editing repo 'rust-lang/bors':
      Rulesets:
          Ruleset for main
            No changes

[Carbonhell]

I tried out rage, here are the results:

Pros

• the implementation, as expected, is considerably simpler: you can see how it'd look like in this branch. The crate is quite ergonomic, allowing parsing a &str as Identity (private key) or Recipient (public key) to use it with decrypt(identity: &impl Identity, ciphertext: &[u8]) and age::encrypt(recipient: &impl Recipient, plaintext: &[u8]), without having to deal with nonces and shared secrets (which is a bit too close to rolling your own crypto to be honest);
• with the ssh feature, the key generation command could be removed in favor of using the ssh feature of age keys generated via ssh-keygen

Cons

• the size of the resulting target directory is considerably larger (size calculated after cargo build -p rust_team_data --release --features email-encryption with du -sh target/release/):
  • 37 dependencies with size 55MB as baseline
  • 54 dependencies with size 81MB with the implementation of this PR
  • 186 dependencies with size 209MB with rage
• the resulting encrypted email is considerably bigger:
  • rage: encrypted+6167652d656e6372797074696f6e2e6f72672f76310a2d3e2058323535313920684832542b487a5a4e5a704552612f4634656632776272313158543477576c667a653061524e57504158300a6c5946664168774669566779327532726e50796468464d68787665703434494677786955423434497749410a2d3e202e5b2d677265617365207e30763621204661397c4f4e730a3333746337476c414233476163374e414a356f31416d4c644d763347647a346e51582f4d7436332b2b59314d64584948325965466f39596f796b59735a66692b0a7a734373476874786974447a487741516a4b456f73477954624a725278485531506b67726c6f51487947490a2d2d2d20566e65754358563777767950316d7546336a41654e7a56726c646970477a576f42673452436863547a38730af607ec84912c7544ba326c066f91ecb3538dfe2f48cadfdba0d2197522233e071b34c9b401a9d72648a9ae52badd5581ff@rust-lang.invalid
  • this PR: encrypted+4d85908caaa32211b5c4ec28bfe3ead598f53c539aa1d5b2666320897cc206315098adc459e0ebd89b7bdd35d4231f609d715c8811e597f0c834ddb70c61577bdf55642f02a1f492a8e3232a715cb36254d4fb64e36d9a1c@rust-lang.invalid

---

Considering the analysis above, I don't think the current implementation is complex enough to warrant bringing in all the dependencies of rage, but I recognize the higher maintenance effort with the current implementation. I'll leave the decision to you and adapt this PR as necessary :)

I also cleaned the signature of init_cipher and replaced the get_kdf function with blake3's own KDF, which slightly simplifies the implementation and avoids reusing the nonce for two different operations.

[Kobzol]

Just wanted to note that a bunch of other infra crates and tools directly depend on rust_team_data, so it would be great if it didn't grow so much. Although we don't enable the email feature on most places, I think, so if it's not the default, it shouldn't be such a big deal.

That being said, it seems weird why e-mail encryption should even play a role in ruat_team_data, that's just a bunch of data structures, right? Wouldn't it be possible to represent the encrypted e-mail as a bunch of bytes or something, and move the feature into team/sync-team?

[Carbonhell]

Just wanted to note that a bunch of other infra crates and tools directly depend on rust_team_data, so it would be great if it didn't grow so much. Although we don't enable the email feature on most places, I think, so if it's not the default, it shouldn't be such a big deal.

That being said, it seems weird why e-mail encryption should even play a role in ruat_team_data, that's just a bunch of data structures, right? Wouldn't it be possible to represent the encrypted e-mail as a bunch of bytes or something, and move the feature into team/sync-team?

Based on a quick search, the email-encryption feature isn't used anywhere else other than in this repository. I think the rationale behind putting it in rust_team_data was to allow sharing the encryption/decryption functions both in the CLI app (under src) and inside sync-team, to decrypt the emails in the mailgun module. But I don't see any blocker in moving them inside sync-team and using it from the CLI (which already depends on sync-team). It does seem clearer to me as right now I believe they're only meant to be used in this repository.

If you want, I can move the module as needed, removing the email-encryption feature which wouldn't be needed anymore, either in this PR or in a new one. Let me know if you'd appreciate that :)

Edit: I was thinking that it might be worth to document, or enforce via a specific type, that the email field under the People struct (here (and I think the TeamWebsite struct too?) may contain an encrypted email that can not be used by downstream consumers of the rust_team_data crate. Just to avoid other people writing automations or displaying those emails, ignoring the existence of encrypted ones. Not sure if you agree.