You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Yubikey-SSH: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Muehlenhoff
(Initial documentation for Yubikey NEO/PIV/SSH)
 
imported>RLazarus
 
(21 intermediate revisions by 17 users not shown)
Line 1: Line 1:
Configuring the Yubikey NEO for SSH authentication in the prod cluster:
This procedure describes how to store your production SSH credentials on a Yubikey physical 2FA device. Doing so provides an additional layer of protection against having your key compromised via your laptop. It's recommended, but not required, for anyone with production SSH access. These instructions are written for a vanilla Ubuntu or Debian installation, and will probably require adjustment for Mac OS or other Linux environments.


= Installation =
When you're done, your production SSH private key will be stored ''only'' on your Yubikey. That means if it's lost or damaged, you won't be able to SSH to production without asking a colleague to help you authorize another key. If your Yubikey is lost or stolen, it will still be protected by your PIN, but you should immediately contact SREs to have the key revoked.


On Debian / Ubuntu:
You can also use the same Yubikey as your 2FA device for Gmail and some other services; that isn't covered in these instructions.


apt-get install yubikey-personalization  yubico-piv-tool opensc
== Prerequisites ==


On MacOS:
* You already have a [[PGP key]]. It can be the same key you'll use for [https://office.wikimedia.org/wiki/Pwstore Pwstore], but it doesn't need to be signed by anyone yet.
* You ''either'' already have SSH access to production, ''or'' don't mind waiting until your Yubikey arrives in the mail (in which case your onboarding buddy will help you with the parts of this procedure that need SSH access).


brew install opensc ykpers yubico-piv-tool
== Instructions ==


= Enabling CCID smartcard mode =
# Email techsupport@wikimedia.org and ask them to send you a Yubikey.
# Done? Great, come back in a couple of days when you get a package in the mail.
# Oh hey, welcome back! Your new Yubikey should be in its original packaging from Yubico. (If it's unwrapped or appears tampered with, stop. If an attacker has compromised your Yubikey, they can gain access to prod and harm our users.) Unwrap it and plug it into your laptop, then visit [https://yubico.com/genuine https://yubico.com/genuine] and follow instructions, to make sure you've got the real thing and it's working correctly. It should say "verification complete" at the end.
# Install the SmartCard daemon: <code>sudo apt install scdaemon</code>. For MacOS, you can follow https://docs.releng.linuxfoundation.org/en/latest/gpg.html#setup-gpg-agent.
# Once the installation is done, unplug your Yubikey and plug it back in. Leave it plugged in for the rest of the instructions.
# The default user PIN and administrator PIN are 123456 and 12345678, respectively. You'll be asked for the user PIN the first time you use your SSH key after the Yubikey loses power -- that is, after each time you plug it in or reboot. If you enter the wrong user PIN three times in a row, you'll need the admin PIN to unlock the Yubikey, including to change the user PIN. Change both PINs away from the defaults now:
## Run <code>gpg --change-pin</code>, which will drop you into an interactive prompt.
## Choose <code>1 - change PIN</code>.
## When prompted, enter first the old user PIN (123456) and then your new user PIN twice. Despite the misleading name, your "PIN" doesn't need to be a number; you should use a strong password.
## Choose <code>3 - change Admin PIN</code>.
## When prompted, enter first the old admin PIN (12345678) and then your new admin PIN twice. Use a different strong password.
## Unlike the user PIN, you'll only rarely need to use the admin PIN, so consider storing it in a password manager or writing it down in a safe place along with your recovery codes for other services. (If you lose both PINs, you'll be able to reset the Yubikey, but your SSH key will be permanently lost.)
## Enter <code>q</code> to quit.
# Add an authentication subkey to your PGP key:
## Run <code>gpg --expert --edit-key &lt;your PGP key ID&gt;</code>, which will drop you into another interactive prompt.
## Type the command <code>addkey</code>.
## When asked what kind of key you want, choose <code>(8) RSA (set your own capabilities)</code>.
## The default capabilities are Sign and Encrypt; you only want the capability Authenticate, so toggle all three options by entering S, then E, then A.
## Next you'll be asked what keysize you want. For a recent-model Yubikey (4 or 5, as shown on the packaging) enter 4096 bits.
## Expiration: You can enter 0 for no expiration.
## Next, still at the GPG prompt, select the new subkey with the command <code>key 2</code>.
## Move the subkey onto your Yubikey with the command <code>keytocard</code>.
## When asked where you want to store the key, choose <code>(3) Authentication key</code>
## When you return to the <code>gpg&gt;</code> prompt, type the command <code>quit</code> to quit, and <code>y</code> to save changes.
# Next, authorize your new SSH key:
## To see the public key, run <code>SSH_AUTH_SOCK=/run/user/$(id -u)/gnupg/S.gpg-agent.ssh ssh-add -L</code>. For MacOS, it should be something like <code>SSH_AUTH_SOCK=/Users/${USER}/.gnupg/S.gpg-agent.ssh ssh-add -L</code>. The output should be a single long line beginning with <code>ssh-rsa</code>; set it aside.
## In the [https://gerrit.wikimedia.org/r/admin/projects/operations/puppet operations/puppet] repository, open [https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/production/modules/admin/data/data.yaml modules/admin/data/data.yaml] in your editor.
## Find the stanza that begins with your shell username and contains an <code>ssh_keys:</code> list. If it doesn't exist yet, stop here and ask your onboarding buddy for help.
## Add your new public key, retrieved above, as a second list entry below the old one -- that is, prefixed with <code>- </code> at the same level of indentation. (Don't delete the old one yet; you'll still need it for the next steps.)
## Commit your change, have it reviewed by your onboarding buddy, then +2 and submit.
## SSH to <code>puppetmaster1001.eqiad.wmnet</code> (at this point, you're still using your old credentials) and run <code>sudo puppet-merge</code>. Verify that your change is the only diff, then confirm the merge. You may want to leave this window open, in case you get stuck and can't log back in.
## Now SSH to your [[Production access|bastion]] of choice and run <code>sudo run-puppet-agent</code>. You may want to leave this window open too, in case you get stuck and can't log back in.
# Configure SSH to use the new key instead of the old one:
## On your laptop, open <code>~/.ssh/config</code> in your editor. If it doesn't exist yet, stop here and ask your onboarding buddy for help, or refer to [[Production_access#Setting_up_your_SSH_config]] and modify it as follows.
## In the <code>Host</code> stanzas for the bastion and for the rest of production, comment out the <code>IdentityFile</code> statement for your existing private key by prepending a <code>#</code>. (We won't change any other hosts, like Gerrit -- remember, your SSH credentials for production shouldn't be reused for anything else.)
## In its place, add <code>IdentityAgent /run/user/%i/gnupg/S.gpg-agent.ssh</code> to both stanzas (or for MacOS, the path aforementioned).
## If you have <code>IdentitiesOnly yes</code>, comment it out in both stanzas. (If you have <code>IdentitiesOnly yes</code> set overall, you’ll have to remove it – it’s not enough to add <code>IdentitiesOnly no</code> to the <code>Host</code> stanzas for production.)
## Save and close the file.
# SSH to the bastion again. If you succeed, your new Yubikey-only credentials are working! If you can't get in, or if you get a <code>Password:</code> prompt, trying again with <code>ssh -v</code> may help to debug.
# Once you're happily SSHing around with your new key, deauthorize your old one, so that your Yubikey is required to access production. (If both keys work, the Yubikey provides no protection, just extra attack surface.)
## Open the puppet repository and edit <code>modules/admin/data/data.yaml</code> again.
## Find your stanza and delete the entry in <code>ssh_keys:</code> for your ''old'' key.
## Commit your change, have it reviewed by your onboarding buddy, then +2 and submit.
## SSH to <code>puppetmaster1001.eqiad.wmnet</code> and run <code>sudo puppet-merge</code>.
## SSH to the bastion and run <code>sudo run-puppet-agent</code>.
## SSH to the same bastion one more time, to verify that the new config is correct.
# Optional, but recommended: For added security, configure your Yubikey to require you to physically touch it each time you use it to authenticate. With this setting, you can leave the Yubikey plugged into your computer without worrying about a remote attacker using it to access prod; using your Yubikey will require physical presence at your machine (or knowledge of your admin PIN to turn the setting back off.)
## Install Yubikey Manager.
##* For Debian/Ubuntu: <code>sudo apt install yubikey-manager</code>
## Run <code>ykman --version</code>.
## Set the touch policy; the correct command depends on your Yubikey Manager version.
##* For ykman version 3.x (Ubuntu 19.10+, Debian bullseye+): Run <code>ykman openpgp set-touch aut cached</code>. You'll need to touch your Yubikey ''once'' each time you SSH. Even though you need to authenticate once to the bastion and once to the destination host, your Yubikey will cache the touch for fifteen seconds.
##* For ykman version 2.x: Run <code>ykman openpgp touch aut on</code>. You'll need to touch your Yubikey ''twice'' each time you SSH, once to authenticate to the bastion and once to authenticate to the destination host.
##* If there is an error <q>Failed connecting to YubiKey 5 [OTP+FIDO+CCID]. Make sure the application have                    the required permissions.</q> <!-- sic, with “have” and spaces --> then try unplugging the YubiKey and plugging it back in.
## Enter the admin PIN you selected earlier.
## Type <code>y</code> to confirm.
## Try SSHing to <code>puppetmaster1001.eqiad.wmnet</code> again to test the new setting. When the LED on the Yubikey blinks, touch it. From now on, you'll need to do this each time you SSH. If the Yubikey ever blinks unexpectedly when you aren't trying to use it, you should become suspicious.
# Optional cleanup: You can now delete the commented-out lines from your <code>.ssh/config</code>. You can also delete the old key from your <code>.ssh/</code> directory if you like.


The smartcard/CCID feature on the Yubikey NEO is disabled by default. You can change that with the following command:
== Resetting your Yubikey ==


ykpersonalize -m86
You can reset the OpenPGP app on the Yubikey by running <code>ykman openpgp reset</code>, and typing <code>y</code> to confirm. This will delete all PGP keys stored on your Yubikey, and reset your PIN and Admin PIN to the defaults above.


(See https://www.yubico.com/2012/12/yubikey-neo-composite-device/ for some more information on the parameter)
== More information ==


= Securing physical access to the Yubikey =
* [[Production access]]
* [[PGP Keys]]
* [https://developers.yubico.com/PGP/SSH_authentication/ SSH authentication (Yubico Developers)]
* [https://support.yubico.com/support/solutions/articles/15000006420-using-your-yubikey-with-openpgp Using Your YubiKey with OpenPGP (Yubico Support)]
* [https://developers.yubico.com/PGP/Card_edit.html Card edit (Yubico Developers)]
* [https://support.yubico.com/support/solutions/articles/15000006421-resetting-the-openpgp-applet-on-the-yubikey Resetting the OpenPGP Applet on the YubiKey (Yubikey Support)]


When accessing the NEO you need to enter a PIN to prevent access for someone who e.g. stole your Yubikey. If that PIN is entered incorrectly thrice, the Yubikey needs to be unlocked with a PUK.
[[Category:How-To]]
If that PUK is also entered incorrectly three time your Yubikey is toast, so make sure to store these in your password manager.
 
In addition the NEO uses a management key: It secures access when updating the feature applets running on the Yubikey and protects you from someone backdooring your NEO.
 
The Yubikey uses default values for PIN, PUK and management key, so we need to change all of them:
 
* First the management key (24 chars hexadecimal):
 
key=`dd if=/dev/random bs=1 count=24 2>/dev/null | hexdump -v -e '/1 "%02X"'`
yubico-piv-tool -a set-mgm-key -n  $key
 
* Now the PIN (6 digits, 123456 is the shipped default PIN):
 
pin=`dd if=/dev/random bs=1 count=6 2>/dev/null | hexdump -v -e '/1 "%u"'|cut -c1-6`
yubico-piv-tool -a change-pin -P 123456 -N $pin
 
* Finally the PUK (8 digits, with the default 12345678)
 
puk=`dd if=/dev/random bs=1 count=6 2>/dev/null | hexdump -v -e '/1 "%u"'|cut -c1-8`
yubico-piv-tool -a change-puk -P 12345678 -N $puk
 
= Key/certificate creation =
 
First of all generate a new prvate key: (9a is a slot number, for others see the introduction of https://www.yubico.com/wp-content/uploads/2015/04/Yubico-PIV-Management-Tools_v1.0.pdf )
 
  yubico-piv-tool -s 9a -a generate -o public.pem
 
Generate a self-signed certificate:
 
  yubico-piv-tool -a verify-pin -P YOUR_PIN -a selfsign-certificate -s 9a -S "/CN=SSH key/" -i public.pem -o cert.pem
 
Import the certificate:
 
  yubico-piv-tool -a import-certificate -s 9a -i cert.pem
 
= Generate an SSH pubkey =
 
On Debian/Ubuntu and amd64:
export OPENSC="/usr/lib/x86_64-linux-gnu/pkcs11/opensc-pkcs11.so"
ssh-keygen -D $OPENSC -e
 
On MacOS X / Brew:
export OPENSC="/usr/local/Cellar/opensc/0.15.0/lib/opensc-pkcs11.so"
ssh-keygen -D $OPENSC -e
 
= Accessing the key =
 
The key is accessed via the PKCS11 interface (which OpenSSH supports):
 
ssh -I $OPENSC $HOST
Enter PIN for 'PIV_II (PIV Card Holder pin)':
 
Of course you don't need to enter the PIN all the time, ssh-agent supports PKCS11 as well. Simply run
 
ssh-add -s $OPENSC
 
to enter the key to the agent.

Latest revision as of 17:44, 1 September 2021

This procedure describes how to store your production SSH credentials on a Yubikey physical 2FA device. Doing so provides an additional layer of protection against having your key compromised via your laptop. It's recommended, but not required, for anyone with production SSH access. These instructions are written for a vanilla Ubuntu or Debian installation, and will probably require adjustment for Mac OS or other Linux environments.

When you're done, your production SSH private key will be stored only on your Yubikey. That means if it's lost or damaged, you won't be able to SSH to production without asking a colleague to help you authorize another key. If your Yubikey is lost or stolen, it will still be protected by your PIN, but you should immediately contact SREs to have the key revoked.

You can also use the same Yubikey as your 2FA device for Gmail and some other services; that isn't covered in these instructions.

Prerequisites

  • You already have a PGP key. It can be the same key you'll use for Pwstore, but it doesn't need to be signed by anyone yet.
  • You either already have SSH access to production, or don't mind waiting until your Yubikey arrives in the mail (in which case your onboarding buddy will help you with the parts of this procedure that need SSH access).

Instructions

  1. Email techsupport@wikimedia.org and ask them to send you a Yubikey.
  2. Done? Great, come back in a couple of days when you get a package in the mail.
  3. Oh hey, welcome back! Your new Yubikey should be in its original packaging from Yubico. (If it's unwrapped or appears tampered with, stop. If an attacker has compromised your Yubikey, they can gain access to prod and harm our users.) Unwrap it and plug it into your laptop, then visit https://yubico.com/genuine and follow instructions, to make sure you've got the real thing and it's working correctly. It should say "verification complete" at the end.
  4. Install the SmartCard daemon: sudo apt install scdaemon. For MacOS, you can follow https://docs.releng.linuxfoundation.org/en/latest/gpg.html#setup-gpg-agent.
  5. Once the installation is done, unplug your Yubikey and plug it back in. Leave it plugged in for the rest of the instructions.
  6. The default user PIN and administrator PIN are 123456 and 12345678, respectively. You'll be asked for the user PIN the first time you use your SSH key after the Yubikey loses power -- that is, after each time you plug it in or reboot. If you enter the wrong user PIN three times in a row, you'll need the admin PIN to unlock the Yubikey, including to change the user PIN. Change both PINs away from the defaults now:
    1. Run gpg --change-pin, which will drop you into an interactive prompt.
    2. Choose 1 - change PIN.
    3. When prompted, enter first the old user PIN (123456) and then your new user PIN twice. Despite the misleading name, your "PIN" doesn't need to be a number; you should use a strong password.
    4. Choose 3 - change Admin PIN.
    5. When prompted, enter first the old admin PIN (12345678) and then your new admin PIN twice. Use a different strong password.
    6. Unlike the user PIN, you'll only rarely need to use the admin PIN, so consider storing it in a password manager or writing it down in a safe place along with your recovery codes for other services. (If you lose both PINs, you'll be able to reset the Yubikey, but your SSH key will be permanently lost.)
    7. Enter q to quit.
  7. Add an authentication subkey to your PGP key:
    1. Run gpg --expert --edit-key <your PGP key ID>, which will drop you into another interactive prompt.
    2. Type the command addkey.
    3. When asked what kind of key you want, choose (8) RSA (set your own capabilities).
    4. The default capabilities are Sign and Encrypt; you only want the capability Authenticate, so toggle all three options by entering S, then E, then A.
    5. Next you'll be asked what keysize you want. For a recent-model Yubikey (4 or 5, as shown on the packaging) enter 4096 bits.
    6. Expiration: You can enter 0 for no expiration.
    7. Next, still at the GPG prompt, select the new subkey with the command key 2.
    8. Move the subkey onto your Yubikey with the command keytocard.
    9. When asked where you want to store the key, choose (3) Authentication key
    10. When you return to the gpg> prompt, type the command quit to quit, and y to save changes.
  8. Next, authorize your new SSH key:
    1. To see the public key, run SSH_AUTH_SOCK=/run/user/$(id -u)/gnupg/S.gpg-agent.ssh ssh-add -L. For MacOS, it should be something like SSH_AUTH_SOCK=/Users/${USER}/.gnupg/S.gpg-agent.ssh ssh-add -L. The output should be a single long line beginning with ssh-rsa; set it aside.
    2. In the operations/puppet repository, open modules/admin/data/data.yaml in your editor.
    3. Find the stanza that begins with your shell username and contains an ssh_keys: list. If it doesn't exist yet, stop here and ask your onboarding buddy for help.
    4. Add your new public key, retrieved above, as a second list entry below the old one -- that is, prefixed with - at the same level of indentation. (Don't delete the old one yet; you'll still need it for the next steps.)
    5. Commit your change, have it reviewed by your onboarding buddy, then +2 and submit.
    6. SSH to puppetmaster1001.eqiad.wmnet (at this point, you're still using your old credentials) and run sudo puppet-merge. Verify that your change is the only diff, then confirm the merge. You may want to leave this window open, in case you get stuck and can't log back in.
    7. Now SSH to your bastion of choice and run sudo run-puppet-agent. You may want to leave this window open too, in case you get stuck and can't log back in.
  9. Configure SSH to use the new key instead of the old one:
    1. On your laptop, open ~/.ssh/config in your editor. If it doesn't exist yet, stop here and ask your onboarding buddy for help, or refer to Production_access#Setting_up_your_SSH_config and modify it as follows.
    2. In the Host stanzas for the bastion and for the rest of production, comment out the IdentityFile statement for your existing private key by prepending a #. (We won't change any other hosts, like Gerrit -- remember, your SSH credentials for production shouldn't be reused for anything else.)
    3. In its place, add IdentityAgent /run/user/%i/gnupg/S.gpg-agent.ssh to both stanzas (or for MacOS, the path aforementioned).
    4. If you have IdentitiesOnly yes, comment it out in both stanzas. (If you have IdentitiesOnly yes set overall, you’ll have to remove it – it’s not enough to add IdentitiesOnly no to the Host stanzas for production.)
    5. Save and close the file.
  10. SSH to the bastion again. If you succeed, your new Yubikey-only credentials are working! If you can't get in, or if you get a Password: prompt, trying again with ssh -v may help to debug.
  11. Once you're happily SSHing around with your new key, deauthorize your old one, so that your Yubikey is required to access production. (If both keys work, the Yubikey provides no protection, just extra attack surface.)
    1. Open the puppet repository and edit modules/admin/data/data.yaml again.
    2. Find your stanza and delete the entry in ssh_keys: for your old key.
    3. Commit your change, have it reviewed by your onboarding buddy, then +2 and submit.
    4. SSH to puppetmaster1001.eqiad.wmnet and run sudo puppet-merge.
    5. SSH to the bastion and run sudo run-puppet-agent.
    6. SSH to the same bastion one more time, to verify that the new config is correct.
  12. Optional, but recommended: For added security, configure your Yubikey to require you to physically touch it each time you use it to authenticate. With this setting, you can leave the Yubikey plugged into your computer without worrying about a remote attacker using it to access prod; using your Yubikey will require physical presence at your machine (or knowledge of your admin PIN to turn the setting back off.)
    1. Install Yubikey Manager.
      • For Debian/Ubuntu: sudo apt install yubikey-manager
    2. Run ykman --version.
    3. Set the touch policy; the correct command depends on your Yubikey Manager version.
      • For ykman version 3.x (Ubuntu 19.10+, Debian bullseye+): Run ykman openpgp set-touch aut cached. You'll need to touch your Yubikey once each time you SSH. Even though you need to authenticate once to the bastion and once to the destination host, your Yubikey will cache the touch for fifteen seconds.
      • For ykman version 2.x: Run ykman openpgp touch aut on. You'll need to touch your Yubikey twice each time you SSH, once to authenticate to the bastion and once to authenticate to the destination host.
      • If there is an error Failed connecting to YubiKey 5 [OTP+FIDO+CCID]. Make sure the application have the required permissions. then try unplugging the YubiKey and plugging it back in.
    4. Enter the admin PIN you selected earlier.
    5. Type y to confirm.
    6. Try SSHing to puppetmaster1001.eqiad.wmnet again to test the new setting. When the LED on the Yubikey blinks, touch it. From now on, you'll need to do this each time you SSH. If the Yubikey ever blinks unexpectedly when you aren't trying to use it, you should become suspicious.
  13. Optional cleanup: You can now delete the commented-out lines from your .ssh/config. You can also delete the old key from your .ssh/ directory if you like.

Resetting your Yubikey

You can reset the OpenPGP app on the Yubikey by running ykman openpgp reset, and typing y to confirm. This will delete all PGP keys stored on your Yubikey, and reset your PIN and Admin PIN to the defaults above.

More information