You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

PKI/Clients: Difference between revisions

From Wikitech-static
< PKI
Jump to navigation Jump to search
imported>Jcrespo
(→‎profile::pki::get_cert: +warning about writability of the output dir)
imported>Jbond
 
(5 intermediate revisions by 4 users not shown)
Line 1: Line 1:
We currently use CFSSL to provide and manage PKI solutions. clients are able to make use of the CFSSL API end point available at https://pki.discovery.wmnet/however this end point is protected via TLS Client authentications and currently requires users to using the puppet agent certificate. The puppet agent certificate was chosen as it is something we already issue when a host is provisioned or re-imaged, however we can change this at a later date or configure additional intermediate CAs which have different authentication mechanisms. Further to the client auth requirement API request also need to be signed with a hmac using a secret key (available in the puppet private repo)
We currently use [https://cfssl.org/ CFSSL] to provide and manage PKI solutions. Clients are able to make use of the CFSSL API endpoint available at https://pki.discovery.wmnet/, however this endpoint is protected via TLS Client authentications and currently requires users to use the puppet agent certificate. The puppet agent certificate was chosen as it is something we already issue when a host is provisioned or reimaged; we can change this at a later date or configure additional intermediate CAs which have different authentication mechanisms. In addition to the client auth requirement, API requests also need to be signed with hmac using a secret key (available in the puppet private repo).


= Puppet integration =
= Puppet integration =
Line 5: Line 5:


== Creating certificates ==
== Creating certificates ==
When a user wants to create a new cerrtificate the will first need to decide which intermediate CA they want to use for issuing the certificates.  Note that each realm of operations should have its own intermediate CA and you should not use cross functional CA's e.g. debmonitor, DB connections, conftool should all have there own intermediated CA's.  If you need to create a new intermediate CA Please check the documentation on the [[PKI/CA Operations#Adding_a_new_intermediate| CA Operations]] Page. Also unless the defaults have been changed all certificates will be issued with an expiry of 96h and set to auto renew 24hours before the certificate expires.
When a user wants to create a new certificate, they will first need to decide which intermediate CA they want to use.  Note that each realm of operations using client auth should have its own intermediate CA and you should not use cross functional CA's. For example: debmonitor, DB connections, and conftool should all have their own intermediate CA's.  If you need to create a new intermediate CA, see the[[PKI/CA Operations#Adding_a_new_intermediate| CA Operations]] Page. Unless the defaults have been changed, all certificates will be issued with an expiry of 672h and set to autorenew 10days hours before they expire.


Once you know which intermediate CA you want to use for issuing your certificate you will need to know which profile to use.  By default intermediate certificates are create with [[PKI/Policy#Default_signing_policies|Two profiles]] which are useful for this consideration.  The default profile which should be used for client auth and a server profile which can be used for daemon services.  If you have created additional profiles for you CA then you will need to use them below. For the below example i am going to assume we want to create two certificates in the "WMF testing CA", on in using the default profile and one using the server profile.   
Once you know which intermediate CA you want to use to issue your certificate, you will need to know which profile to use.  By default, intermediate certificates are created with [[PKI/Policy#Default_signing_policies|two profiles]].  The default profile is for client auth and the server profile is for daemon services.  If you have created additional profiles for your CA, you can use them as below. The below example assumes we want to create two certificates in the "WMF testing CA", one using the default profile and one using the server profile.   


=== profile::pki::get_cert ===
=== profile::pki::get_cert ===


The simplest interface to use to create certificates is the <code>profile::pki::get_cert</code> functions. This function has a similar signature to the <code>cfssl::cert</code> resource, however it also returns the path of the cert, private key and CA bundle (if requested). As you most often need to have theses paths for other parts of puppet configurations this is likely the most useful interface.
The simplest interface to for creating certificates is the <code>profile::pki::get_cert</code> function. This function has a similar signature to <code>cfssl::cert</code>, but it also returns the path of the cert, the private key, and the CA bundle (chained). As you most often need to have these paths for other parts of puppet configurations, this is likely the most useful interface.


<syntaxhighlight lang="puppet">
<syntaxhighlight lang="puppet">
Line 27: Line 27:
</syntaxhighlight>  
</syntaxhighlight>  


Also fetch the CA bundle
To also fetch the CA bundle:
<syntaxhighlight lang="puppet">
<syntaxhighlight lang="puppet">
profile::pki::get_cert('WMF_testing_CA', $facts['fqdn']) >> {
profile::pki::get_cert('WMF_testing_CA', $facts['fqdn']) >> {
   'cert' => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_$fqdn.pem',     # owned by root
   'cert'   => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_$fqdn.pem',     # owned by root
   'key' => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_$fqdn-key.pem',  # owned by root
   'key'     => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_$fqdn-key.pem',  # owned by root
   'ca' => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_chain.pem',       # owned by root
   'chain'   => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_chain.pem',     # owned by root
  'chained' => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_chained.pem',    # owned by root
}
}
</syntaxhighlight>  
</syntaxhighlight>  


Also fetch the CA bundle, change the outputdir and user
To fetch the CA bundle, changing the outputdir and user:
<syntaxhighlight lang="puppet">
<syntaxhighlight lang="puppet">
profile::pki::get_cert(
profile::pki::get_cert(
Line 43: Line 44:
   {'outdir' => '/etc/foobar', 'owner' => 'foobar'}
   {'outdir' => '/etc/foobar', 'owner' => 'foobar'}
) >> {
) >> {
   'cert' => '/etc/foobar/WMF_testing_CA_$fqdn.pem',   # owned by foobar
   'cert'   => '/etc/foobar/WMF_testing_CA_$fqdn.pem',     # owned by foobar
   'key' => '/etc/foobar/WMF_testing_CA_$fqdn-key.pem', # owned by foobar
   'key'     => '/etc/foobar/WMF_testing_CA_$fqdn-key.pem', # owned by foobar
   'ca' => '/etc/foobar/WMF_testing_CA_chain.pem',     # owned by foobar
   'chain'   => '/etc/foobar/WMF_testing_CA_chain.pem',     # owned by foobar
  'chained' => '/etc/foobar/WMF_testing_CA_chained.pem',  # owned by foobar
}
}
</syntaxhighlight>
</syntaxhighlight>


{{note|<code>outdir</code> directory, if used, should be declared by yourself if it doesn't already exist, and the owner should have write permissions (e.g. <code>0700</code>), or the chained certificate generation will fail on puppet run.}}
{{note|If you use the <code>outdir</code> directory, make sure it is declared in puppet, and its owner should have write permissions (e.g. <code>0700</code>). If the directory is missing, the chained certificate generation will cause the puppet run to fail.}}


=== Hiera ===
=== Hiera ===


We can also add certificates by adding the configuration to hiera via the <code>profile::pki::client::certs</code>.  Entries here are passed directly to [https://github.com/wikimedia/puppet/blob/production/modules/cfssl/manifests/cert.pp cfssl::cert] and support all parameters for that resource.  In the following example we add <code>provide_chain: true</code> which should probably be the default but at the time of writing there are [[PKI/root_ca#Renewing_a_new_intermediate|some improvements]] which could be made to bundle handling and for puppet managed hosts it may make more senses to push all intermediate cert to all puppet agents.
We can also add certificates by adding the configuration to hiera via the <code>profile::pki::client::certs</code>.  Entries here are passed directly to [https://github.com/wikimedia/puppet/blob/production/modules/cfssl/manifests/cert.pp cfssl::cert] and support all parameters for that resource.  In the following example we add <code>provide_chain: true</code>:<ref>This should probably be the default, but at the time of writing there are [[PKI/root_ca#Renewing_a_new_intermediate|some improvements]] which could be made to bundle handling, and for puppet-managed hosts it may make more sense to push all intermediate certs to all puppet agents.</ref>


<syntaxhighlight lang="yaml">
<syntaxhighlight lang="yaml">
Line 67: Line 69:
</syntaxhighlight>
</syntaxhighlight>


As mentioned above certificates are set with a short TTL and are set to auto expiry as such it is likely that you may need to auto restart some servie when the certificate is renewed for this you can use the <code>notify_service</code> parameter.
As mentioned above certificates are set with a short TTL and are set to auto expire; as such you may need to auto restart some service when the certificate is renewed. For this you can use the <code>notify_service</code> parameter.
<syntaxhighlight lang="yaml">
<syntaxhighlight lang="yaml">
profile::pki::client::certs:
profile::pki::client::certs:
Line 76: Line 78:
     notify_service: foobar
     notify_service: foobar
</syntaxhighlight>
</syntaxhighlight>
in the above example <code>Service['foobar']</code> will be notified when the certificate is renewed.
In the above example, <code>Service['foobar']</code> will be notified when the certificate is renewed.
 
 
=== Puppet ===
=== Puppet ===
Some times it is required to have more complex relations ships or keep resources in specific manifests to ease debugging and readability as such it is also possible to use the <code>cfssl::cert</code> resource directly.
Sometimes it is required to have more complex relationships or keep resources in specific manifests to ease debugging and readability. As such it is also possible to use the <code>cfssl::cert</code> resource directly.


<syntaxhighlight lang="puppet">
<syntaxhighlight lang="puppet">
Line 96: Line 96:
</syntaxhighlight>
</syntaxhighlight>


Defining the resource in puppet allows for more complex relations as indicated in the following example
Defining the resource in puppet allows for more complex relations as indicated in the following example:


<syntaxhighlight lang="puppet">
<syntaxhighlight lang="puppet">
Line 128: Line 128:


= Proxy Signing =
= Proxy Signing =
The pki client supports the ability to start a cfssl api service and proxy the signing request to the main multicaroot instance.  The proxy server still needs to have access to a valid puppet agent certificate to use for outbound connections to the multiroot ca server.  however it is able to offer a http endpoint locally which does not have this restrictions.  This allows for an infrastructure where production puppet host can provide PKI services to none-puppet hosts.  The main driver for this was to allow k8s hosts to provide PKI services to there containers over localhost.  To enable the proxy services simply add the following configuration to hiera
The PKI client can start a cfssl api service and proxy the signing request to the main multicaroot instance.  The proxy server still needs to have access to a valid puppet agent certificate to use for outbound connections to the multiroot ca server, but it offers an http endpoint locally which does not have these restrictions.  This allows production puppet hosts to provide PKI services to non-puppet hosts.  The main driver for this was to allow k8s hosts to provide PKI services to their containers over localhost.  To enable the proxy services simply add the following configuration to hiera:


<syntaxhighlight lang="yaml">
<syntaxhighlight lang="yaml">
Line 142: Line 142:
= CLI Interactions =
= CLI Interactions =


Users are also able interact with CFSSL using the <code>cfssl</code> command line tool.  The instructions below assume you are running commands from a puppet enabled host in the production network.  the pki server is needs is protected via TLS mutal auth using the puppet CA, further there is an additional secret token held in /etc/cfssl/client-cfssl.conf.
Users are also able interact with CFSSL using the <code>cfssl</code> command line tool.  The instructions below assume you are running commands from a puppet-enabled host in the production network.  The pki server is protected via TLS mutual auth using the puppet CA; in addition there is an additional secret token held in /etc/cfssl/client-cfssl.conf.
 
First, create a certificate signing request in json format e.g. <code>crs.json</code>


First need to create a certificate signing request in json formate.g. <code>crs.json</code>
First, create a certificate signing request in json format (e.g. <code>crs.json</code>):


<syntaxhighlight lang="json">
<syntaxhighlight lang="json">
Line 164: Line 166:
</syntaxhighlight>
</syntaxhighlight>


Once this is in place you can request send it to the pki server and as for a signature with the following command:
Once this is in place you can send it to the pki server for a signature with the following command:


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
Line 176: Line 178:
</syntaxhighlight>
</syntaxhighlight>


<code>${ca_label}</code> is the name of the intermediate, see of [[PKI/CA Operations#Current intermediates]].
<code>${ca_label}</code> is the name of the intermediate; see [[PKI/CA Operations#Current intermediates]].


This command will produce a json response with the certificate, key and csr.  To automatically parse out the json you can pip the command to <code>cfssljson -bare $dir/$basename<code> e.g.
This command will produce a json response with the certificate, key and csr.  To automatically parse out json you can pipe the command to <code>cfssljson -bare $dir/$basename</code> e.g.
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
$ /usr/bin/cfssl gencert \
$ /usr/bin/cfssl gencert \
Line 195: Line 197:
</syntaxhighlight>
</syntaxhighlight>


Users can also resign the certificate by using pretty much the same command however change <code>gencert</code> to <code>sign</code> and use the current pem certificate instead of the json csr
Users can also re-sign the certificate by using pretty much the same command, changing <code>gencert</code> to <code>sign</code> and using the current pem certificate instead of the json csr:


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
Line 206: Line 208:
   /path/to/certs/basename.pem| /usr/bin/cfssljson -bare /path/to/certs/basename
   /path/to/certs/basename.pem| /usr/bin/cfssljson -bare /path/to/certs/basename
</syntaxhighlight>
</syntaxhighlight>
= Kubernetes Integration =
Kubernetes clusters may use a combination of [[Kubernetes/cert-manager|cert-manager and cfssl-issuer]] to provide certificates for workload inside the cluster.

Latest revision as of 13:28, 16 March 2022

We currently use CFSSL to provide and manage PKI solutions. Clients are able to make use of the CFSSL API endpoint available at https://pki.discovery.wmnet/, however this endpoint is protected via TLS Client authentications and currently requires users to use the puppet agent certificate. The puppet agent certificate was chosen as it is something we already issue when a host is provisioned or reimaged; we can change this at a later date or configure additional intermediate CAs which have different authentication mechanisms. In addition to the client auth requirement, API requests also need to be signed with hmac using a secret key (available in the puppet private repo).

Puppet integration

Puppet hosts can start using the PKI infrastructure by adding the profile::pki::client to their role. This will configure the host with the required CFSSL configuration to be able request new certificates for the configured intermediate certificates.

Creating certificates

When a user wants to create a new certificate, they will first need to decide which intermediate CA they want to use. Note that each realm of operations using client auth should have its own intermediate CA and you should not use cross functional CA's. For example: debmonitor, DB connections, and conftool should all have their own intermediate CA's. If you need to create a new intermediate CA, see the CA Operations Page. Unless the defaults have been changed, all certificates will be issued with an expiry of 672h and set to autorenew 10days hours before they expire.

Once you know which intermediate CA you want to use to issue your certificate, you will need to know which profile to use. By default, intermediate certificates are created with two profiles. The default profile is for client auth and the server profile is for daemon services. If you have created additional profiles for your CA, you can use them as below. The below example assumes we want to create two certificates in the "WMF testing CA", one using the default profile and one using the server profile.

profile::pki::get_cert

The simplest interface to for creating certificates is the profile::pki::get_cert function. This function has a similar signature to cfssl::cert, but it also returns the path of the cert, the private key, and the CA bundle (chained). As you most often need to have these paths for other parts of puppet configurations, this is likely the most useful interface.

class profile::foobar () {
    $ssl_paths = profile::pki::get_cert('WMF_testing_CA')
    $config = {
      ssl_cert = $ssl_paths['cert']
      ssl_key = $ssl_paths['key']
    }
    file {'/etc/ssl_config.json':
      ensure => file,
      content = $config.to_json
    }
 }

To also fetch the CA bundle:

profile::pki::get_cert('WMF_testing_CA', $facts['fqdn']) >> {
  'cert'    => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_$fqdn.pem',      # owned by root
  'key'     => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_$fqdn-key.pem',  # owned by root
  'chain'   => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_chain.pem',      # owned by root
  'chained' => '/etc/cfssl/ssl/WMF_testing_CA/WMF_testing_CA_chained.pem',    # owned by root
}

To fetch the CA bundle, changing the outputdir and user:

profile::pki::get_cert(
  'WMF_testing_CA',
  $facts['fqdn'],
  {'outdir' => '/etc/foobar', 'owner' => 'foobar'}
) >> {
  'cert'    => '/etc/foobar/WMF_testing_CA_$fqdn.pem',     # owned by foobar
  'key'     => '/etc/foobar/WMF_testing_CA_$fqdn-key.pem', # owned by foobar
  'chain'   => '/etc/foobar/WMF_testing_CA_chain.pem',     # owned by foobar
  'chained' => '/etc/foobar/WMF_testing_CA_chained.pem',   # owned by foobar
}

Hiera

We can also add certificates by adding the configuration to hiera via the profile::pki::client::certs. Entries here are passed directly to cfssl::cert and support all parameters for that resource. In the following example we add provide_chain: true:[1]

profile::pki::client::certs:
  '%{facts.networking.fqdn}':
    # label is escaped with regsubst('[^\w\-]', '_', 'G')
    label: WMF_testing_CA
    provide_chain: true
  foobar.example.com:
    label: WMF_test_intermediate_ca
    provide_chain: true
    profile: server

As mentioned above certificates are set with a short TTL and are set to auto expire; as such you may need to auto restart some service when the certificate is renewed. For this you can use the notify_service parameter.

profile::pki::client::certs:
  foobar.example.com:
    label: WMF_test_intermediate_ca
    provide_chain: true
    profile: server
    notify_service: foobar

In the above example, Service['foobar'] will be notified when the certificate is renewed.

Puppet

Sometimes it is required to have more complex relationships or keep resources in specific manifests to ease debugging and readability. As such it is also possible to use the cfssl::cert resource directly.

class profile::foobar () {
  cfssl::cert{ $facts['networking']['fqdn']:
    label         => WMF_testing_CA,
    provide_chain => true,
  }
  cfssl::cert{ 'foobar.example.com':
    label         => WMF_testing_CA,
    provide_chain => true,
    profile       => 'server',
  }
}

Defining the resource in puppet allows for more complex relations as indicated in the following example:

class profile::foobar () {
  exec {'foobar_exec':
    command     => 'run a command every time the client cert is renewed'
    refreshonly => true,
  }
  service {'foobar':
    ensure => 'running',
  }
  cfssl::cert{ $facts['networking']['fqdn']:
    label         => WMF_testing_CA,
    provide_chain => true,
    notify        => Exec['foobar_exec'],
  }
  cfssl::cert{ 'foobar.example.com':
    label          => WMF_testing_CA,
    provide_chain  => true,
    profile        => 'server',
    notify_service => 'foobar',
    # We could also use the following however notify_service is a bit more optimal
    # notify        => Service['foobar'],
  }
  service {'another_foobar':
    ensure => 'running',
    subscribe => Cfssl::Cert['foobar.example.com'],
  }
}

Proxy Signing

The PKI client can start a cfssl api service and proxy the signing request to the main multicaroot instance. The proxy server still needs to have access to a valid puppet agent certificate to use for outbound connections to the multiroot ca server, but it offers an http endpoint locally which does not have these restrictions. This allows production puppet hosts to provide PKI services to non-puppet hosts. The main driver for this was to allow k8s hosts to provide PKI services to their containers over localhost. To enable the proxy services simply add the following configuration to hiera:

profile::pki::client::enable_proxy: true
# By default the proxy service listens on http://127.0.0.1:8888/
# This can be changed with the following settings however one should 
# be very cautious in considering having this listen on anything other then localhost 
profile::pki::client::listen_addr: '127.0.0.1'
profile::pki::client::listen_port: 8888

CLI Interactions

Users are also able interact with CFSSL using the cfssl command line tool. The instructions below assume you are running commands from a puppet-enabled host in the production network. The pki server is protected via TLS mutual auth using the puppet CA; in addition there is an additional secret token held in /etc/cfssl/client-cfssl.conf.

First, create a certificate signing request in json format e.g. crs.json

First, create a certificate signing request in json format (e.g. crs.json):

{
  "hosts": [
    "cumin1001.eqiad.wmnet"
  ],
  "key": {
    "algo": "ecdsa",
    "size": 256
  },
  "names": [
    {
      "organisation": "SnakeOil", 
      "country": "uk"
    }
  ]
}

Once this is in place you can send it to the pki server for a signature with the following command:

$ /usr/bin/cfssl gencert \
  -config /etc/cfssl/client-cfssl.conf \
  -tls-remote-ca $(facter -p puppet_config.localcacert) \
  -mutual-tls-client-cert $(facter -p puppet_config.hostcert) \
  -mutual-tls-client-key $(facter -p puppet_config.hostprivkey) \
  -label ${ca_label} -profile server \  
  csr.json

${ca_label} is the name of the intermediate; see PKI/CA Operations#Current intermediates.

This command will produce a json response with the certificate, key and csr. To automatically parse out json you can pipe the command to cfssljson -bare $dir/$basename e.g.

$ /usr/bin/cfssl gencert \
  -config /etc/cfssl/client-cfssl.conf \
  -tls-remote-ca $(facter -p puppet_config.localcacert) \
  -mutual-tls-client-cert $(facter -p puppet_config.hostcert) \
  -mutual-tls-client-key $(facter -p puppet_config.hostprivkey) \
  -label ${ca_label} -profile server \  
  csr.json | /usr/bin/cfssljson -bare /path/to/certs/basename
$ ls -la /path/to/certs/
drwxr-xr-x  2 root root 4096 Jun 28 12:11 .
drwx------ 11 root root 4096 Jun 28 12:11 ..
-rw-r--r--  1 root root  395 Jun 28 12:11 basename.csr
-rw-------  1 root root  227 Jun 28 12:11 basename-key.pem
-rw-r--r--  1 root root 1054 Jun 28 12:11 basename.pem

Users can also re-sign the certificate by using pretty much the same command, changing gencert to sign and using the current pem certificate instead of the json csr:

$ /usr/bin/cfssl sign \
  -config /etc/cfssl/client-cfssl.conf \
  -tls-remote-ca $(facter -p puppet_config.localcacert) \
  -mutual-tls-client-cert $(facter -p puppet_config.hostcert) \
  -mutual-tls-client-key $(facter -p puppet_config.hostprivkey) \
  -label ${ca_label} -profile server \  
  /path/to/certs/basename.pem| /usr/bin/cfssljson -bare /path/to/certs/basename

Kubernetes Integration

Kubernetes clusters may use a combination of cert-manager and cfssl-issuer to provide certificates for workload inside the cluster.

  1. This should probably be the default, but at the time of writing there are some improvements which could be made to bundle handling, and for puppet-managed hosts it may make more sense to push all intermediate certs to all puppet agents.