You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Portal:Cloud VPS/Admin/Hiera: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>David Caro
imported>Majavah
No edit summary
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
{{draft}}
This page contains information on how '''hiera''' data is managed in [[Portal:Cloud_VPS|Cloud VPS]], which is a bit different from the [[Puppet_Hiera | main production hiera/puppet workflow]].
This page contains information on how '''hiera''' data is managed in [[Portal:Cloud_VPS|Cloud VPS]], which is a bit different from the [[Puppet_Hiera | main production hiera/puppet workflow]].


= Precedence of hiera keys =
== Precedence of hiera keys ==


Hiera keys may come from different backends, not only puppet hieradata files.
Hiera keys may come from different backends, not only puppet hieradata files.
Line 39: Line 38:
Only users with the ''projectadmin'' role can edit hiera data in Horizon and on Wikitech. Everyone can propose a patch in gerrit to change the hiera data in git, which can then be merged by all of SRE/Ops.
Only users with the ''projectadmin'' role can edit hiera data in Horizon and on Wikitech. Everyone can propose a patch in gerrit to change the hiera data in git, which can then be merged by all of SRE/Ops.


== puppetmaster configuration ==
== Puppet enc system ==


In order to select a backend/source for hiera data, puppetmasters have a '''/etc/pupppet/hiera.yaml''' configuration file with several backends in order, for example:
The enc mechanism is a custom made API that serves up hiera config (it's what horizon talks to). See the image below for a quick overview.


<pre>
[[File:Cloud puppet enc.png|thumb|right|Overview of the ENC system]]
tools-puppetmaster-02:~# cat /etc/puppet/hiera.yaml
[...]
:backends:
  - httpyaml
  - mwyaml
  - nuyaml
[...]
</pre>


In the example above:
=== ENC API ===
* nuyaml == local config files
The ENC API is powered by a Flask microservice running on small VMs in the [[Portal:Cloud VPS/Admin/Cloudinfra|cloudinfra]] project. It makes two types of endpoints accessible to different instances, one with read-write permissions and one with only read ones. It uses a database on <code>cloudinfra-db*.cloudinfra.eqiad1.wikimedia.cloud</code> to store the hiera and roles information.
* mwyaml == wikitech
* httpyaml == data that horizon writes on the puppetmaster


= Puppet enc system =
The code for the service is [https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/production/modules/openstack/files/puppet/master/encapi/puppet-enc.py here].


The enc mechanism is a custom made API that serves up hiera config (it's what horizon talks to). See the image to the right for a quick overview.
The read only endpoint endpoint is exposed via Nginx on port 8100. It rejects all requests with a status code not equal to <code>GET</code>. This endpoint is used by various puppetmasters to query node information.


[[File:Cloud puppet enc.png|frame|Cloud puppet enc overview (source https://phabricator.wikimedia.org/F34401412)]]
The writable endpoint is exposed directly from uwsgi on port 8101. It is secured by Ferm firewalls and app-layer code for IP-based access control, although migration to Keystone authentication tokens is planned. This endpoint is used by Horizon, wmfsink hosted on the cloudcontrol nodes and manual operations from cloudcontrol nodes.


=== Horizon UI ===
The primary way of interacting with the ENC backend is via [https://gerrit.wikimedia.org/g/openstack/horizon/wmf-puppet-dashboard the custom Horizon panel].


== Labspuppetbackend ==
Note that the dashboard mirrors all writes to the cloud/instance-puppet.git Gerrit repository. That repository is not read by the infrastructure itself but is useful if you're trying to grep for a particular setting.
This web service makes two types of endpoints accessible to different instances, one with read-write permissions and one with only read ones.


It uses a database on <code>db01.cloudinfra.equiad.wmflabs</code> to store the hiera and roles information.
=== cli ===
The code for the service is [https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/openstack/files/puppet/master/encapi/labspuppetbackend.py here].
 
=== Read endpoint ===
The read only endpoint (referred to as labspuppetbackendgetter in many places), is hosted by nginx, on <code>cloud-puppetmaster-03</code> (when writing this), listening on port <code>8100</code> (http) and <code>8143</code> (https).
 
It's ensuring read only by declining any non <code>GET</code> http request.
 
=== Read-write endpoint ===
The read-write one is hosted directly to uwsgi, open on the same host on port <code>8000</code>.
 
=== Cli ===
There's a small cli that will allow interacting with the enc api installed on all the control nodes (<code>cloudcontrol*</code>):
There's a small cli that will allow interacting with the enc api installed on all the control nodes (<code>cloudcontrol*</code>):
{{Codesample
{{Codesample
Line 94: Line 73:
   -h, --help            show this help message and exit
   -h, --help            show this help message and exit
   --enc-url ENC_URL    Full url to the enc, for example
   --enc-url ENC_URL    Full url to the enc, for example
                         <nowiki>http://puppetmaster.cloudinfra.wmflabs.org:8101/v1</nowiki>
                         <nowiki>http://puppet-enc.cloudinfra.wmcloud.org:8101/v1</nowiki>
   --openstack-project OPENSTACK_PROJECT
   --openstack-project OPENSTACK_PROJECT
| line = yes
| line = yes
}}
}}


The code for it is hosted [https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/openstack/files/util/wmcs-enc-cli.py here].
The code for it is hosted [https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/openstack/files/util/wmcs-enc-cli.py here]. Note that it does not sync changes to the cloud/instance-puppet.git Gerrit repository.


= puppet-enc =
=== puppet-enc ===
This is a cli that puppet runs to retrieve the actual node ENC data <ref>[https://puppet.com/docs/puppet/5.5/nodes_external.html]Puppet external node classifier docs</ref>.
This is a cli that puppet runs to retrieve the actual node ENC data <ref>[https://puppet.com/docs/puppet/5.5/nodes_external.html]Puppet external node classifier docs</ref>.
It's a wrapper on top of <code>labspuppetbackend</code> that just transforms the output format to something puppet understands.
It's a wrapper on top of the read-only enc api endpoint that just transforms the output format to something puppet understands.


The code for it is hosted [https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/openstack/files/puppet/master/labs-puppet-enc.py here].
The code for it is hosted [https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/profile/files/openstack/base/puppetmaster/enc_client/puppet_enc.py here].
Note that it uses the configuration file <code>/etc/puppet-enc.yaml</code> to retrieve the host for the <code>labspuppetbackend</code> service.
Note that it uses the configuration file <code>/etc/puppet-enc.yaml</code> to retrieve the host for the ENC api endpoint (instead of using Keystone service discovery).


= Wikitech integration =
=== Wikitech integration ===


Our intention is to kill the wikitech hiera keys definitions. But wikitech has editing history and Horizon doesn't ([[phab:T153036]]).
In the past you could configure hieradata in Wikitech, but that is no longer possible. The person writing this text does not know any further details, if you have them, feel free to fill them here!


= See also =
== See also ==
* Main article about [[Puppet_Hiera|Puppet/Hiera]].
* Main article about [[Puppet_Hiera|Puppet/Hiera]].
= References =
== References ==


<references />
<references />

Latest revision as of 18:40, 12 April 2022

This page contains information on how hiera data is managed in Cloud VPS, which is a bit different from the main production hiera/puppet workflow.

Precedence of hiera keys

Hiera keys may come from different backends, not only puppet hieradata files.

You can set hiera data by using Horizon or through diverse yaml-files in the operations/puppet and labs/private git repos. You can see the exact, always up-to-date resolution order at [2].

By default the hierarchy is searched using the following list top to bottom with a first match win policy. Also note the following considerations when reading this list

  • cloudlib::httpyaml: is a custom backbend to pull data from to Horizon(either at the instance level, prefixes, or project level).
  • /etc/puppet/hieradata/: this is a checkout of the operations/puppet git repo
  • /etc/puppet/private/hieradata: this is a checkout of the labs/private git repo
  • /etc/puppet/secret/hieradata: this is a directory on the stand-alone puppet master which doesn't exist by default
  • %{::wmcs_deployment}: This refers to the openstack deployment currently eqiad1 or codfw1dev
  • %{::labsproject}: This refers to the openstack project e.g. deployment-prep
  • %{::hostname} This refers to the instance name e.g. tools-mail.

The following is a dump as of 2021-03-24.

hierarchy:
- cloudlib::httpyaml
- "/etc/puppet/hieradata/cloud/%{::wmcs_deployment}/%{::labsproject}/host/%{::hostname}.yaml"
- "/etc/puppet/hieradata/cloud/%{::wmcs_deployment}/%{::labsproject}/common.yaml"
- "/etc/puppet/hieradata/cloud/%{::wmcs_deployment}.yaml"
- "/etc/puppet/secret/hieradata/labs/%{::labsproject}/common.yaml"
- "/etc/puppet/secret/hieradata/%{::labsproject}.yaml"
- "/etc/puppet/secret/hieradata/labs.yaml"
- "/etc/puppet/private/hieradata/labs/%{::labsproject}/common.yaml"
- "/etc/puppet/private/hieradata/%{::labsproject}.yaml"
- "/etc/puppet/private/hieradata/labs.yaml"
- "/etc/puppet/hieradata/common.yaml"
- "/etc/puppet/secret/hieradata/common.yaml"
- "/etc/puppet/private/hieradata/common.yaml"

Note that providing Hiera settings per role is not possible on Cloud Services.

Only users with the projectadmin role can edit hiera data in Horizon and on Wikitech. Everyone can propose a patch in gerrit to change the hiera data in git, which can then be merged by all of SRE/Ops.

Puppet enc system

The enc mechanism is a custom made API that serves up hiera config (it's what horizon talks to). See the image below for a quick overview.

File:Cloud puppet enc.png
Overview of the ENC system

ENC API

The ENC API is powered by a Flask microservice running on small VMs in the cloudinfra project. It makes two types of endpoints accessible to different instances, one with read-write permissions and one with only read ones. It uses a database on cloudinfra-db*.cloudinfra.eqiad1.wikimedia.cloud to store the hiera and roles information.

The code for the service is here.

The read only endpoint endpoint is exposed via Nginx on port 8100. It rejects all requests with a status code not equal to GET. This endpoint is used by various puppetmasters to query node information.

The writable endpoint is exposed directly from uwsgi on port 8101. It is secured by Ferm firewalls and app-layer code for IP-based access control, although migration to Keystone authentication tokens is planned. This endpoint is used by Horizon, wmfsink hosted on the cloudcontrol nodes and manual operations from cloudcontrol nodes.

Horizon UI

The primary way of interacting with the ENC backend is via the custom Horizon panel.

Note that the dashboard mirrors all writes to the cloud/instance-puppet.git Gerrit repository. That repository is not read by the infrastructure itself but is useful if you're trying to grep for a particular setting.

cli

There's a small cli that will allow interacting with the enc api installed on all the control nodes (cloudcontrol*):


root@cloudcontrol1003:~# wmcs-enc-cli --help
 usage: wmcs-enc-cli [-h] [--enc-url ENC_URL]
                     [--openstack-project OPENSTACK_PROJECT]
                     {get_node_consolidated_info,get_node_info,get_prefix_hiera,get_project_hiera,set_prefix_hiera}
                     [params [params ...]]
 positional arguments:
   {get_node_consolidated_info,get_node_info,get_prefix_hiera,get_project_hiera,set_prefix_hiera}
   params                Any parameters needed for the action chosen.
 optional arguments:
   -h, --help            show this help message and exit
   --enc-url ENC_URL     Full url to the enc, for example
                         http://puppet-enc.cloudinfra.wmcloud.org:8101/v1
   --openstack-project OPENSTACK_PROJECT

The code for it is hosted here. Note that it does not sync changes to the cloud/instance-puppet.git Gerrit repository.

puppet-enc

This is a cli that puppet runs to retrieve the actual node ENC data [1]. It's a wrapper on top of the read-only enc api endpoint that just transforms the output format to something puppet understands.

The code for it is hosted here. Note that it uses the configuration file /etc/puppet-enc.yaml to retrieve the host for the ENC api endpoint (instead of using Keystone service discovery).

Wikitech integration

In the past you could configure hieradata in Wikitech, but that is no longer possible. The person writing this text does not know any further details, if you have them, feel free to fill them here!

See also

References

  1. [1]Puppet external node classifier docs