You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Talk:Puppet Hiera: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Giuseppe Lavagetto
 
imported>BCornwall
(→‎How to associate node and role: Replace deprecated source tag with syntaxhighlight)
Line 7: Line 7:


''site.pp''
''site.pp''
<source lang="ruby">
<syntaxhighlight lang="ruby">
node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
     include role::foo
     include role::foo
}
}
</source>
</syntaxhighlight>


''hieradata/regex.yaml''
''hieradata/regex.yaml''
<source lang="yaml">
<syntaxhighlight lang="yaml">
foocluster:
foocluster:
   __regex: !ruby/regex /^mc10[0-9][0-9]\.eqiad\.wmnet$/
   __regex: !ruby/regex /^mc10[0-9][0-9]\.eqiad\.wmnet$/
   mainrole: foo_runner
   mainrole: foo_runner
</source>
</syntaxhighlight>


''hieradata/mainrole/foo_runner.yaml''
''hieradata/mainrole/foo_runner.yaml''
<source lang="yaml">
<syntaxhighlight lang="yaml">
foo::queue_len: 100
foo::queue_len: 100
foo::threads: 8
foo::threads: 8
Line 28: Line 28:
   - "apache"
   - "apache"
...
...
</source>
</syntaxhighlight>


Which, while pretty organized, has the non-secondary disadvantage of asking us to replicate regexes twice.
Which, while pretty organized, has the non-secondary disadvantage of asking us to replicate regexes twice.
Line 36: Line 36:


''site.pp''
''site.pp''
<source lang="ruby">
<syntaxhighlight lang="ruby">
node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
     $mainrole = "foo_runner"
     $mainrole = "foo_runner"
     include role::foo
     include role::foo
}
}
</source>
</syntaxhighlight>


''hieradata/mainrole/foo_runner.yaml''
''hieradata/mainrole/foo_runner.yaml''
<source lang="yaml">
<syntaxhighlight lang="yaml">
foo::queue_len: 100
foo::queue_len: 100
foo::threads: 8
foo::threads: 8
Line 51: Line 51:
   - "apache"
   - "apache"
...
...
</source>
</syntaxhighlight>
The drawback of this solution is that anything evaluated before the node scope is entered will not be correctly affected by this. Also, it makes sites.pp ugly (but this is a personal taste of mine), and still forces us to declare a relationship that should be more relative to the applied role than to the nodes. Subgroup-specific settings could still be defined in one of the two preceding ways.
The drawback of this solution is that anything evaluated before the node scope is entered will not be correctly affected by this. Also, it makes sites.pp ugly (but this is a personal taste of mine), and still forces us to declare a relationship that should be more relative to the applied role than to the nodes. Subgroup-specific settings could still be defined in one of the two preceding ways.


Line 60: Line 60:


''manifests/role/foo.pp''
''manifests/role/foo.pp''
<source lang="ruby">
<syntaxhighlight lang="ruby">
class role::foo {
class role::foo {


     # All the code that usually is in the role class.
     # All the code that usually is in the role class.
}
}
</source>
</syntaxhighlight>
''manifests/site.pp''
''manifests/site.pp''
<source lang="ruby">
<syntaxhighlight lang="ruby">
node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
     role foo
     role foo
}
}
</source>
</syntaxhighlight>


''hieradata/role/foo.yaml''
''hieradata/role/foo.yaml''
<source lang="yaml">
<syntaxhighlight lang="yaml">
foo::queue_len: 100
foo::queue_len: 100
foo::threads: 8
foo::threads: 8
Line 81: Line 81:
   - "apache"
   - "apache"
...
...
</source>
</syntaxhighlight>


which could be overridden in ''hieradata/role/eqiad/foo.yaml''.
which could be overridden in ''hieradata/role/eqiad/foo.yaml''.
Line 88: Line 88:
* If more than one role is defined in a node def, all the hiera lookups on the roles must be identical, or an error will be thrown
* If more than one role is defined in a node def, all the hiera lookups on the roles must be identical, or an error will be thrown
* Things that depend on this declared in the top scope (and not in the node scope) will not be able to access this.
* Things that depend on this declared in the top scope (and not in the node scope) will not be able to access this.
== hiera.yaml no longer used? ==
The introductory section mentions the following:
<blockquote>Every variable that puppet looks up in hiera will be searched via one or more backends (at the moment, two a yaml-based ones) according to what we configured in the ''hiera.yaml'' file in the base puppet directory.</blockquote>
Looking at the repo seems to suggest that hiera.yaml is not used but instead hieradata/ is used instead. Does this make more sense?
<blockquote>Every variable that puppet looks up in hiera will be searched via one or more backends (at the moment, two a yaml-based ones) according to what we configured in the ''hieradata/'' directory (Located in the base directory of the Puppet repository).</blockquote>
I'm a newbie to Puppet so forgive me if I'm overlooking something. But given that this page was written in 2014 it seems more likely that something has changed here :)
--[[User:BCornwall|BCornwall]] ([[User talk:BCornwall|talk]]) 23:07, 11 May 2022 (UTC)

Revision as of 01:04, 12 May 2022

How to associate node and role

In most cases, we want a set of servers to have very similar configuration, which is in most cases a 1:1 correspondence with the role class that is included in the node itself; but this is not always the case. So the solution is to have some "label" applied to the server, and then use it in looking up variables in hiera.

How we do this today

Right now, you would probably want to do it like this:

site.pp

node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
    include role::foo
}

hieradata/regex.yaml

foocluster:
  __regex: !ruby/regex /^mc10[0-9][0-9]\.eqiad\.wmnet$/
  mainrole: foo_runner

hieradata/mainrole/foo_runner.yaml

foo::queue_len: 100
foo::threads: 8
admin::groups:
  - "oompah_loompahs"
  - "apache"
...

Which, while pretty organized, has the non-secondary disadvantage of asking us to replicate regexes twice.

The simplest alternative

The simplest and most obvious alternative would be to have a node-scope variable to use in every stanza to define the role we're applying:

site.pp

node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
    $mainrole = "foo_runner"
    include role::foo
}

hieradata/mainrole/foo_runner.yaml

foo::queue_len: 100
foo::threads: 8
admin::groups:
  - "oompah_loompahs"
  - "apache"
...

The drawback of this solution is that anything evaluated before the node scope is entered will not be correctly affected by this. Also, it makes sites.pp ugly (but this is a personal taste of mine), and still forces us to declare a relationship that should be more relative to the applied role than to the nodes. Subgroup-specific settings could still be defined in one of the two preceding ways.

Another idea

We can make the role/node relationship unique; this will be ok in 99.9% of the cases, as long as it can be manually overridden. The main advantage of this is that we tie functionality to the role class, and not to the node group. We can also declare multiple roles, and we'll have a dedicated hiera backend for this

So we would have:

manifests/role/foo.pp

class role::foo {

    # All the code that usually is in the role class.
}

manifests/site.pp

node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
    role foo
}

hieradata/role/foo.yaml

foo::queue_len: 100
foo::threads: 8
admin::groups:
  - "oompah_loompahs"
  - "apache"
...

which could be overridden in hieradata/role/eqiad/foo.yaml.

This has some slight caveats I can think of:

  • If more than one role is defined in a node def, all the hiera lookups on the roles must be identical, or an error will be thrown
  • Things that depend on this declared in the top scope (and not in the node scope) will not be able to access this.

hiera.yaml no longer used?

The introductory section mentions the following:

Every variable that puppet looks up in hiera will be searched via one or more backends (at the moment, two a yaml-based ones) according to what we configured in the hiera.yaml file in the base puppet directory.

Looking at the repo seems to suggest that hiera.yaml is not used but instead hieradata/ is used instead. Does this make more sense?

Every variable that puppet looks up in hiera will be searched via one or more backends (at the moment, two a yaml-based ones) according to what we configured in the hieradata/ directory (Located in the base directory of the Puppet repository).

I'm a newbie to Puppet so forgive me if I'm overlooking something. But given that this page was written in 2014 it seems more likely that something has changed here :) --BCornwall (talk) 23:07, 11 May 2022 (UTC)