Bolt

Puppet Bolt

Puppet Bolt supports a variety of uses cases, but one in particular is interesting for our use, masterless Puppet catalog applies. In a masterless apply rather than obtaining the catalog from the puppet master the catalog is compiled directly from the source repository. One advantage of a masterless apply is that it can offer faster feedback on code changes, since you don’t need to commit in order to apply or noop your changes against a destination host, i.e.:

# Edit
$ vi modules/example/manifests/init.pp

# Noop to see changes
$ bolt apply --noop -t node.example.com

# Commit
$ git commit -a -m 'add example module'

Other projects which support a masterless workflow:

• supply_drop

Install Directions

Install Bolt, not currently in Debian unfortunately

$ wget https://apt.puppet.com/puppet-tools-release-bullseye.deb
$ sudo dpkg -i puppet-tools-release-bullseye.deb
$ sudo apt-get update
$ sudo apt-get install puppet-bolt

Patch Bolt to read from named pipes

Upstream pull request

$ sudo patch -p1 -d /opt/puppetlabs/bolt/lib/ruby/gems/2.7.0/gems/bolt-3.21.0/ <<EOF
commit 5d70449800c86d76f2f6775710cec39984984af5
Author: Jesse Hathaway <jesse@mbuki-mvuki.org>
Date:   Tue Feb 15 15:00:25 2022 -0600

    bolt apply: allow reading manifest from a named pipe
    
    Prior to this commit bolt threw an error when given a manifest which
    resolves to a named pipe:
    
        $ bolt apply -t butter.example.com <(cat manifests/*.pp)
    
    After this commit bolt allows reading from the named pipes!

diff --git a/lib/bolt/util.rb b/lib/bolt/util.rb
index 2980ea3d..a28c5d14 100644
--- a/lib/bolt/util.rb
+++ b/lib/bolt/util.rb
@@ -344,7 +344,7 @@ module Bolt
 
         if !stat.readable?
           raise Bolt::FileError.new("The #{type} '#{path}' is unreadable", path)
-        elsif !stat.file? && (!allow_dir || !stat.directory?)
+        elsif !allow_dir && stat.directory?
           expected = allow_dir ? 'file or directory' : 'file'
           raise Bolt::FileError.new("The #{type} '#{path}' is not a #{expected}", path)
         elsif stat.directory?
EOF

Delete Bolt’s system module

This module conflicts with our system module, Upstream pull request or rename our module

$ sudo rm -r /opt/puppetlabs/bolt/lib/ruby/gems/2.7.0/gems/bolt-3.21.0/bolt-modules/system

Add Bolt config, labs-private, and supporting modules

$ cat >boltup <<EOM
#!/bin/bash

set -o errexit
set -o nounset

cat >bolt-project.yaml <<EOF
---
name: wmf
apply-settings:
  show_diff: true
hiera-config: 'hiera.yaml'
modulepath:
  - 'private/modules'
  - 'modules'
  - 'bolt/modules'
EOF

cat >inventory.yaml <<EOF
config:
  transport: ssh
  ssh:
      interpreters:
          # Use our system ruby
          rb: /usr/bin/ruby
      # Switch to root before running puppet
      run-as: root
features:
    # Do not try to install the puppet-agent
    - puppet-agent
EOF

# setup hiera
sed -E 's#/etc/puppet/##' modules/puppetmaster/files/production.hiera.yaml >hiera.yaml

# clone repos
if [[ ! -e 'private' ]]; then
    git clone git@github.com:wikimedia/labs-private.git private
fi
mkdir -p bolt/modules
if [[ ! -e 'bolt/modules/nagios_core' ]]; then
    git clone git@github.com:puppetlabs/puppetlabs-nagios_core.git bolt/modules/nagios_core
fi
if [[ ! -e 'bolt/modules/mailalias_core' ]]; then
    git clone git@github.com:puppetlabs/puppetlabs-mailalias_core.git bolt/modules/mailalias_core
fi
EOM
$ bash boltup

Usage

Noop a server

$ bolt apply --noop -t mirror1001.wikimedia.org <(cat manifests/*.pp)

Outstanding Issues

1. Bolt does not preserve symlinks on files with recurse and source directories, https://github.com/puppetlabs/bolt/issues/3056
2. PuppetDB queries do not work, nor do exported resources
3. acme_chief module does not work
4. Nooping a server with Bolt will take out a Puppet lock, which would block a cron based Puppet run which began after the Bolt run