You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
Puppet 3.x migration
Migration from puppet 2.7.x to 3.y (where y should be 2 or higher) is not going to be pain-free, but we want it at least to be as safe as possible.
To this end, we have created a set of tools that could help us all in finding out the differences in produced catalogs between the two puppet versions.
At the moment the best way to check which differences there are between the catalogs produced by puppet 2.7 and puppet 3.x is to compile the modules with both versions and control the differences.
Things to watch out for
Here is an (under construction) list of things to watch out for while we migrate.
Dynamic scope is gone
Dashes in class names are not allowed
Well, sort of. The trick is, you can include a class with dashes in its name, but you will not be able to reference ANY of its variables. With the dynamic scope lookup gone, this is going to make your classes useless. Don't use dashes in class names.
Templates should be written correctly
The second biggest change is in how variables are looked up and treated in templates; the changes are both in scoping and in syntax, see the puppetlabs template reference.
- You should reference variables with @myvariable if that variable is defined in the same scope where the template is defined, or by using the scope.lookupvar function. See this blog post for a practical example.
- Undefined values have changed behaviour;
- If you are using functions within templates, your invokation should:
- be a child of the scope object
- use an array as argument even if you're passing on just one value.
Custom function argument must always be an array
This is painful, as if you have written a scalar function this will break your work completely. Say you had a function to capitalize a string
module Puppet::Parser::Functions newfunction(:myfunc, :type => :rvalue) do |args| fail 'myfunc() requires an argument' if args.empty? args.capitalize end end
and you are using that in puppet as follows:
you need to transform it to a function working on arrays, in example:
module Puppet::Parser::Functions newfunction(:myfunc, :type => :rvalue) do |args| fail 'myfunc() requires an argument' if args.empty? fail 'myfunc() takes exactly one argument' if args.length > 1 args.first.capitalize end end
Puppet Catalogs compiler
compare-puppet-catalogs (inside operations/software repo) is a small utility to assist in the migration of your Puppet manifests from Puppet 2.7 to Puppet 3.0.
It has several levels of inceptions but should be easily runnable using the provided Vagrant file.
There is more information about catalog compilation in the puppet-compiler page.
Preparing the environment
First of all, provision and start the vagrant box
Fetch a tar archive of our production facts, i.e. on palladium:
sudo tar cJf /tmp/puppet-facts.tar.xz /var/lib/puppet/yaml
and copy it to the base directory of compare-puppet-catalog on your host.
Now, you should ssh into your vagrant box and become root
vagrant ssh sudo -i export PATH="$PATH:/vagrant" # Set up your environment /vagrant/shell/helper install
This last command will take a while to execute. Now you can see the help of the command
Comparator has three main working modes:
1) See the differences between catalogs compiled in puppet 2.7 and puppet 3 on the production branch:
comparator --numthreads 1 \ --nodes amssq55.esams.wikimedia.org,fenari.wikimedia.org
if you omit --nodes, it will run on a predetermined set of nodes (see below). If you have configured your vagrant box to have more than one core, raise the --numthreads parameter to speed up catalog compilation.
2) See the effects of a Gerrit changeset on the puppet catalogs by using the --change argument:
comparator --numthreads 1 --change 130591 \ --nodes amssq55.esams.wikimedia.org,fenari.wikimedia.org
3) Same as 1), but for a Gerrit changeset:
comparator --numthreads 1 --change 130591 --transition\ --nodes amssq55.esams.wikimedia.org,fenari.wikimedia.org
The run should produce, apart from some useless logs on stdout, a tree of results, that you will be able to browse from your browser at one of the following locations:
http://localhost:8080/html (case 1)
http://localhost:8080/change/<change_id>/html/ (case 2 or 3)
Default node selection
By default, comparator selects the nodes to run onto intelligently: it will inspect your site.pp (from the puppet tree to examine) and match all the nodes you have facts for to it; then it will pick only one (random) node matching any specific node definition. This way, we should have 99% coverage with tests running on less than 15% of the hosts.
GPLv2, see file LICENSE.