Netbox

Netbox is a "IP address management (IPAM) and data center infrastructure management (DCIM) tool".

• https://github.com/digitalocean/netbox

At Wikimedia it has been evaluated in Phab:T170144 as a replacement for Racktables.

In Phab:T199083 the actual migration between the systems took place.

Web UI

• https://netbox.wikimedia.org/
• login using your LDAP/Wikitech credentials
• Currently you need an LDAP group membership in "ops" to be able to login.

Backups

The following paths are backed up in Bacula:

/srv/netbox-media/
/srv/postgres-backup/

A puppetized cron job (class postgresql::backup) automatically creates a daily dump file of all local Postgres databases (pg_dumpall) and stores it in /srv/postgres-backup.

This path is then backed up by Bacula.

For more details, the related subtask to setup backups was Phab:T190184.

Restore

To restore files from Bacula back to the client, use bconsole on helium and refer to Bacula#Restore_(aka_Panic_mode) for detailed steps.

To restore postgres databases from a dump file:

• unzip the latest dump file from /srv/postgres-backup
• sudo -u postgres /usr/bin/psql < psql-all-dbs-20180804.sql

Some more details from when restore was tested on Phab:T190184#4481629.

Dumping Database for Testing Purposes

The Netbox database contains a few bits of sensitive information, and if it is going to be used for testing purposes in WMCS it should be sanitized first.

1. Create a copy of the main database createdb netbox-sanitize && pg_dump netbox | psql netbox-sanitize
2. Run the below SQL code on netbox-sanitize database.
3. Dump and drop database pg_dump netbox-sanitize > netbox-sanitized.sql; dropdb netbox-sanitize

-- truncate secrets
TRUNCATE secrets_secret CASCADE;
TRUNCATE secrets_sessionkey CASCADE;
TRUNCATE secrets_userkey CASCADE;

-- sanitize dcim_serial
UPDATE dcim_device SET serial = concat('SERIAL', id::TEXT);

-- truncate user table
TRUNCATE auth_user CASCADE;

-- sanitize dcim_interface.mac_address
UPDATE dcim_interface SET mac_address = CONCAT(
                   LPAD(TO_HEX(FLOOR(random() * 255 + 1) :: INT)::TEXT, 2, '0'), ':',
                   LPAD(TO_HEX(FLOOR(random() * 255 + 1) :: INT)::TEXT, 2, '0'), ':',
                   LPAD(TO_HEX(FLOOR(random() * 255 + 1) :: INT)::TEXT, 2, '0'), ':',
                   LPAD(TO_HEX(FLOOR(random() * 255 + 1) :: INT)::TEXT, 2, '0'), ':',
                   LPAD(TO_HEX(FLOOR(random() * 255 + 1) :: INT)::TEXT, 2, '0'), ':',
                   LPAD(TO_HEX(FLOOR(random() * 255 + 1) :: INT)::TEXT, 2, '0')) :: macaddr;

-- sanitize cricuits_circuit.cid
UPDATE circuits_circuit SET cid = concat('CIRCUIT', id::TEXT);

Reports

Netbox reports are a way of validating data within Netbox. They are available in https://netbox.wikimedia.org/extras/reports/., and are defined in the repository https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/netbox-reports/.

In summary, reports produce a series of log lines that indicate some status connected to a machine, and may be either error, warning, or success. Log lines with no particular disposition for information purposes may also be emitted.

Report Conventions

Because of limitations to the UI for Netbox reports, certain conventions have emerged:

1. Reports should emit one log_error line for each failed item. If the item doesn't exist as a Netbox object, None may be passed in place of the first argument.
2. If any log_warning lines are produced, they should be grouped after the loop which produces log_error lines.
3. Reports should emit one log_success which contains a summary of successes, as the last log in the report.
4. Log messages referring to a single object should be formatted like <verb/condition> <noun/subobject>[: <explanatory extra information>]. Examples:
   1. malformed asset tag: WNF1212
   2. missing purchase date
5. Summary log messages should be formatted like <count> <verb/condition> <noun/subobject>