You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

DNS/Netbox

From Wikitech-static
< DNS
Revision as of 18:27, 7 September 2020 by imported>Volans (Expanded FAQ section)
Jump to navigation Jump to search

Part of the DNS records and IP allocations are or will soon be automatically generated from data that resides in Netbox.

Infrastructure

  • IP allocation is done on Netbox.
  • Netbox data is exported via Netbox#DNS.
  • Netbox data is checked out on the authoritative DNS servers in /srv/git/netbox_dns_snippets.
  • When compiling the gdnsd final zones, the Netbox data is copied into /etc/gdnsd/zones/netbox for later inclusion.
  • In the actual zonefiles, within an $ORIGIN, the related snippet file is included using the $INCLUDE directive.

IP Allocation

The migration to the automated system requires that we move the allocation of IPs to Netbox that will gradually become the authoritative source of truth for IPAM.

Cutoff dates

  • [Wednesday June 24th 2020 10am UTC] All the management IP address allocation will be performed in Netbox from now on. Either via the Add interfaces and IPs to devices Netbox script for provisioning new devices or manually via the Add an IP Address button in the IP Addresses tab of any IP Prefix that will assign the first available IP in that subnet. The Offline a device with extra actions Netbox script takes instead care of the removal of interfaces and IPs when setting a device offline.
  • [TBD] All the primary IPv4 and IPv6 address allocation will be performed in Netbox from now on.

DNS records involved

  • Management forward (A) and reverse (PTR) records for both the hostname (foo.mgmt.eqiad.wmnet) and the asset tag (wmf1234.mgmt.eqiad.wmnet)
  • Primary IPv4 (A) and IPv6 (AAAA) and related reverse (PTR) records for the hostname (foo.eqiad.wmnet or foo.wikimedia.org)

Active

Management

  • ulsfo
  • eqsin
  • esams
  • frack in codfw
  • frack in eqiad
  • codfw
  • eqiad

Primary IPs

  • NONE

To be migrated

Management

  • NONE, all migrated

Primary IPs

  • ulsfo
  • eqsin
  • esams
  • frack in codfw
  • frack in eqiad
  • codfw
  • eqiad

Operations

Update generated records

To update the dynamically generated records based on the current Netbox data and deploy them to all the authoritative DNS servers, the sre.dns.netbox cookbook must be run. Please take a look to the above list of DCs already migrated to the new workflow, because you might not need to run this yet. See also Cookbooks#Cookbook_Operations. For example:

 sudo cookbook sre.dns.netbox -t T12345 "Add newly racked cp hosts in eqiad"

Convert an hardcoded $ORIGIN to Netbox

This is an example patch to convert an hardcoded $ORIGIN to the dynamically generated data.

Transition FAQ

Am I affected?

If your workflows will be affected by this change depends entirely on your interaction with the operations/dns repository:

  • I never read or contribute to this repository:
    • you're not affected and nothing will change for you. You can stop reading here.
  • I sometimes read or search for things in this repository:
    • you're marginally affected as the manual records will gradually disappear from the operations/dns repository to be replaced by the auto-generated files. You can clone the auto-generated repository to read or search in it following the instructions in Netbox#DNS. You can optionally read the rest of the document.
  • I contribute to the repository:
    • you're affected and should keep reading this FAQ section and the rest of the document.

What is changing

  • IP allocation that is currently done together with the DNS record definition manually in the DNS repository zone files is moving to Netbox that will be our IPAM tool. This transition will be done all at once to ensure consistency. Only Fundraising-tech (frack) records will be left out of this transition.
    • The cutoff date for all IP allocation to be moved to Netbox is Monday September 14th around 11:00am UTC.
    • All existing IPs except frack ones will be automatically imported into Netbox (a sneak peak can be found in netbox-next.wikimedia.org).
    • The changes in the Server Lifecycle procedure are outlined in the Server_Lifecycle/DNS_Transition page.
    • After that date all IPs except frack ones must be allocated in Netbox prior to assign them a DNS record in the DNS repository.
    • All new host's primary IPv4/IPv6 will be automatically assigned to them at provision time.
    • Additional IPs will require manual allocation in Netbox [see below]
  • The DNS records automatic generation (see above DNS/Netbox#Update_generated_records) generates all records present in Netbox, but they will be included in the DNS repository and hence in production on a per-$ORIGIN basis.
    • If a given $ORIGIN has been migrated to the automated zone file, updating Netbox and running the cookbook will change the DNS
    • If a given $ORIGIN has not been migrated to the automated zone file, a manual change to the DNS repository is still needed after the Netbox allocation.

How to allocate primary IPs for a server

See Server_Lifecycle/DNS_Transition#Provisioning_2

How to manually allocate a special purpose IP address in Netbox

This procedure is meant to be used only to create IPs in Netbox that are not attached to any device's interface because have special purposes like VIP addresses.

  1. Go to the VLANs page in Netbox
  2. Search for the correct VLAN based on datacenter, type, row (if applicable), etc.
  3. Click on the desired prefix (v4 or v6) in the Prefixes column for that VLAN
  4. Click on the IP Addresses tab in the prefix page
  5. Click on the Add an IP Address green button on the top-left, Netbox will automatically select the first available IP in that subnet
    • To create an IPv6 that is a mapped version of an existing IPv4, modify the Address field at the top to override the automatically selected address.
  6. Select the relevant Role (VIP, anycast, etc.)
  7. Set the DNS Name field with the FQDN to assign to this IP
  8. Select the Tenant if applicable (FR-Tech, RIPE, etc.)
  9. Click on the Create blue button at the bottom