You are browsing a read-only backup copy of Wikitech. The live site can be found at


From Wikitech-static
Jump to navigation Jump to search

High level description on


Gather network level (Layer 4) traffic flows metadata to assist with traffic engineering and DoS mitigation.

How does it work?

Netflow diagram.
Netflow architecture

On the routers:

  • 1 out of 1000 flows crossing the routers' external interfaces (both inbound and outbound) gets its metadata sent to a configured collector once the flow timeout is reached (here 10s)
    • Example metadata are: source/dest IP/port/AS#, IP protocol, TCP flag...
  • The routers share their full BGP view with the collector

On the collectors:

  • Samplicator duplicates the IPFIX packets to Fastnetmon and Pmacct, while spoofing the source IP (so they still seem to come from the routers)
  • Pmacct (nfacct) extrapolates the flow size and packets based on the sampling rate (eg. do *1000)
  • Pmacct uses a prefix list (exported from Puppet) to enrich the collected flows with traffic direction
  • Pmacct uses the BGP data provided by the routers to enrich the collected flows metadata (adds peer src/dst AS#, AS path, src/dst AS#)
  • Pmacct uses an IP to location database to enrich the collected flows metadata (adds source and destination country)
  • Pmacct exports the enriched flow data to Druid via Kafka
  • Fastnetmon monitors inbound traffic for both known attack patterns and traffic level threshold and if any condition is met:
    • sends a notification email including a traffic signature if able
    • Triggers our monitoring system

How to deploy?

  1. Apply role::netinsights to a server
  2. Configure sampling on the router
  3. Add a BGP session from router to collector


Check if pmacct is sending data to kafka

$ kafkacat -b kafka-jumbo1001.eqiad.wmnet -t netflow -C

Real time Fastnetmon dashboard

$ fastnetmon_client

Check the logs

Both Pmacct and Fastnetmon log to syslog, grep for nfacctd or fastnetmon

Detected attack details are logged in /var/log/fastnetmon_attacks/



  • Fastnetmon misreports attack type and protocol - T241374


roll out sensible flow-table-sizes to Juniper core routers with sampling enabled - T248394