MRTG is great. I've been using it to keep an eye on our bandwidth for years now, monitoring per port traffic and aggregate outbound traffic through the router. It's always good to know your traffic just in case your IP carrier makes a "billing error".

As an admin, your life gets alot more complicated when your network starts to get over 100mbit/s. You have to toss out your cisco catalyst 2924 XLs and order something a little more exotic. When the time came for us, I opted to go with this sucker: the 2960G (WS-C2960G-24TC-L).

Now on to the nitty gritty. MRTG will bottom out at 114 mbit/s - there will be this nice line tracking across the top, then bam, it'll hit the bottom and everyone will freak out because they think the network is down. It's not a bug, it's counters rolling.

MRTG works by reading some SNMP counters every 5 minutes, then computing the difference between them, then infering the 5 minute avg data rate needed to get from the previous value to the current. It's like computing your speed by looking at the odometer in your car every 5 minutes. In the screenshot, the line ifInOctets.1 = Counter32: 425201421 is an example of one of those odometers. Below 114, the 32bit odometer MRTG looks at works fine, and when it occasionaly rolls over, MRTG is smart enough to pick it up and keep going. At 114 mbit/s, the 32bit odometer is rolling over in between samples, so MRTG has no idea how fast things are going - it has to assume things are going slow.

Luckily there is a solution, and it lies in the 64 bit counters, eg IF-MIB::ifHCOutOctets.5 = Counter64: 10508301908821
The HC is for high capacity, and MRTG is ready to deal with them. A target line in mrtg.cfg will look something like this:
Target[xx.xx.xxx.xxx_5]: 5:yo-mama@xx.xx.xxx.xxx:, but when you want it to look at the counter64 values, it needs to look like this:
Target[xx.xx.xxx.xxx_5]: 5:yo-mama@xx.xx.xxx.xxx:::::2

Now I had a hell of a time getting the counter64 values to even show up on my linux boxes. SNMP needs to be configured with --enable-mfd-rewrites. Gentoo docs reported that use flag mfd-rewrites would enable this, but the stable package net-snmp-5.2.1.2-r1 doesn't support this. Time to edit the ebuild file. This was a little scary for me since I had never done that, but it tunred out to be no big deal. I just opened /usr/portage/net-analyzer/net-snmp/net-snmp-5.2.1.2-r1.ebuild in an editor, added the line --enable-mfd-rewrites \
in the econf section, and rebuilt the digest by running
ebuild /usr/local/portage/net-analyzer/net-snmp/net-snmp-5.2.1.2-r1.ebuild digest