Exporting NetFlow from Linux to a collector over IPv6

There is another project out there in the ether that I have a hand in providing input for. One of the features that I felt was necessary for it is exporting NetFlow information from traffic the Linux machine handled, to a collector. This is dual-stack traffic, but I have the collector listening on IPv6.

Firstly, I needed something that would gather and export the data, so I found softflowd. My ubuntu server had it in the repo, so a quick apt install got it onto the machine easily enough. You need to edit /etc/default/softflowd and set what interface(s) you want it capturing & generating flow data from, and what options to feed to the daemon, like what server:port to export that data to:

INTERFACE="eth#"
OPTIONS="-v 9 -n [x:x:x:x::x]:9995"

Fill in the correct interface name you want to gather data from. The -v 9 option tells it to use Netflow v9, which has IPv6 support. The -n option is used for specifying the collector machine’s IP and port, so fill in for the correct IPv6 address of that collector. And that is the format for specifying an IPv6 host running a collector, like nfcapd. Then you can fire up the softflowd daemon, and you should start getting data sent to the collector:

Date flow start          Duration Proto                             Src IP Addr:Port                                 Dst IP Addr:Port   Packets    Bytes Flows
2015-02-13 23:18:13.316     0.001 UDP                              72.52.116.23:53    ->                            72.52.116.26:41933        1      213     1
2015-02-13 23:18:13.316     0.001 UDP                              72.52.116.26:41933 ->                            72.52.116.23:53           1       55     1
2015-02-13 23:15:17.715   180.139 ICMP6                         2001:470:1:9::1.0     ->                      2001:470:1:9::6666.0.0          4      288     1
2015-02-13 23:15:17.715   180.139 ICMP6                      2001:470:1:9::6666.0     ->                         2001:470:1:9::1.0.0          4      256     1
Summary: total flows: 75, total bytes: 291951, total packets: 1559, avg bps: 10006, avg pps: 6, avg bpp: 187
Time window: 2015-02-13 23:15:05 - 2015-02-13 23:18:58
Total flows processed: 75, Blocks skipped: 0, Bytes read: 5300
Sys: 0.008s flows/second: 9149.7     Wall: 0.006s flows/second: 12209.0

IPv6 in my streaming media? More likely than you think!

Not that I go out of my way to endorse one project/product over another, there is one that I have recently fallen in love with for streaming my media. Especially when it can use IPv6! So I needed a cross-platform solution for my streaming media needs. I was originally using XBMC, but only had it tied into the TV. I use several other computers and devices, in other locations outside of the house. So I read up on Plex. Got it installed with little to no effort, and could readily access my content where ever I was. I even tested this on my last trip to London, UK and was able to get a decent 1.2mbit/s stream from my house. Only issue was that it wasn’t using IPv6 in the app or accessing via plex.tv (server on that site only comes up with an IPv4 address).

So poking around I discovered 2 things: 1) I could access the Plex server directly at the IP/hostname of the server, and 2) there was a checkbox to enable IPv6!!

plex-ipv6

Simply browse to your Plex server, click on the settings icon (screwdriver + wrench), select Server, click on Networking and then “Show Advanced”. You’ll see the checkbox at the top, click it and save settings. Now you should be good to go! You can either access the server directly using the literal address inside []:32000:/web/ or the hostname:32000/web/

I still need to check if apps on Android are accessing over IPv6 or not, but for now I know a browser directly connecting to the Plex server does. A quick netstat shows us it is listening:

~# netstat -tnlp | grep -i plex
tcp 0 0 0.0.0.0:32469 0.0.0.0:* LISTEN 17642/Plex DLNA Ser
tcp 0 0 0.0.0.0:51107 0.0.0.0:* LISTEN 17606/Plex Plug-in
tcp 0 0 0.0.0.0:1541 0.0.0.0:* LISTEN 17642/Plex DLNA Ser
tcp6 0 0 :::32400 :::* LISTEN 17597/Plex Media Se
tcp6 0 0 :::32401 :::* LISTEN 17597/Plex Media Se

“Fun” with RFC4620 Section 6.4 and discovering IPv4 information over IPv6

As part of a request at work to figure out IPv4 addresses of devices on a network where broadcast pings don’t work, and no administrative access to the switches/routers, I took a look at solving this with IPv6. We know that you can ping6 the all-nodes multicast address, and get DUP! replies from IPv6 enabled hosts on that LAN segment. These will typically be link-local addresses, from which you can determine a MAC address. How to resolve that MAC address on a client host and not the router/switch, I was thinking reverse ARP or something, but support for that wasn’t present in my Ubuntu 13.10 kernel on the main machine I was working with. I started looking around for other options using IPv6 and found RFC4620, Section 6.4.

The gist of it is that you send an ICMPv6 Type 139 packet to an IPv6 address, asking if it has any IPv4 addresses configured either on that interface the target address is on, or any interfaces on the machine itself. And this is why this is disabled by default on hosts, and *IF* you insist on filtering ICMP6 Types, definitely make certain this is one of them. It works if you can enable it. I found a Git repo of ninfod, a userspace compiled binary, that needs to run as root for raw socket access. Compiled, ran with the flag to allow replies to Globally reachable hosts (I was testing between 2 remote sites, as my older VMs and their ping6 command didn’t support the -N flag), and gave it a test:

$ ping6 -N ipv4-all ipvsix.me
PING ipvsix.me(ipvsix.me) 56 data bytes
40 bytes from ipvsix.me: 127.0.0.1, 10.23.23.254, 72.52.116.26; ttl=63
40 bytes from ipvsix.me: 127.0.0.1, 10.23.23.254, 72.52.116.26; ttl=63
^C
--- ipvsix.me ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms

So while this works as intended, it makes absolute sense why it needs to be disabled by default, and even potentially filtered out completely. But at least it was fun to play with IPv6 to determine IPv4 information of remote hosts.

T-Mobile IPv6 in the US

Swapped from AT&T to T-Mobile in order to take advantage of their 4G/LTE IPv6 network. Since I dogfood IPv6 every chance I get, and the cost to swap saved me a whopping $0.50, I moved forward with it. I find that if I set their EPC.TMOBILE.COM APN to IPv4/IPv6, I don’t really see much in the way of dual-stack actually working on the phone. So I set it from the default IPv4 to just IPv6, and that got it working with native IPv6 and using CLAT+NAT64/DNS64 for IPv4 sites. Screenshot from my Galaxy Nexus running 4.3:

Screenshot_2013-10-11-15-51-06

IPv6 with A10 Load Balancers

So I got to do some honest IPv6 related work at the job the last 2 weeks. One task was to verify we had IPv6 working on the load balancers to hosts behind it. I was a bit wary of the state of IPv6 security on these A10 LBs, so I opted to keep the globally routed IPv6 space on the LB’s uplink interface, and the VIPs. And behind the scenes, use ULA.

Step 1: I generated a /48 of ULA for the location, and assigned a /64 for use on the VLAN that the inside interface of the LB sits on with the servers themselves.

Step 2: Configure ::1/64 on the LB inside vlan interface, and ::2/64 on a server, and verified that they could reach each other.

Step 3: I installed lighttpd on the server and configured it to listen on the ULA address.

Step 4: From my ARIN allocation, I have a /64 reserved for configuring /126s on device links to the router, so I configured it on the LB’s dedicated interface on the router. Using ::1/126 on the router; ::2/126 on the LB’s interface; ::3/126 as the VIP.

Step 5: Create on the LB an “IPv6 NAT Pool”, which is really a set of IPs that will act as source IP when talking to the webserver from the LB. I used ::3 through ::ff/64 of the same ULA space. The A10 LB only allows you to create a pool of 1000 IPs, so keep that in mind.

Step 6: Next you create the “Server” entry which is a description referencing an IPv6 address, in this case the ULA address of the web server. You also specify what IP services it will host that the LB can healthcheck, so I only set TCP 80.

Step 7: Then a “Service Group” needs to be created, and this is where you set what kind of LB algorithm and which servers will be used.

Step 8: Now a “Virtual Service” is defined that will tie in what service is forwarded to servers behind the LB, in this case HTTP on port 80, as well as what “NAT Pool” to use.

Step 9: Finally we create the “Virtual Server” (or VIP) with what globally routed IPv6 address you want to use, and what host/service will be used internally.

Now the above is just for getting IPv6 working through the unit. You can obviously attain dual-stack status by doing the same using IPv4. As well as actual load balancing when creating multiple “Server” entries and adding them to the “Service Group”.

Comcast filtering SMTP (TCP 25) for IPv6, but 587 still works!

Noticed this weekend that I couldn’t respond to emails on my personal hosted domains. I thought at first first they changed my PD prefix, but it was up to date in Postfix. Tried submission port and it worked just fine. So looks like Comcast finally caught up with “feature parity” in disallowing outbound SMTP connections on TCP 25.

My first received spam delivered over IPv6!

Not certain how much this actually counts as “Spam over IPV6” though. It was only the last bit of delivery to my account where IPv6 was involved. It still originated from IPv4.

 
Received from relay-6.dlfw.twtelecom.net ([2001:4870:6082:1::72]) by he.net for ; Tue, 13 Nov 2012 11:57:38 -0800

Received from localhost (unknown [127.0.0.1]) by relay-6.dlfw.twtelecom.net (Postfix) with ESMTP id 223346021E; Tue, 13 Nov 2012 12:47:42 -0700 (MST)

Received from relay-6.dlfw.twtelecom.net ([127.0.0.1]) by localhost (relay-6.dlfw.twtelecom.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TMxIEAmBj2TU; Tue, 13 Nov 2012 12:47:42 -0700 (MST)

Received from aol.com (unknown [209.234.184.51]) by relay-6.dlfw.twtelecom.net (Postfix) with SMTP id D73BD60094; Tue, 13 Nov 2012 12:47:32 -0700 (MST)

IPv6 and flows (using nfsen)

This will be about already having nfsen/nfdump configured, and are looking to just make a flow profile to graph IPv6 traffic from your routers. If you are looking to get nfsen iniitially configured, definitely follow their instructions on their site.

Say you have an sFlow capable router like…picking one totally not at random…..a Brocade XMR or MLX(e), and you want some basic flow data, especially IPv6. Depending on how many routers you are going to collect flow data from, will determine how beefy of a machine you will need. I know that at $lastjob, it was a hefty CPU (and definitely more than 1), tons of RAM, and hardware RAID. Right now, I’m using dual quad-core Xeon, tons of RAM and a small hardware RAID, but this machine serves many purposes. Right now I’m also only polling 4 MLX routers.

Go ahead and access your nfsen website, and on the Profiles pulldown, select “New Profile …”. In the creation dialog, give the profile whatever title you like; I went with the generic title of “IPv6”. If you want to add it to a group or make one for it, do as you please. I left that alone so I’d have it as a readily available profile to view. Select all the sources, or at least the ones you KNOW have IPv6 configured on as well as possible traffic (hosts, peers, transits, etc.). In the filter option, simply put: inet6

Save the profile, wait a while for data to get newly collected, and your graphs should start populating with delicious IPv6 flow information.

IPv6 speedtest comparison since moving to DOCSIS 3.x

Last time I ran any kind of flash/java based speedtest was back in Feb. when I was still on my tunnel and behind a DOCSIS 2.x modem:

Decided that I needed to re-run the test now that I’m both behind a DOCSIS 3.x modem, and have native IPv6 from Comcast:

Today is World IPv6 Launch Day!

After the success of last year’s World IPv6 Day, where success was measured with little to no problems reported, World IPv6 Launch Day has arrived! For a while major players like Google and Facebook had been white-listing their AAAA records to specific ISP recursive nameservers. This meant you had to query one of those in order to see IPv6 entries for their websites. Now that white-listing has been removed and all properly operating recursive nameservers will now serve up these records. They aren’t the only companies participating of course! Take a look at this list to see whom else signed up to promote their enabling of IPv6. Want some stats, take a look at the following:

But what does this mean to the non-techies out there? It is meant to be a passive change. You shouldn’t notice much in the way of interruptions. Perhaps your ISP will have a shorter route to a destination over IPv6, and you might get what you are trying to access a tiny bit quicker. If you like video/chat applications, this will soon mean that you’ll be able to have proper end-to-end communication between your machine/application and someone remotely. No more NAT to muddle connections. If XBOX and similar console devices migrate, you could actually have more than one in the household online at the same time and not get mangled because of IPv4 NAT!