A brief departure from talking IPv6

There is a lot of news surrounding Net Neutrality, and potential repercussions of decisions made by courts, and some players out there that want to grab as much cash as they can, and claim it is in the best interest of their customers.

Netflix is just an example people love citing because it is bandwidth intensive, yet is not the entire story itself. Take a moment and understand how the Internet is pieced together. The Internet is a mass of interconnections between networks. These interconnections happen basically 1 of 3 ways:

transit: network A pays network B to reach every other network that isn’t A or B. Good networks usually get multiple transits for failover, and/or alternate paths to those other networks. You can buy multiple ports for bonding to increase capacity, etc. Average transit price without a Service Level Agreement (SLA, guaranteed connectivity or you can yell at us a lot and we credit you) is around $1-2/mbit, and with a SLA can hit upwards of $10/mbit. These are current avg. prices when buying 10G at a time of connectivity/capacity right now.

peering (settlement free, or “free”): Network A spends a bunch of money to get into popular (and less popular but regionally situated) Internet Exchange Points (IXP), and advertises its customers to other participants at no charge, usually over a series of network switches operated by that IXP. These networks pay for a number of switch ports and those ports’ capacity. This can also happen as a “private interconnect” where, at this mutual IXP, the networks can pay the monthly cost of running a fiber between them and exchange their customers’ routes that way. This means if you have 1G on the public exchange, but want 10G to a specific network, you pay the cheaper of either upgrading your exchange port to handle more capacity, or the private interconnect. The private interconnect is generally cheaper, as a single exchange 10G port can run around $7k/mo, versus a $350/mo for a fiber connection capable of 1/10/40/100G capacities (depending on either sides’ hardware capabilities and negotiated speed). You can also bond multiple ports to have loads more capacity, and still be cheaper than the cost of that single flat rate port on the exchange.

paid peering: Network A pays network B to only reach B’s customers/routes. Comcast, AT&T, etc., these are the players that usually play this game. “Oh you want to reach our users over a dedicated line, pay us $x as well as the cost of the connection.”

Netflix, since it appears to be everyone’s favorite example, has paid for at least 2 of the 3 (because almost no one in the networking world divulges if they have done the “paid peering” option, but it can be assumed in some cases these content companies do). So lets look at Netflix’s interconnection map. The thick lines are where reaching Netflix is most common, and assumed their paid transits. Lots of people traverse Level3 to reach Netflix, so their reliance on that would point to a paid transit. Thinner lines are the other ISPs that do not have as many networks behind them like Level3 might, so users don’t traverse them as much. We could assume those are peering connections at IXPs. Netflix has also deployed caching hardware at locations close to the Comcasts and AT&Ts that talk with players like Level3, to reduce latency and give you a better viewing experience.

So in order for Netflix to provide you with their service, they have now bought: transit, peering capacity, deployed de-centralized hardware to bring their content closer to you, and quite possibly already pays the Comcasts, etc. the paid peering costs. They already pay to put their content on the Internet, the whole Internet, and nothing but the Internet (with the usual regional restrictions to enforce copyright blah blah). You pay $60/mo (or more, or less, whatever) to reach that WHOLE INTERNET, and an additional $7 or whatever for the Netflix service.

But wait, now if Netflix doesn’t cough up more money, to not only your provider but any provider that decides to go this route, the quality of their service could be degraded ARTIFICIALLY. Alllll that capacity is still there, and never went away until the place you pay $60/mo to decided they want more money, and from services/destinations you pay them to deliver to you already. So Netflix eats the cost of doing business and you maybe now look at $11/mo because Netflix is out for profit too, without a doubt.

Now the kicker. Take away the name Netflix, and replace it with some other service you like. Actually, make it your project/next-great-idea. Hope you have the cash to pay to play.

That isn’t what the Internet should be, and for other parts of the world it won’t be. But with the US being a decent driving force behind changes that make their way throughout the Internet, it very well could be. Setting bad precedents here in the US, make it so others might follow suit elsewhere.

IPv6 OSPFv3 MTU Quirk with Brocade and Quagga

One of the things learned in my time at previous job, was the interoperability between IPv6 routing protocols and platforms. I ran into a quirk when deploying some Quagga boxes (0.99.x version) talking OSPFv3 with some Brocade routers running NetIron 5.x. The issue was that if MTU wasn’t specifically configured on the Brocade interface facing the Quagga box, OSPFv3 wouldn’t properly negotiate and was pretty much useless. However setting the MTU on the Brocade interface to 1500 fixed this. It is possible that this would also work with jumbo frame MTU settings, however the machines weren’t getting configured or deployed for that. Just something to look out for in the lab or production, if OSPFv3 appears configured correctly everywhere, but won’t establish. Quagga’s BGP had no issue with communicating properly with a Brocade router that didn’t specify MTU on its interface, it was only OSPFv3 that appeared to be affected.

Basic BGP configuration for IPv6

So you have your ASN and just got your PI (Provider-Independent Address Allocation) straight from an RIR, or perhaps a LoA (Letter of Authority) to announce some PA (Provider-Assigned Address Allocation) space from your ISP. You’ll need to start getting that announced out there to your peers, customers and transits. So lets use some documentation prefixes and ASNs and sort out a basic working config. I’ll base the examples on my experience with Brocade NetIron software on XMRs, which can translate over to Quagga or Cisco IOS with a few tweaks.

Assuming you are familiar with BGP4+ with IPv4, IPv6 is not so different or any more complex when getting started. Lets start off with some numbers:

Upstream ISP ASN: 64500
Your ASN: 64501
Specific /126 configured on interfaces out of allocated /64: 2001:db8:0:1::/126
PA allocation to announce: 2001:db8:1234::/48

Start off with making certain that you’ve configured IPv6 on your upstream facing interface, and they’ve configured your side, and you can ping each other over the link. The upstream provider’s configuration can be done as so:

isp#conf t
isp#(config)ipv6 prefix-list as64501-ipv6-filter seq 1 permit 2001:db8:1234::/48
isp#(config)router bgp
isp#(config-bgp)nei 2001:db8:0:1::2 remote-as 64501
isp#(config-bgp)nei 2001:db8:0:1::2 desc Customer_Name
isp#(config-bgp)no nei 2001:db8:0:1::2 activate
isp#(config-bgp)add ipv6 uni
isp#(config-bgp-af)nei 2001:db8:0:1::2 activate
isp#(config-bgp-af)nei 2001:db8:0:1::2 filter-list as64501-ipv6-filter in
isp#(config-bgp-af)exit
isp#wr mem

So the breakdown of these steps is:

  • enter configuration mode on router
  • build a filter to restrict what the customer (you) are allowed to announce (seq optional but required for multiple entries in list)
  • enter BGP configuration mode
  • create a specific session using target/destination IPv6 address and ASN of customer
  • optionally add a description of the session perhaps to track who it is more clearly
  • you don’t want IPv4 routes going out or coming in over the session, IPv6 routes only
  • change address-family for IPv6 specific BGP settings
  • activate sending and learning IPv6 routes over the session
  • apply the filter for accepting INBOUND routes from you/customer
  • exit & then write out the config

On the customer side it is similar but not exact:

you#conf t
you#(config)ipv6 prefix-list outbound-ipv6-filter seq 1 permit 2001:db8:1234::/48
you#(config)router bgp
you#(config-bgp)nei 2001:db8:0:1::1 remote-as 64500
you#(config-bgp)nei 2001:db8:0:1::1 desc ISP_Name
you#(config-bgp)no nei 2001:db8:0:1::1 activate
you#(config-bgp)add ipv6 uni
you#(config-bgp-af)network 2001:db8:1234::/48
you#(config-bgp-af)nei 2001:db8:0:1::1 activate
you#(config-bgp-af)nei 2001:db8:0:1::1 filter-list outbound-ipv6-filter out
you#(config-bgp-af)exit
you#wr mem

The differences being:
1) outbound filter list on your session to the ISP
2) network statement for the allocation to be announced by BGP

You’ll also need some sort of anchor route so BGP knows to announce the route. This can be either a local null-route or static-route for the covering prefix, or an IP out of the prefix configured on an interface. So once BGP is configured, and establishes between both routers, the ISP side should see something similar to:

isp#sh ipv6 bgp nei 2001:db8:0:1::2 routes
       There are 1 accepted routes from neighbor 2001:db8:0:1::2
Searching for matching routes, use ^C to quit...
Status A:AGGREGATE B:BEST b:NOT-INSTALLED-BEST C:CONFED_EBGP D:DAMPED
       E:EBGP H:HISTORY I:IBGP L:LOCAL M:MULTIPATH m:NOT-INSTALLED-MULTIPATH
       S:SUPPRESSED F:FILTERED s:STALE
       Prefix             Next Hop        Metric     LocPrf     Weight  Status
1      2001:db8:1234::/48  2001:db8:0:1::2
                                          1          140        0       E
         AS_PATH: 64501

And that should be it. You can obviously play around with peer-groups, route-maps, etc. Communities should work as long as your ISP offers them, so be sure to ask. Ideally there is at least a blackhole community in place. That should allow you to announce a specific range or IP that you want to have null-routed upstream in cases of abuse or attacks. That filter could look like either of the following, depending on how specific they allow you to announce:

ipv6 prefix-list as64501-ipv6-filter seq 1 permit 2001:db8:1234::/48 le 64

or

ipv6 prefix-list as64501-ipv6-filter seq 1 permit 2001:db8:1234::/48 le 128

With the obvious requirement that the BGP sessions would need to be configured as blackhole community enabled on both sides.