Sunday, November 29, 2015

Quad, eh?

DNS is your IPv6 friend

by Craig Miller

IPv6  The Memory Test

Needs DNS
Old hands at IPv4 pride themselves on knowing the address of their servers, routers, name servers, etc. However with IPv6 and eight (8) groups of four (4) hexidecimal numbers as an address, this becomes more taxing on the memory. Of course an address like 2001:470:1d:583:fc48:b08e:c438:9e5d isn't impossible to remember, brighter minds than mine will be able to.

DNS to the rescue

As the internet grew in the early 80's it became apparent that having a service which would translate names to IP addresses would be extremely helpful. Humans, after all, are used to remembering names. In 1983, the first DNS (Domain Name Service, RFC 1034) was created, and BIND (Berkeley Internet Name Daemon) remains the standard for DNS today.

Internet Humour

DNS information is kept in a structured file, with name to address mapping in 'A' records. An example would look like this:
obake IN A
; IN HINFO "Intel Core I7 VM Host"

With the introduction of IPv6 for DNS (RFC 3596) a new type of line or record was added to the DNS structured file, an AAAA record, or Quad A record. 

Since an A record represented an IPv4 address (or 32 bit address) and an IPv6 is an 128 bit address, which is 4 time longer, it is an inside joke to make a DNS record 4 times longer, or a AAAA record.

Quad AAAA records

An example of a single A and Quad A record is:
obake IN A
obake IN AAAA 2001:470:1d:583:224:1dff:fed3:a117
; IN HINFO "Intel Core I7 VM Host"

As you can see it is much easier to type ping6 obake, rather than ping6 2001:470:1d:583:224:1dff:fed3:a117. And a lot easier to remember as well.

DNS as a transition tool

A really nice feature of DNS is that it will accept queries on both IPv4 and IPv6, returning either A or AAAA records. This means that if your host only makes DNS queries over IPv4, it can resolve IPv6 addresses (or if you prefer, query AAAA records).

Running a DNS service with A and AAAA records means any host, legacy or not, can resolve IPv6 addresses on your network. There is no cost beyond adding the AAAA records to your DNS. And suddenly, troubleshooting your network will get much easier.

A key difference from IPv4

DNS will typically not have reverse entries for IPv6. As I have mentioned in earlier posts, with SLAAC, and temporary (or private) addresses which change every 24 hours, your reverse entries in your DNS would have to be updated every day!

IPv6 is supposed to make your life easier, but updating reverse entries daily is not easier. So most network operators will have a few statically defined IPv6 reverse entries (usually for servers) and that is it.


Another solution to address management, DNS, and reverse entries is the use of IPAM (IP Address Management) software. Of course you could keep track of your address management in an excel spreadsheet, and many do. But when adding IPv6, you will find the IPAM software to free you from typos, versioning, and locked shared file problems. BlueCat and BT Diamond are a couple of the many IPAM vendors out there today.

Hammer Time

Remember, DNS is a tool to make your life easier. Sure you can use a rock to pound a nail, but a hammer is so much nicer. DNS will make ping6 hammer time.

* Quad, Eh? is not an internet joke, but a Canadian one

Monday, November 23, 2015


IPv6 Router Advertisements

by Craig Miller

IPv6  The Network is in control

Router Advertisement
The authors of IPv6 wanted to learn from the IPv4 mistakes, and one key issue (at the time) was statically assigned addresses (DHCP hadn't been invented yet). As I have written in a previous post, SLAAC (Stateless Address AutoConfig) was a huge step forward in end nodes being able to get an address auto-magically.

End station (host, nodes, machines, etc) addressing is controlled by the Router Advertisements (RAs). In this article, I will delve into RAs a bit more, and explain how one can configure RAs to give you the result you are looking for.

RA Bits

Not surprisingly, RAs are sent by routers. The RA is a Type 134 ICMPv6 message (IANA ICMPv6 assign numbers). The individual interfaces on your router must be configured to send RAs when IPv6 is enabled. The RA has several flags (bit fields) which can be set or unset. The important ones for addressing are the A, M, and O bits. These control how the end station will get an address. Jeff Carrell summarized it well in a 2012 presentation.
AutoConfiguration Options

I had this slide up on my cube wall for years, and gave a copy to anyone who seemed remotely interested in IPv6.

Simplifying the RA Bits

L bit

Let's start with the easy one. The L bit (or link bit) as you can see it is always on, and has no bearing on how an end station gets an address. 'Nuff said.

A bit

The next one to tackle is the A bit, this is the one that controls whether SLAAC is used by the end station.

M bit

The M bit is the Managed bit, which tells the end station to initiate DHCPv6 client request (RFC 3315). Unlike DHCPv4, IPv6 clients should NOT initiate a DHCPv6 request unless the M bit is set (not all clients respect this).

O bit

The Other bit, is a combination bit of sorts. The ideas is that SLAAC or manual may not provide all the information and end station may need, such as location of DNS and NTP servers. By setting the O bit, the network administrator is asking the end station to do a DHCPv6 for the options, but NOT for an address.

As you can see from Jeff's slide (above) the O bit is used with either the A bit (SLAAC provides the address) or the M bit (where DHCPv6  provides the address)

But wait, there's more 

RAs also advertise one or more prefixes for the link. Remember in What's with all those IPv6 Addresses end stations can have many IPv6 addresses. They can also have addresses in different subnets (on the same link). This would be useful for an address transition (from one address to another).

Since DNS is so useful (you were memorizing all those IPv6 addresses, right?), Recursive Domain Name Server Serivce (RDNSS) is useful (RFC 6106).

Lastly, for now, when the end station hear's the RA, and it assigns the default route next hop, as the link-local address of the router. By looking at the route table of the end station, you can see the default route:
$ ip -6 route
default via fe80::224:95ff:fef1:8ca dev mlan0  proto ra  metric 1024  expires 1608sec

There are other things in the RA, router lifetime, reachability, etc, which I may cover in a later post. But these are the important addressing bits.

RA Implementation Isues

This how it is supposed to work. The RFC's specify the ideal. But then it falls upon the developers to implement the ideal, and sometimes they fall short. When non-obvious combinations of RA bits (or flags) are set, non-obvious behaviours occur. Although the following RFC memo (it was only a draft, and has no number) is now expired, it still contains some good info about implementations of Windows 7, MacOS X 10.7 (aka Lion), and Ubuntu 12.04. It is well worth a read if you want to see how the OSs have really been implemented.

RA Troubleshooting

Want to see what your router is advertising?  You could fire up wireshark (or tcpdump) and wait, and wait, and wait for the router to send an RA (the configurable time is usually from every few minutes to much longer). Or you could issue a RS (Router Solicitation) and see the look at the RA that comes back.

The easiest way to do this is to use the linux utility rdisc6 (on Ubuntu, part of the ndisc6 package). This will do all the work, and show the results in a fairly human readable form.
$ rdisc6 eth0
Soliciting ff02::2 (ff02::2) on eth0...

Hop limit                 :           64 (      0x40)
Stateful address conf.    :           No   <--- M bit
Stateful other conf.      :           No   <--- O bit
Router preference         :       medium
Router lifetime           :         1800 (0x00000708) seconds
Reachable time            :  unspecified (0x00000000)
Retransmit time           :  unspecified (0x00000000)
 Source link-layer address: 00:24:95:F1:08:CA
 MTU                      :         1280 bytes (valid)
 Prefix                   : 2001:470:1d:584::/64
  Valid time              :         7200 (0x00001c20) seconds
  Pref. time              :         1800 (0x00000708) seconds
 Prefix                   : 2607:c000:815e:c400::/64
  Valid time              :         7200 (0x00001c20) seconds
  Pref. time              :         1800 (0x00000708) seconds
 Route                    : 2607:c000:815e:c400::/56
  Route preference        :       medium
  Route lifetime          :         7200 (0x00001c20) seconds
 Recursive DNS server     : 2001:470:1d:584::1
  DNS server lifetime     :         1800 (0x00000708) seconds
 from fe80::224:95ff:fef1:8ca

As you can see this router is advertising two prefixes into my network, and each of my hosts have five (5) IPv6 addresses (link-local, prefix 1 SLAAC, prefix 1 SLAAC/Temporary, prefix 2 SLAAC, prefix 2 SLAAC/Temporary).

Do not adjust your TV set, the RAs are in control

Knowing the RA bits, A, M, and O will help you control addressing on your IPv6 network. I have focused on the addressing components of the RA, but there is more  (home agent, reachability time, MTU size, etc), which can help you solve more challenging problems in your network. The RA is a useful tool to simplify your IPv6 network, hip-hip-hooRA for the RA.

Monday, November 16, 2015

Slaac-ing Off


by Craig Miller

IPv6  Auto Magic

Slaacer is a good thing
20 years ago when IPv6 was being standardized, the IPv4 world was busy statically defining addresses on every machine. This was becoming a burden to network administrators, and it was quickly becoming a problem of scale. DHCPv4 hadn't been invented yet.

Back in 1996, the IPv6 authors created a method by which devices on the network could auto-magically create a valid global routable address using SLAAC (StateLess Address Auto Config, RFC 4862). By using SLAAC, all hosts on a network could, without network administrators help, get on the internet.

Two halves of SLAAC

Because the simplification of subnet masking in IPv6 (everything is a /64, see Simplifying Subnetting) the address can be split in half, with the left (MSB) being the network portion of the address, and the right (LSB) being the host portion. 

Host Part

Let's start with the right half, the host portion. Since on a subnet, or broadcast domain (if you thinking IPv4), all MAC addresses must be unique, the IPv6 authors use this to create the host portion of the address. First the 48 bit MAC address is converted to a 64 bit number, or EUI-64, by inserting FFFE in the middle. For example a MAC address is: 60:21:c0:e0:8f:0a, would become: 6021:c0ff:fee0:8f0a. Then the last step in this transformation is to flip the universal/local bit (bit position 7, see RFC 2373 Appendix A) which results in: 6221:c0ff:fee0:8f0a. This host portion is not only used in the global routable SLAAC address, but also in the link-local address.

Network Part

The left half, or network portion is easier. When the network is defined on the router, the router will send out a router advertisement (RA) which will include the network prefix for that subnet. There is more to the RA, such as the M & O bits, but I'll cover that in a later post. Using tcpdump, a typical RA looks like:
15:12:21.640794 fe80::2a0:a512:85:2940 > ip6-allnodes: [icmp6 sum ok] ICMP6, router advertisement, length 48
    hop limit 64, Flags [none], pref medium, router lifetime 0s, reachable time 0s, retrans time 0s
    prefix info option (3), length 32 (4): 2607:c000:8000:8b93::/64, Flags [onlink, auto], valid time 2592000s, pref. time 604800s

On the Internet

The host, hears the RA, and forms a full 128 bit global routable address by prepending the advertised prefix to the host portion (lower 64 bits). And magically, the host is ready to use the internet!

OK, that was a bit simplified somewhat. There is DAD (Duplicate Address Detection, RFC 4862) which has to happen, but since as we have seen above, the IPv6 address is formed from the MAC address, it should be unique. If it isn't, DAD will figure that out, and create a unique host portion of the address, and then boom! you are on the internet.

Global Auto-configuration

As you can see a key difference from IPv4 is that a host can get a global address without user, or network administrator intervention. But you might ask, what about auto-configuration in IPv4, the address (RFC 3927)? This is different in two key ways:
  •  IPv6 SLAAC predates ipv4 auto-config, which was inspired by IPv6 SLAAC
  • It is not a global routable address, but used more on-link communication when a DHCPv4 server is not available

The Network is in charge

Sun Microsystems (now part of Oracle) used to have a marketing phrase, "The Network is the Computer." IPv6 creators, took this to heart, and much of IPv6 is controlled by the network, specifically, the RA (Router Advertisement). This architecture creates great power in the Network Administrator's hands. If a subnet must be renumbered, it is as simple as changing the prefix in the RA, all hosts will pick up the new prefix, and generate new global addresses.

SLAAC Evolving

SLAAC continues to evolve. Privacy concerns about using the same global address (without NAT) were raised, and SLAAC temporary addresses (RFC 4941) were created to address this issue, and is widely deployed in OSs. A newer standard which has yet to see wide deployment is attempting to address the privacy issue, while creating a stable (non-changing, and not tied to the MAC) address (RFC 7217), which will make it easier for firewall (ACL, Access Control Lists) configuration.

SLAAC Auto-magical

SLAAC auto-magic continues to make getting on the internet easy, nearly 20 years later. Address auto-magic is good, making your transition to IPv6 even easier.

Monday, November 9, 2015

Goodbye Broadcast

IPv6, Goodbye Broadcast

by Craig Miller

No Bullhorns

IPv6  does NOT use broadcast

The difference between Broadcast and Multicast is kind of like the difference between yelling at the top of your lungs in a large room, and tapping the shoulders of the people you want to talk to, and talking to them in a normal voice. Multicast allows a select group to listen to a message, rather than everyone hearing it, and most discarding it.

By the time  IPv6 was standardized, there was already had 20 years of experience with IPv4. And many of the short-comings of IPv4 were clearly apparent. A common one was broadcast storms, where there is a loop in your network, and broadcasts are allowed to circulate consuming all of your bandwidth, all of it.

So what is wrong with broadcasts, when they aren't storming? They steal cycles from your computer. Each broadcast must be processed by the stack, to determine if the packet is of importance, or should be discarded. Modern NIC cards are smart, but most of this is processing is still done in software. This slows down your computer.

The creators of IPv6 were looking for a better way than using broadcast to announce services, resolve MAC addresses, and tell everyone you are a printer by the corner cube. They decided to use Multicast. IANA has assigned IPv6 multicast addresses starting with FF.

Using Multicast Listener Discovery (MLDv2)

How does a device subscribe to a multicast group? In IPv6, the node uses MLDv2 (Multicast Listener Discovery version 2 RFC 3810). Think of it like an IPv6 version of IGMP (Internet Group Management Protocol RFC 2236). Listeners will announce that they are listening with a MLDv2 listener report. tcpdump show this as:
10:15:45.119950 IP6 fe80::2a0:a512:85:2940 > ff02::1: HBH ICMP6, multicast listener reportmax resp delay: 0 addr: ff02::1, length 24

As you can see the MLDv2 packet is sent to the same multicast address that it is listening on, FF02::1 in the above example. In this example, FF02::1  special multicast used for all IPv6 nodes. But there are other multicast groups. To see which your computer is listening to, run the following ip command:
~$ ip -6 maddr
1: lo
inet6 ff02::1
2: eth0
inet6 ff02::1:ff63:38b6
inet6 ff02::fb
inet6 ff02::1:ffc8:2ebf
inet6 ff02::1:ff54:3534 users 2
inet6 ff02::1

We have seen that FF02::1 is all nodes, and each interface will listen to that multicast address, including the loopback. But what are those other multicast addresses? You will remember that IPv6 uses ICMPv6 to resolve MAC addresses (see IPv6 using ICMP6), when a node needs to resolve a MAC address, it sends an ICMP6 packet to a special multicast address, the solicited node address, which is comprised of FF02, and the last 6 bytes of the IPv6 address (RFC 4291). 

Multiple IPv6 addresses results in Multiple Multicast Listener addresses

This machine has 3 of these addresses. A quick look at the addresses assigned to eth0 will explain a lot:
~$ ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2001:470:1d:583:d0d6:d79b:353:38b6/64 scope global temporary dynamic 
       valid_lft 7181sec preferred_lft 1781sec
    inet6 2001:470:1d:583:9:4d6c:adc8:2ebf/64 scope global temporary deprecated dynamic 
       valid_lft 7181sec preferred_lft 0sec
    inet6 2001:470:1d:583:129a:ddff:fe54:3534/64 scope global dynamic 
       valid_lft 7181sec preferred_lft 1781sec
    inet6 fe80::129a:ddff:fe54:3534/64 scope link 
       valid_lft forever preferred_lft forever

As stated in a previous post, IPv6 interfaces can have multiple addresses, and this is a fine example. Starting from the bottom up, there is the ever-present link-local address (always starts with FE80). The rest (going up) are all global addresses. 

The next up is a SLAAC (StateLess Address AutoConfig) formed from the MAC address. Next up is a temporary address, but it is deprecated. Temporary addresses by default last 24 hours, but what if you had connections using that address when the 24 hours rolls around? Rather than break the connection, it keeps the address in deprecated state (meaning it will not create new connections with this address), and the top address is the current temporary address.

But what of ff02::fb? This has a direct correlation to IPv4 which is used by mDNS (multicast Domain Name Service).

Multicast Scope

Another advantage of Multicast in IPv6 is the concept of scope, or how widely the multicast will propagate. The 4th digit in a multicast address, e.g. FF0n, represents scope, the following scopes are defined in RFC 4291 as:
1Interface local
8Organizational Local

Note: the examples on eth0 above are all of link-local scope (e.g. FF02::).

Multicast, it is just better than Broadcast

The advantage of Multicast is that only the listeners are disturbed by requests, rather than everyone. Thus saving all those hosts from wasting all those CPU cycles processing and discarding broadcasts. IPv6 is just easier on your network.

*no bullhorns is licensed under creative commons by LucidVisual

Monday, November 2, 2015

IPv6 using ICMP6

by Craig Miller

More than just echo reply

Using ICMP6

We are all familiar with the really useful network troubleshooting application, ping. Ping uses ICMP to determine if a remote host is alive.

ICMP (Internet Control and Management Protocol RFC 4443) is used more extensively in IPv6. It is not only used for echo request and echo reply (ping), but also for MAC address resolution (think ARP, but different).

Layer 3 in the OSI model

ICMP is a part of the IP protocol suite, but is rides on top of the IPv6 address header. Prior to sending a ping packet several things must happen. Both the Layer 3 (IP address) and the Layer 2 (MAC address) must be known. This is required for both IPv4 and IPv6. However, how the Layer 2 address is discovered is quite a bit different in IPv6, since there is no ARP (Address Resolution Protocol) in IPv6.

How to get a MAC address without ARP

In the scenario where Host A will ping Host B, ICMP6 is used to resolve a destination MAC address (L2) of Host B. This leaves a conundrum, of how do you resolve a L2 or MAC address when you need to build a packet all the way up to ICMP? This is where multicast comes in. All nodes must subscribe to the all nodes address FF02::1, which has a special MAC address 33:33:00:00:00:01 (RFC 2464). And individual nodes will also subscribe to a host specific multicast address ending with the last 4 bytes of their IP address, so that Host B, which has an IPv6 address ending in feb3:0f1a, would listen to multicast address FF02::1:ffb3:f1a

IPv6 uses ICMP6 type 135, Neighbour Solicitation and type 136 Neighbour Advertisement to resolve MAC addresses (RFC 4861). A solicitation (NS) will be sent to the all nodes multicast address, and the host with the desired IPv6 address will respond with a neighbour advertisement (NA) with will include its own MAC address.

The ICMP6  Neighbour Solicitation packet sent by Host A to resolve a MAC address of Host B will be look like:
DA=33:33:00:00:00:01 | DA=FF02::1:ffb3:f1a | ICMP type 135 | Host B IP address

Link-local address saves the day

But what about the source addresses? That is where link-local addresses (those which start with FE80::/10) come in. A neighbour solicitation (NS) uses its link-local address (a Layer 3 address) as the source address. Since link-local addresses are always present in IPv6 (they are auto-generated using the MAC address), it is used as the source IPv6 address. When the Host B responds with a neighbour advertisement (NA), it is sent to the unicast link-local address of the requesting host. The NA packet (again following the destination) will look like:
DA=Host A MAC | DA=Host A link-local address | ICMP type 136 | Host B MAC address

Tying it all together

Using tcpdump, it is easy to see the ICMP6 conversation:
~$ sudo tcpdump -i eth0 -e icmp6
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

09:50:17.799113 60:21:c0:d0:8f:0a (oui Unknown) > 33:33:ff:b3:0f:1a (oui Unknown), ethertype IPv6 (0x86dd), length 86: fe80::6221:c0ff:fed0:8f0a > ff02::1:ffb3:f1a: ICMP6, neighbor solicitation, who has fe80::211:24ff:feb3:f1a, length 32

09:50:17.803851 00:11:24:b3:0f:1a (oui Unknown) > 60:21:c0:d0:8f:0a (oui Unknown), ethertype IPv6 (0x86dd), length 86: fe80::211:24ff:feb3:f1a > fe80::6221:c0ff:fed0:8f0a: ICMP6, neighbor advertisement, tgt is fe80::211:24ff:feb3:f1a, length 32

09:50:17.803885 60:21:c0:d0:8f:0a (oui Unknown) > 00:11:24:b3:0f:1a (oui Unknown), ethertype IPv6 (0x86dd), length 118: fe80::6221:c0ff:fed0:8f0a > fe80::211:24ff:feb3:f1a: ICMP6, echo request, seq 1, length 64

09:50:17.805229 00:11:24:b3:0f:1a (oui Unknown) > 60:21:c0:d0:8f:0a (oui Unknown), ethertype IPv6 (0x86dd), length 118: fe80::211:24ff:feb3:f1a > fe80::6221:c0ff:fed0:8f0a: ICMP6, echo reply, seq 1, length 64

So where is Host B's MAC address? tcpdump is simplifying things too much, and using the -vv (for more protocol decode), the MAC address can be seen here:
10:06:29.292393 00:11:24:b3:0f:1a (oui Unknown) > 60:21:c0:d0:8f:0a (oui Unknown), ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::211:24ff:feb3:f1a > fe80::6221:c0ff:fed0:8f0a: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is fe80::211:24ff:feb3:f1a, Flags [solicited, override]
 destination link-address option (2), length 8 (1): 00:11:24:b3:0f:1a
   0x0000:  0011 24ce 0f1a

ICMP6 is more than just an echo reply

Since IPv6 does not have ARP, ICMP6 is used to resolve MAC addresses, and quite a few other things (ICMP Type numbers) which I'll cover in a later post. ICMP6 is an important part of the IPv6 protocol suite, without it, nothing would work in IPv6. Understanding how IPv6 and ICMP6 operate, will allow you to debug your network problems faster.

*ICMPv6 is used as a tcpdump or wireshark capture filter as 'ICMP6' hence I get lazy, and often drop the 'v'
**In Linux or Unix, one must use ping6 to initiate an IPv6 ping