Showing posts with label IPv6. Show all posts
Showing posts with label IPv6. Show all posts

Sunday, March 16, 2014

I've left Penn for a new job

After more than 20 years of working at Penn (University of Pennsylvania), I've decided to take a new job as Principal Research Scientist at Verisign Labs, the applied research division of Verisign Inc. You might know that Verisign is one of the world's largest DNS infrastructure providers. It runs the .com, .net, .edu, and .gov DNS top level domains, two of the thirteen DNS root servers (A and J), and performs the critically important root zone management function. Verisign also provides managed DNS, Distributed Denial of Service (DDoS) mitigation, and several other services. (Note: Verisign's certificate services business was sold off to Symantec several years ago).

I've been at Penn for so long, that I startled many of my colleagues with the news of my departure, so I'll say a few things about how this came about. I originally came to Penn in 1989 as an undergraduate. I became a full time IT staff member after I graduated - my first job was the system administrator of the new e-mail server for the School of Arts & Sciences, the largest school within Penn. I completed a Masters degree in Computer Science part-time while working. I later moved to the central IT organization, where I remained until last week, first as a Network Engineer, then as the Lead Engineer, and most recently as an Engineering Director. In addition, I've been a part time Adjunct Faculty in Penn's School of Engineering, teaching a laboratory course on Network Protocols.

I was approached by Allison Mankin and Matt Larson (I've known both of them for a while) about a year ago to see if I'd be interested in considering a research scientist position at Verisign Labs. Matt has since left Verisign to work at Dyn, and Allison is now a Director at Verisign Labs. At the time, I thought this was distinctly a long shot, but over the course of many months, I thought more seriously about the possibility. I visited Verisign Labs in September 2013, did an interview with them, and also met Burt Kaliski, Verisign CTO (Burt is a noted cryptographer and was the founding scientist of RSA Labs, and the developer of the PKCS standards). I kept in touch with Allison since, but it took me until the beginning of this year to finally come to the conclusion that I wanted to take this opportunity, so here I am. My first day at Verisign will be tomorrow (March 17th).

I've enjoyed my job and career at Penn a lot. I've been extensively involved in a very diverse range of technical projects, ranging from software development, systems engineering, and network design. Among others, I was responsible for the design and operation of much of the authentication and security infrastructure at Penn, as well as a variety of other services, like the DNS and DHCP (and many more that I don't have the time to enumerate here). I was the principal architect of IPv6 and DNSSEC deployment at Penn. I was the chief engineer of the MAGPI gigapop and through that role, was involved in R&E regional and national networking activities. Increasingly though, a lot of time is taken up by non-technical activities, and the longer I stay at Penn, the greater the possibility that I'll end up as a full time IT staff manager, which isn't the role I envision for myself. I've always seen my primary role as that of a technologist, and this job change will allow me to continue in that direction.

Verisign Labs appears to have a true applied research agenda, which I find appealing. Obviously DNS and DNSSEC research is an area in which I expect to be working. But there are many other interesting areas of work: routing, IPv6, reputation systems, security protocols, future internet architectures, etc. I'm also looking forward to attending more NANOG and IETF meetings to get myself more plugged into those communities than I've been able to thus far. Verisign Labs also has frequent, productive collaborations with computer science researchers through its university collobaration program.

In the process, I'm moving to the Washington DC metro area (specifically Reston, VA) where Verisign is located, another big change for me!

More later ..

--Shumon Huque.

Thursday, February 6, 2014

IPv6 versus IPv4 Performance

Yesterday from a post at the Deploy360 website, I learned of Comcast's IPv4 and IPv6 network speed testing tool:

        http://speedtest.comcast.net/

I did a quick test from my laptop in my office and got some very surprising results. The measured IPv6 performance was better than IPv4 by a gigantic margin. With IPv6, I got 822 Mbps download and 667Mbps download throughput. With IPv4, a mere 99Mbps upload and 18Mbps download!


Something seemed fishy, but I had to run off to other work, so I quickly posted the result to Twitter, planning to look into it later.


This generated quite a bit discussion with numerous folks on twitter and elsewhere. My initial speculation was that we do some rate limiting of IPv4 traffic at the Penn border routers for selected areas of the campus, and perhaps this was throttling the IPv4 performance. My other suspicion was that there was something significantly different in the IPv4 and IPv6 routing paths contributing to the difference. The graphic above does show a round-trip time difference of 63ms for the IPv4 path and 32ms for the IPv6 path, which suggests this. Furthermore, if the TCP window is not scaled properly to keep the pipe filled for this path at 63ms (but was for 32ms), then that would decrease throughput also - but not enough to account by itself for the observed difference.

Patrik Falstrom suspected a DPI device or other middlebox causing the problem. The only problem is that we don't have any such middleboxes (unless you consider an IP border router imposing IP address based rate limits a middlebox). In any case, I was leaning towards the rate limits as the cause myself, until I confirmed that those rate limits weren't being applied to any of the traffic from my office network. The rate limits are primarily targeted at the student residential dormitories - without them, our external links typically get overwhelmed with traffic to/from the dorms (most likely due to file sharing, a very common activity on college campuses). The border routers are configured to apply a token bucket rate policer to each individual IPv4 address within the network prefixes that cover the residential networks. Note that this rate limiting is completely application agnostic.  Also note that this scheme cannot scale to IPv6 (a single IPv6 subnet has more than 18 quintillion addresses!), a problem we're ignoring for the time being :-)

Repeat of the test


This morning, I decided to do another test (same laptop), but more carefully, and along with a packet capture. I also explicitly turned off the wireless interface (hmmmm) to make sure that all tests were using the wired gigabit ethernet interface. This time, I got much more reasonable looking results, both address families in the neighborhood of each other: IPv4 853Mbps down, 547Mbps up, and for IPv6 827Mbps down, 730Mbps up. One other difference I notice is that the roundtrip (ping) times to the destination server are 12ms for both IPv4 and IPv6. This is substantially different from yesterday's test (63 and 32ms respectively) despite the fact that I choose the same destination server at Comcast (Washington, DC).


A packet capture reveals that the destination server at Comcast for IPv4 was 68.87.73.52, and for IPv6 was 2001:558:1010:5:68:87:73:52. Are these the same endpoint? Hard to tell, but the fact that the last 4 fields of the IPv6 address spell out the IPv4 address in decimal might be a hint. The traffic streams use TCP port 5050. A traceroute to the IPv4 destination shows the outbound path takes one of Penn's commercial ISP links (Cogent) to New York and then back to Washington/VA. An IPv6 traceroute shows the outbound path goes out via our Internet2 link, the I2 commercial peering service, then Cogent (New York), Level3 (New York), and then Comcast to DC. So the IPv4 and IPv6 paths are substantially different in the forward direction. Harder to tell the path for the return traffic without the aid of some reverse traceroute tools or similar.

Getting a substantial fraction of a gigabit ethernet is not suprising - that's probably the bottleneck bandwidth along the measured path. My laptop has a gigabit ethernet connection to the building network, which in turn has dual 10 Gigabit Ethernet links to a 100 Gig campus core, and then multiple 10Gig links out to commercial ISPs/Internet2 etc. Most tier-1 ISP links and peerings are typically at least 10Gig.

The bandwidth-delay product on these paths is about 1,464 KB (1000Mbps * 12ms). The Comcast endpoint's receive window exceeds this, but my laptop's is slightly undersized, so I could probably do a bit of host tuning to boost the download numbers a bit more.

So, what's the explanation for the strange results I got yesterday? I wish had a packet capture to investigate, but my leading suspicion is that my laptop's wireless adapter (lower bandwidth, shared medium) was used in the IPv4 test, and the wired connection for the IPv6 one. If I have time later, I'll try to reproduce the issue.

--Shumon Huque



Addendum (February 9th 2014) - On closer inspection of the packet trace, the speed test appears to use multiple TCP streams in parallel, so scaling the window as high as the bw*delay product of the path isn't necessary.

Wednesday, January 22, 2014

An IPv6 Success Story (Galois)

The following article was contributed by Paul Heinlein, a systems administrator at Galois. Paul attended my full day IPv6 training course at USENIX LISA and just a couple of months later sent me a report of his successful deployment of IPv6. So I asked if he'd like to contribute an article on the topic.

It's great to hear of IPv6 success stories like this. (And of course, I'm glad folks are finding my courses useful). Significantly, his network is already seeing a very substantial amount of IPv6 traffic!

(A note: the version of iptables in Redhat Enterprise Linux6/CentOS 6 has fixed the stateful IPv6 inspection capability. We use it successfully at Penn).

You can find the slides from my IPv6 training course on my website.

-Shumon Huque



An IPv6 Success Story

Paul Heinlein


Galois is a software-engineering firm located in Portland, OR that specializes in the hard problems of computing trust and assurance.

One of our IT goals for 2014 is enabling IPv6 on all our current IPv4 subnets. I was pleased to see Shumon's full-day IPv6 tutorial on the LISA '13 schedule. I hoped he could fill the gaps in my limited
experience with IPv6 and provide me the knowledge I'd need to configure our network hardware and applications.

Full-day technical tutorials can sometimes test my patience, but Shumon's presentation moved quickly while remaining clear. I came away from the day thinking that I had enough basic knowledge to plan and implement our IPv6 rollout.

Returning to Portland, I had to upgrade upgrade a couple core switches and apply to our ISP for a netblock prior to enabling it on our network, but once the hardware was in place, everything went reasonably well.

Shumon's overview had done exactly what I'd hoped it would: allow me to interpret the various bits of vendor-specific and application-specific documentation and assemble a reasonable roll-out plan.

Our DMZ and client networks are now fully IPv6 enabled. (Internal development networks will take a bit more time due to VPN-related complexity.)

Along the way, I learned a couple things Shumon didn't explicitly cover in his presentation.

First, make sure that reverse DNS pointers are in place for any mail server before enabling IPv6. gmail (among others) will reject messages from any mail server without reverse DNS pointers in place.

Second, the ip6tables that ships with RHEL/CentOS 5 has a limited ability to do stateful packet inspection. The generic rule allowing packets from established or related sessions does not work:

   # this is broken in RHEL/CentOS 5
   -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

(We've only got one CentOS 5 machine in our DMZ, so this issue hasn't impacted us in any major way.)

Third, it was a lot of fun! Nearly all our DMZ services (NTP, DNS, Jabber, e-mail, www, ssh, host-based firewalls) needed updated configurations, so I learned a bunch. The day after I did the initial rollout, one of our engineers came into the office, turned off IPv4 on his Mac, and tested what percentage of Google result links he could follow. (He thought about 5%, though he said the queries he chose probably resulted in a higher success rate than might be normal.)

Finally, I've been mildly surprised at the quantity of IPv6 packets traversing our border firewall. Over the past week, IPv6 has comprised 38% of overall inbound traffic and 11% of outbound. The numbers are similar when scoped to the past month.

I'd like to express my thanks to Shumon for the talk and the LISA organizers for putting it on the schedule!

  -- Paul Heinlein <heinlein@galois.com>

Friday, January 17, 2014

World IPv6 Launch Measurements

From the Internet Society's most recent (January 16th 2014) round of IPv6 measurements, here again are the top 100 lists ranked by the two measures of (1) percentage of requests that used IPv6, and (2) total volume of IPv6 requests.

My past articles on this topic:
  * Measurements from December 12th 2013
  * Measurements from  September 17th 2013
  * Measurements from June 6th 2012

ISOC's blog post highlights that Deutsche Telekom, a large German telecommunications carrier, (of which T-Mobile US is a subsidiary*) has been experiencing very substantial IPv6 traffic growth. They've reached a figure of 15.5% of requests composed of IPv6 (to the select set of content providers providing measurement data). In terms of total volume of IPv6 requests, they are in 6th place behind five other large ISPs (See the second ranked list below). Comcast still leads the pack, followed by AT&T, KDDI, Free, and Verizon Wireless.

(*Note: T-Mobile's traffic is counted separately in the measurements from Deutsche Telekom)

Interestingly, while Verizon Wireless has a substantial IPv6 deployment, Verizon FIOS (their fiber-optic landline network) has made no visible progress in IPv6 deployment. Although many people, myself included, have noted this lack of deployment over the years, they were just recently taken sharply to task by folks on the NANOG (North American Network Operators Group) mailing list.

Looking at the rankings by proportion of IPv6 requests from each network, there is a new entry at the top - NYSERNet (a regional R&E network in New York State), coming in at a staggering 99.95%. On the Internet2 IPv6 working group list, I asked Bill Owens of NYSERNet if he wanted to comment. He claims that we shouldn't be too impressed because this is just the small NYSERNet office network and the backbone, with a relatively small population of clients. Regardless, I think it's a noteworthy achievement. As I've mentioned before, a number of universities and other educational organizations still show up prominently in this ranking. Two small american colleges, Gustavus Adolphus in Minnesota at 74.22% and Marist College in upstate New York at 65.77%! And Panamerican University, in Mexico City is at 66.22%.

Penn is slowly inching up at 45%. We'll have IPv6 deployed more extensively on our wireless network pretty soon, at which point I expect our numbers to go up noticeably.


World IPv6 Launch Measurements, by % IPv6 requests:
---------------------------------------------------

   1 NYSERNet 99.95%
   2 SpeedPartner GmbH 95.83%
   3 TOP-IX Consortium 86.99%
   4 Fundacao Parque Tecnologico Itaipu - Brasil 79.92%
   5 DirectVPS 78.63%
   6 interscholz Internet Services GmbH & Co. KG 75.49%
   7 Gustavus Adolphus College 74.22%
   8 Google Fiber 71.87%
   9 Universidad Panamericana 66.22%
  10 Marist College 65.77%
  11 University of Vermont 65.13%
  12 Virginia Tech 65.11%
  13 mur.at - Verein zur Frderung von Netzkwerkkunst 64.97%
  14 Association tetaneutral.net 55.67%
  15 Utility Line Italia srl 53.40%
  16 ThaiSarn 53.23%
  17 DegNet GmbH 52.84%
  18 Trunk Networks Limited 52.60%
  19 Ji?? ?trohalm 52.20%
  20 Alhambra Eidos 52.12%
  21 Louisiana State University 51.93%
  22 Ponto de Presen?a da RNP na Bahia 51.30%
  23 SuperInternet Access Pte Ltd 50.45%
  24 University of New Hampshire 50.44%
  25 Region 7 ESC 50.24%
  26 PREGINET 50.20%
  27 CICA/Junta de Andaluc?a 50.05%
  28 Tanzania Network Information Center 50.00%
  29 Karlsruhe Institute of Technology (KIT) 49.03%
  30 Maxiweb Internet Provider 48.76%
  31 Critical Colocation 48.23%
  32 University of Pennsylvania 45.38%
  33 SPAWAR 44.15%
  34 AMS-IX 42.99%
  35 University of Iowa 42.81%
  36 Kasetsart University 41.99%
  37 DreamHost 41.79%
  38 University of Minnesota 40.80%
  39 Rensselaer Polytechnic Institute 40.74%
  40 Bulgaria NREN 40.57%
  41 Verizon Wireless 40.03%
  42 CZ.NIC 39.02%
  43 guifi.net 37.86%
  44 Hughes Network Systems 35.60%
  45 Sauk Valley Community College 35.52%
  46 Tulane University 34.96%
  47 Free 34.28%
  48 ARNES 33.73%
  49 Greek Research & Technology Network 33.14%
  50 Leibniz Supercomputing Centre 32.71%
  51 Host Virtual, Inc 31.74%
  52 Netwerkvereniging Coloclue 28.41%
  53 Opera Software ASA 27.54%
  54 UNESP 27.26%
  55 UNINETT 27.13%
  56 FCCN 26.00%
  57 VOO 24.12%
  58 RCS & RDS 23.80%
  59 mc.net 23.37%
  60 UFSCar 22.96%
  61 EPT Luxembourg 22.43%
  62 FranTech Solutions 22.03%
  63 AAISP 21.22%
  64 Comcast 20.61%
  65 Chubu Telecommunications 18.90%
  66 Swisscom 18.66%
  67 XS4ALL 17.66%
  68 UFSC - Universidade Federal de Santa Catarina - Brazil 17.50%
  69 LENTEL 16.73%
  70 StarHub 16.28%
  71 Deutsche Telekom AG 15.50%
  72 NIIF/Hungarnet 14.56%
  73 LITNET 14.50%
  74 Red Acad?mica de Centros de Investigaci?n y Universidades Nacionales REACCIUN 13.85%
  75 Indiana University 13.68%
  76 ATT 13.53%
  77 SARENET 13.13%
  78 DMZGlobal 12.97%
  79 NetAssist 12.89%
  80 Academia Sinica Network 12.14%
  81 Cisco 11.95%
  82 Solcon 11.46%
  83 CESNET 11.04%
  84 Funet 10.94%
  85 PT. Wifian Solution 10.19%
  86 FidoNet 10.12%
  87 prgmr.com 9.89%
  88 T-Mobile USA 9.58%
  89 Monash University 9.28%
  90 OVH 9.26%
  91 KDDI 8.94%
  92 Spectrum Networks 8.30%
  93 AMRES - Serbian National Research and Education Network 7.84%
  94 Belnet 7.65%
  95 SWITCH 7.62%
  96 Hurricane Electric 7.58%
  97 M1 Limited 7.22%
  98 RedIRIS 6.96%
  99 Init7 6.93%
 100 CJSC Progressive Technologies 6.71%


World IPv6 Launch Measurements, by volume of IPv6:
--------------------------------------------------


   1 Comcast 20.61%
   2 ATT 13.53%
   3 KDDI 8.94%
   4 Free 34.28%
   5 Verizon Wireless 40.03%
   6 Deutsche Telekom AG 15.50%
   7 Time Warner Cable 3.88%
   8 RCS & RDS 23.80%
   9 Liberty Global 2.43%
  10 Swisscom 18.66%
  11 Telefonica del Peru 4.60%
  12 Hughes Network Systems 35.60%
  13 SoftBank BB 1.46%
  14 Chubu Telecommunications 18.90%
  15 Opera Software ASA 27.54%
  16 VOO 24.12%
  17 XS4ALL 17.66%
  18 StarHub 16.28%
  19 T-Mobile USA 9.58%
  20 Google Fiber 71.87%
  21 China Telecom 0.23%
  22 Forthnet 2.79%
  23 M1 Limited 7.22%
  24 Internode 3.94%
  25 EPT Luxembourg 22.43%
  26 Janet 3.81%
  27 its communications Inc.(iTSCOM) 2.64%
  28 CESNET 11.04%
  29 MediaCat Div./Community Netowork Center Inc. 6.58%
  30 Kasetsart University 41.99%
  31 NTT Communications 4.51%
  32 Belnet 7.65%
  33 ARNES 33.73%
  34 Leibniz Supercomputing Centre 32.71%
  35 LITNET 14.50%
  36 NIIF/Hungarnet 14.56%
  37 OVH 9.26%
  38 Cisco 11.95%
  39 BelWue 5.71%
  40 FCCN 26.00%
  41 UNINETT 27.13%
  42 Altibox AS 0.90%
  43 Monash University 9.28%
  44 SWITCH 7.62%
  45 TUBITAK ULAKBIM / ULAKNET 1.50%
  46 Australian Academic and Research Network (AARNet) 2.00%
  47 Indiana University 13.68%
  48 University of Minnesota 40.80%
  49 AAISP 21.22%
  50 UniNet 3.36%
  51 Xfone 018 0.96%
  52 Ji?? ?trohalm 52.20%
  53 Voxel / Internap 3.51%
  54 GITN Sdn Berhad 2.69%
  55 Louisiana State University 51.93%
  56 University of Pennsylvania 45.38%
  57 SuperCSI 1.69%
  58 JARING Communications Sdn Bhd 0.38%
  59 Solcon 11.46%
  60 CJSC Progressive Technologies 6.71%
  61 Starlink 1.88%
  62 inexio KGaA 2.44%
  63 Virginia Tech 65.11%
  64 green.ch AG 3.26%
  65 Hurricane Electric 7.58%
  66 Dhiraagu 0.60%
  67 Gustavus Adolphus College 74.22%
  68 LENTEL 16.73%
  69 DMZGlobal 12.97%
  70 DegNet GmbH 52.84%
  71 RedIRIS 6.96%
  72 Academia Sinica Network 12.14%
  73 GlobalConnect 1.44%
  74 University of Iowa 42.81%
  75 Funet 10.94%
  76 University of Wisconsin - Madison 6.16%
  77 Storm Internet 3.76%
  78 AMRES - Serbian National Research and Education Network 7.84%
  79 National Informatics Centre 2.32%
  80 SoftLayer Technologies 0.84%
  81 RENATER 4.57%
  82 Karlsruhe Institute of Technology (KIT) 49.03%
  83 guifi.net 37.86%
  84 Init7 6.93%
  85 Host Virtual, Inc 31.74%
  86 Rensselaer Polytechnic Institute 40.74%
  87 Choopa, LLC 0.55%
  88 NetAssist 12.89%
  89 Tulane University 34.96%
  90 DreamHost 41.79%
  91 Bulgaria NREN 40.57%
  92 FranTech Solutions 22.03%
  93 Greek Student Network 0.47%
  94 RESTENA 5.33%
  95 UNESP 27.26%
  96 iway AG 4.06%
  97 FX Networks 0.49%
  98 Tanzania Network Information Center 50.00%
  99 edpnet 1.11%
 100 ADDIX Internet Services 3.05%


Thursday, December 19, 2013

World IPv6 Launch Measurements

The Internet Society has posted their latest IPv6 measurements (December 12th 2013). Read the section titled "Notes on network operator measurements" to understand how the measurements are being made and which content providers (Google, Facebook, Yahoo!, Akamai) are providing the data.

I've pulled out some of the data, and put together ranked (top 100) lists of networks by two measures: (1) percentage of requests that used IPv6, and (2) total volume of IPv6 requests. As I've done a few times in the past, I'm going to continue periodically writing these entries to have a snapshots in time of IPv6 deployment progress.

Many networks are posting some pretty impressive numbers for IPv6 usage. For the leading network in the %v6 category (the 1st list below), TOP-IX Consortium, an Italian Internet Exchange point has 86% of their requests to the participating content providers using IPv6! Several universities in the US R&E community are doing well too, Gustavus Adolphus College at 74%, Virginia Tech at 62%, University of New Hampshire at 51% among them. The University of Pennsylvania (my own institution) is posting a respectable 40% - we'll have IPv6 fully deployed on our wireless network in early 2014, at which time our numbers should go up substantially. There's an interesting story about why Penn hasn't had IPv6 on its wireless network for so long - I'm planning to write a separate article on that topic in the near future.

In the total volume category (the 2nd list below) Comcast now leads. John Brzozowski, Comcast's chief IPv6 architect, has written a more detailed article on their leadership position in IPv6 deployment. They are followed by several other ISPs: AT&T (US), KDDI (Japan), Free (France), Verizon Wireless, (US) Deutsche Telekom (Germany), RCS & RDS (Romania), Time Warner Cable (US).


World IPv6 Launch Measurements, by % IPv6 requests:
---------------------------------------------------

   1 TOP-IX Consortium    86.27%
   2 Fundacao Parque Tecnologico Itaipu - Brasil    79.65%
   3 DirectVPS         77.66%
   4 ThaiSarn         76.62%
   5 Gustavus Adolphus College   17234    74.17%
   6 Google Fiber      70.22%
   7 Universidad Panamericana    13679    66.84%
   8 mur.at - Verein zur Frderung von Netzkwerkkunst    66.73%
   9 interscholz Internet Services GmbH & Co. KG     66.20%
  10 Virginia Tech     61.69%
  11 Trunk Networks Limited     56.99%
  12 Association tetaneutral.net    56.41%
  13 DegNet GmbH     51.91%
  14 Ponto de PresenC'a da RNP na Bahia     51.75%
  15 Critical Colocation   51.38%
  16 Jiri strohalm    51.29%
  17 SPAWAR     51.12%
  18 ITsjefen AS     51.08%
  19 SuperInternet Access Pte Ltd     50.88%
  20 University of New Hampshire     50.57%
  21 AIMES Grid Services CIC     50.38%
  22 Region 7 ESC     50.27%
  23 PREGINET 50.24%
  24 Maxiweb Internet Provider    50.20%
  25 CICA/Junta de AndalucC-a    50.04%
  26 DreamHost    49.41%
  27 Alhambra Eidos    49.32%
  28 Karlsruhe Institute of Technology (KIT)    48.85%
  29 Sauk Valley Community College     46.13%
  30 University of Minnesota           45.87%
  31 Marist College     45.83%
  32 AMS-IX     44.09%
  33 University of Iowa     42.90%
  34 Kasetsart University     41.31%
  35 Verizon Wireless     40.40%
  36 University of Pennsylvania     40.06%
  37 Bulgaria NREN     38.74%
  38 guifi.net     38.69%
  39 Rensselaer Polytechnic Institute    37.91%
  40 Louisiana State University     35.74%
  41 Leibniz Supercomputing Centre     35.17%
  42 Netwerkvereniging Coloclue     32.66%
  43 Free       31.03%
  44 UNESP       30.15%
  45 NetAssist       29.72%
  46 ARNES       28.98%
  47 Tulane University     28.88%
  48 Utility Line Italia srl     28.19%
  49 Host Virtual, Inc     27.79%
  50 Hughes Network Systems     27.28%
  51 FCCN   26.96%
  52 UFSCar     26.36%
  53 VOO     25.94%
  54 Opera Software ASA     25.69%
  55 Greek Research & Technology Network    24.94%
  56 UNINETT        24.58%
  57 LENTEL         23.48%
  58 Chubu Telecommunications    22.76%
  59 RCS & RDS     22.01%
  60 mc.net     20.20%
  61 Comcast    20.15%
  62 Monash University     19.82%
  63 Swisscom     19.64%
  64 UFSC - Universidade Federal de Santa Catarina - Brazil    19.25%
  65 XS4ALL 18.52%
  66 AAISP  18.13%
  67 SIDN   17.11%
  68 EPT Luxembourg    16.72%
  69 manitu GmbH    15.88%
  70 VentraIP Group (Australia) Pty Ltd    15.73%
  71 LITNET   15.09%
  72 Hurricane Electric    14.88%
  73 ATT     14.82%
  74 NIIF/Hungarnet    14.43%
  75 Funet        12.60%
  76 CESNET        12.33%
  77 Deutsche Telekom AG    12.28%
  78 Indiana University        12.14%
  79 OVH           11.63%
  80 FranTech Solutions    10.98%
  81 Red Academica de Centros de InvestigaciC3n y Universidades Nacionales REACCIUN    10.95%
  82 UniNet       10.48%
  83 Host.MD      10.35%
  84 Belnet       9.73%
  85 Cisco       9.70%
  86 CJSC Progressive Technologies    9.65%
  87 KDDI     8.87%
  88 Academia Sinica Network     8.22%
  89 SARENET  8.01%
  90 DMZGlobal     7.99%
  91 SWITCH     7.86%
  92 Init7     7.70%
  93 MediaCat Div./Community Netowork Center Inc.    7.52%
  94 AMRES - Serbian National Research and Education Network    7.10%
  95 RedIRIS      6.74%
  96 T-Mobile USA      6.49%
  97 Voxel / Internap     6.48%
  98 Defense Research and Engineering Network    6.41%
  99 prgmr.com      6.35%
 100 M1 Limited      6.29%


World IPv6 Launch Measurements, by volume of IPv6:
--------------------------------------------------

   1 Comcast    20.15%
   2 ATT    14.82%
   3 KDDI    8.87%
   4 Free     31.03%
   5 Verizon Wireless     40.40%
   6 Deutsche Telekom AG    12.28%
   7 RCS & RDS          22.01%
   8 Time Warner Cable     4.07%
   9 Liberty Global    2.52%
  10 Telefonica del Peru    5.14%
  11 Swisscom    19.64%
  12 SoftBank BB     1.65%
  13 Hughes Network Systems     27.28%
  14 Chubu Telecommunications     22.76%
  15 Opera Software ASA     25.69%
  16 VOO   25.94%
  17 XS4ALL     18.52%
  18 China Telecom    0.18%
  19 Janet         4.29%
  20 T-Mobile USA    6.49%
  21 Forthnet         3.35%
  22 StarHub        4.81%
  23 University of Minnesota     45.87%
  24 Indiana University        12.14%
  25 CESNET  12.33%
  26 Google Fiber     70.22%
  27 M1 Limited     6.29%
  28 Virginia Tech    61.69%
  29 Internode        4.53%
  30 FCCN        26.96%
  31 EPT Luxembourg     16.72%
  32 Cisco        9.70%
  33 Belnet        9.73%
  34 Louisiana State University     35.74%
  35 RedIRIS   6.74%
  36 UNINETT   24.58%
  37 Leibniz Supercomputing Centre    35.17%
  38 its communications Inc.(iTSCOM)     3.14%
  39 SWITCH     7.86%
  40 NIIF/Hungarnet     14.43%
  41 ARNES     28.98%
  42 MediaCat Div./Community Netowork Center Inc.    7.52%
  43 BelWue     5.87%
  44 LITNET     15.09%
  45 University of Pennsylvania    40.06%
  46 NTT Communications     4.38%
  47 RENATER     4.10%
  48 Kasetsart University     41.31%
  49 University of Iowa     42.90%
  50 TUBITAK ULAKBIM / ULAKNET     1.29%
  51 OVH     11.63%
  52 Tulane University    28.88%
  53 Monash University     19.82%
  54 UNESP  53166     30.15%
  55 AMRES - Serbian National Research and Education Network    7.10%
  56 Rensselaer Polytechnic Institute  37.91%
  57 UFSC - Universidade Federal de Santa Catarina - Brazil    19.25%
  58 University of Wisconsin - Madison      4.84%
  59 Gustavus Adolphus College 74.17%
  60 Funet       12.60%
  61 GARR       1.25%
  62 Marist College     45.83%
  63 SPAWAR     51.12%
  64 SURFnet     1.36%
  65 SuperCSI     2.71%
  66 Altibox AS     0.70%
  67 AAISP       18.13%
  68 Australian Academic and Research Network (AARNet)    2.32%
  69 Karlsruhe Institute of Technology (KIT)  48.85%
  70 Xfone 018       1.11%
  71 UFSCar       26.36%
  72 Voxel / Internap    6.48%
  73 Solcon         5.12%
  74 UniNet         10.48%
  75 CJSC Progressive Technologies    9.65%
  76 CICA/Junta de AndalucC-a     50.04%
  77 Starlink    1.47%
  78 Hurricane Electric     14.88%
  79 Greek Research & Technology Network     24.94%
  80 Jiri strohalm  51.29%
  81 JARING Communications Sdn Bhd    0.27%
  82 Defense Research and Engineering Network    6.41%
  83 GITN Sdn Berhad  3.11%
  84 Dhiraagu         0.85%
  85 Academia Sinica Network    8.22%
  86 green.ch AG         2.83%
  87 Louisiana Optical Network Initiative    5.04%
  88 LENTEL    23.48%
  89 Fundacao Parque Tecnologico Itaipu - Brasil    79.65%
  90 DegNet GmbH         51.91%
  91 National Informatics Centre    2.19%
  92 guifi.net     38.69%
  93 GlobalConnect     1.54%
  94 The Tertiary Education and Research Network of South Africa (TENET)    1.53%
  95 DMZGlobal      7.99%
  96 Init7      7.70%
  97 SoftLayer Technologies    1.40%
  98 Ponto de PresenC'a da RNP na Bahia    51.75%
  99 Storm Internet    5.40%
 100 inexio KGaA    1.15%


Sunday, December 1, 2013

EDU Top Level Domain statistics

Some DNS Top Level Domain (TLD) operators publish statistics about their DNS zones. Some others have a zone file access program that allows others to examine their data and publish statistics. Frederic Cambus (@fcambus on Twitter) maintains a site called statdns ( http://www.statdns.com/ ) that keeps statistics for several of the TLDs.

The EDU top level domain is conspicuously absent from the statdns site because the operators don't publish any statistics and also don't have a zone file access program in place. The EDU domain has a very complicated operational policy arrangement. It is managed by Educause (a higher education IT consortium), but operated by Verisign, under a contract with the United States Department of Commerce. I recently spoke with colleagues at Educause about current prospects for publishing some statistics or making the zone data available. The good news is that a zone file access program request is in fact in the queue to be approved by the Dept of Commerce. But it's stuck behind a few other requests, so it may still take some time to come to fruition.

In the meantime, to satisfy my own curiosity, I've been looking at other ways to obtain some statistics. In particular I'm interested in seeing how much DNSSEC deployment has happened so far, and how EDU compares with some of the other TLDs in this respect. One way to gain visibility into zone contents is to examine passive DNS databases. A number of folks and organizations run such databases that collect historical information seen from DNS responses at collections of resolvers. By searching records over a period of time in these databases, it's possible to reconstruct a substantial portion of the active records in a zone. I did this for EDU and analyzed the results recently.

The passive DNS database search managed to find about 7,158 second level domains under EDU. Of these, 6955 domains turned out to be valid (the others probably existed at one point but don't any more). EDU is known to have in the neighborhood of 7,000 delegations, so this is most probably a pretty good approximation of the active contents of the zone.

EDU Zone Statistics:

Number of Domains from passive DNS db: 7158
Number of Valid Domains: 6955

Total NS records: 19527
Unique NS records: 9757
Number of (glue) IPv4 address records: 4555
Number of (glue) IPv6 address records: 246

DNSSEC Specific Stats for EDU:

Number of DNSSEC Signed Zones: 94 (1.37%)
Number of NSEC3 Zones: 29 (30.1% of the signed zones)
Number of Zones with DS records: 76
Number of Zones with DLV records at dlv.isc.org: 7

As expected, only a very small fraction (1.37%) of domains have deployed DNSSEC. This compares with about 0.25% in .COM, 0.41% in .NET, and 0.30% in .ORG.

The 94 zones in EDU signed with DNSSEC are:

acadiana.edu
baker.edu
beloit.edu
berkeley.edu
bucknell.edu
cameron.edu
carnegiemellon.edu
catc.edu
chattanoogastate.edu
cltc.edu
cmu.edu
coloradomesa.edu
cookman.edu
csupomona.edu
cuhk.edu
desales.edu
drake.edu
example.edu
fhsu.edu
fhtc.edu
gfcmsu.edu
gsu.edu
gtc.edu
hfg.edu
highlands.edu
indiana.edu
indianatech.edu
internet2.edu
iu.edu
iub.edu
iup.edu
iupui.edu
jhuapl.edu
kestrel.edu
kiropraktik.edu
k-state.edu
ksu.edu
kutztown.edu
lctcs.edu
lsu.edu
ltc.edu
ma.edu
mansfield.edu
mcpherson.edu
merit.edu
mesa.edu
millikin.edu
mnsfld.edu
monmouth.edu
monterey.edu
mst.edu
nau.edu
northcentral.edu
northshorecollege.edu
nwltc.edu
okstate.edu
pacificu.edu
penn.edu
pitt.edu
psc.edu
richland.edu
rockefeller.edu
rose-hulman.edu
scl.edu
sdsmt.edu
sfcollege.edu
shoreline.edu
suu.edu
tbu.edu
tiasnimbas.edu
tilburguniversity.edu
tiss.edu
truman.edu
uaa.edu
ualr.edu
ucaid.edu
ucb.edu
ucberkeley.edu
ucdavis.edu
ucr.edu
uiowa.edu
umbc.edu
uni-stuttgart.edu
unt.edu
untsystem.edu
upenn.edu
upf.edu
usnwc.edu
uwm.edu
uwstout.edu
valencia.edu
waketech.edu
washjeff.edu
weber.edu

The 29 zones that use the NSEC3 variety of DNSSEC are:

csupomona.edu
cuhk.edu
gfcmsu.edu
internet2.edu
jhuapl.edu
kestrel.edu
kiropraktik.edu
k-state.edu
ksu.edu
lsu.edu
ma.edu
mansfield.edu
mcpherson.edu
millikin.edu
mnsfld.edu
pitt.edu
richland.edu
rose-hulman.edu
sdsmt.edu
suu.edu
tiasnimbas.edu
tilburguniversity.edu
ualr.edu
ucaid.edu
ucr.edu
uni-stuttgart.edu
unt.edu
untsystem.edu
washjeff.edu

There are 18 zones that do not have DS records published (not sure why):

beloit.edu
cameron.edu
cookman.edu
iup.edu
kiropraktik.edu
kutztown.edu
mansfield.edu
merit.edu
mnsfld.edu
okstate.edu
shoreline.edu
tbu.edu
tiasnimbas.edu
uaa.edu
usnwc.edu
uwm.edu
uwstout.edu
waketech.edu

There are also 7 zones with DLV records published at ISC's DLV registry, but this set is disjoint with the set that doesn't have DS records:

bucknell.edu
internet2.edu
k-state.edu
ksu.edu
ualr.edu
ucaid.edu
ucr.edu

-- Shumon Huque

Tuesday, October 22, 2013

IPv6 and DNSSEC at LISA in DC

LISA '13

Once again, I'm teaching a couple of courses at the USENIX LISA conference, this time in Washington, DC. The first is a half day course on DNSSEC on Sunday, November 3rd. And the second is a full day course on IPv6 on Monday, November 4th. I hope to see you there if you're interested in learning or talking about these topics. Early bird registration discounts for the conference end on October 22nd (sorry for the short notice).

Matt Simmons (@standaloneSA) interviewed me about both classes: DNSSEC and IPv6.

-- Shumon Huque

Wednesday, October 2, 2013

Latest World IPv6 Launch Measurements

The Internet Society recently published results of their latest round (September 17th 2013) of IPv6 measurements. The measurement data is provided by Google, Facebook, Yahoo!, and Akamai. From the description on the website: "We present measurements of network operator participants in World IPv6 Launch, based on data received from major website participants, as described in more detail below. We present a simple average of the data received, and list all networks with measurements from at least two sources, with a simple average above 0.1%."

I find it instructive to sort the results by the percentage of requests from each participating network that are composed of IPv6. This is a pretty good indicator of how extensively these networks have deployed IPv6 to their end users.

Note: the measurements are only done for networks that have signed up as participants in World IPv6 Launch. If you've deployed IPv6 to your users, you should consider registering your network to take part in these measurements.

Here's a ranked list of these networks sorted by percentage of IPv6 requests of the total from each.

     1    interscholz Internet Services GmbH & Co. KG    81.22%
     2    Sauk Valley Community College                  71.23%
     3    ThaiSarn                                       69.41%
     4    Rensselaer Polytechnic Institute               61.25%
     5    Virginia Tech                                  59.54%
     6    Universidad de Carabobo                        58.50%
     7    Sistemas Fratec S.A.                           58.19%
     8    Universidad Panamericana                       57.89%
     9    Bayu Krisnawan                                 56.64%
    10    Dedicated Zone Inc                             56.55%
    11    Google Fiber                                   55.64%
    12    REACCIUN                                       52.41%
    13    NETIS TELECOM                                  52.17%
    14    Gustavus Adolphus College                      46.64%
    15    DreamHost                                      46.32%
    16    Alhambra Eidos                                 45.37%
    17    VOO                                            45.32%
    18    SPAWAR                                         45.28%
    19    Greek Research & Technology Network            43.96%
    20    Karlsruhe Institute of Technology (KIT)        43.51%
    21    AIMES Grid Services CIC                        42.37%
    22    Host Virtual, Inc                              42.24%
    23    ARNES                                          41.90%
    24    FCCN                                           40.95%
    25    Marist College                                 40.89%
    26    guifi.net                                      39.97%
    27    University of Pennsylvania                     38.94%
    28    Zimcom Internet Solutions, Inc                 35.75%
    29    Verizon Wireless                               35.73%
    30    NIIF/Hungarnet                                 29.92%
    31    LITNET                                         29.07%
    32    DirectVPS                                      29.05%
    33    Jiri strohalm                                  28.56%
    34    Hughes Network Systems                         28.00%
    35    DegNet GmbH                                    26.76%
    36    Louisiana State University                     26.61%
    37    University of Minnesota                        26.44%
    38    iway AG                                        25.73%
    39    RedIRIS                                        25.39%
    40    University of Iowa                             22.56%
    41    Universidade Federal de Santa Catarina, Brazil 22.26%
    42    Cisco                                          22.14%
    43    Monash University                              21.82%
    44    Hurricane Electric                             21.80%
    45    RENATER                                        21.55%
    46    TUBITAK ULAKBIM / ULAKNET                      21.46%
    47    Aristotle University of Thessaloniki           20.89%
    48    DataChambers                                   20.28%
    49    UNESP                                          19.85%
    50    Chubu Telecommunications                       19.06%
    51    Swisscom                                       18.83%
    52    Indiana University                             18.08%
    53    Free                                           18.04%
    54    FranTech Solutions                             17.69%
    55    Tulane University                              17.55%
    56    University of New Hampshire                    17.45%
    57    Leibniz Supercomputing Centre                  16.60%
    58    HEAnet                                         16.59%
    59    US Dept of Transportation                      16.55%
    60    GARR                                           16.27%
    61    XS4ALL                                         16.14%
    62    Defense Research and Engineering Network       15.24%
    63    DMZGlobal                                      14.26%
    64    PCextreme B.V.                                 13.80%
    65    RCS & RDS                                      13.25%
    66    PowerTech Information Systems AS               12.24%
    67    SURFnet                                        12.07%
    68    BIT BV                                         11.93%
    69    ATT                                            11.52%
    70    Academia Sinica Network                        11.34%
    71    Honesty Net Solutions (I) Pvt Ltd               9.85%
    72    UNINETT                                         9.48%
    73    Storm Internet                                  9.25%
    74    CESNET                                          9.18%
    75    University Of Lampung                           9.12%
    76    AAISP                                           8.88%
    77    prgmr.com                                       8.61%
    78    KDDI                                            8.49%
    79    Voxel / Internap                                8.29%
    80    Init7                                           8.19%
    81    AMRES - Serbian National R&E Network            8.06%
    82    Comcast                                         7.95%
    83    CJSC Progressive Technologies                   7.92%
    84    MediaCat Div./Community Network Center Inc.     7.17%
    85    green.ch AG                                     7.02%
    86    StarHub                                         6.68%
    87    OVH                                             6.30%
    88    UniNet                                          6.26%
    89    CORPORACION NACIONAL DE TELECOMUNICACIONES      6.02%
    90    National Technical University of Athens         5.91%
    91    Forthnet                                        5.81%
    92    Deutsche Telekom AG                             5.18%
    93    EPT Luxembourg                                  5.11%
    94    Energy Group Networks                           4.62%
    95    M1 Limited                                      4.56%
    96    Internode                                       4.33%
    97    BelWue                                          4.32%
    98    Quonix Networks                                 4.26%
    99    SMELLY BLACK DOG                                4.22%
   100    LENTEL                                          4.15%

Friday, June 14, 2013

LOPSA East Class reviews - IPv6 & DNSSEC

I just received the reviews and attendee feedback for the IPv6 and DNSSEC classes I taught at the recent LOPSA-East conference. So far my recent stint as a technical course instructor at various conferences has been going well. Students are generally very pleased with the courses, and the positive feedback often results in invitations to teach at other venues.

The DNSSEC class is new. At past conferences, I've taught a combined DNS and DNSSEC class. But I've received feedback that many folks would like to see a course focussed on DNSSEC, so I created one. I also incorporated some live demos of setting up DNSSEC, which attendees found to be very useful.

I'll most likely be teaching these classes again at the USENIX LISA conference in Washington, DC later this year.

The possible responses for each question in the feedback survey are "Unsatisfactory", "Missed Some Expectations", "Met Expectations", "Exceeded Expectations", and "Greatly Exceeded Expectations". The data below is only for the (small) subset of the class that offered feedback of course.

IPv6 Course Feedback


==> SA1: Using and Migrating to IPv6 / Huque

Rate this training session: [Description matched the contents of the class]
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Met Expectations
    * Met Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Class material was useful to my job]
    * Greatly Exceeded Expectations
    * Met Expectations
    * Greatly Exceeded Expectations
    * Met Expectations
    * Met Expectations
    * Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was knowledgeable]
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was able to answer students questions]
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Course material quality]
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

What was the single BEST part of this class?
    * good material, awesome presenter
    * Excellent balance of technical detail with beginner introduction
    * instructor clear experience and ability to communicate topic
    * He has done the work, and could provide many answer from experience
    * Quality of the instructor - good speaker and very knowledgable.
    *
    * The instructor was able to explain a very complex topic clearly & in terms that are directly applicable to my future use of the material.
    * Shumon's depth of knowledge; he adapted what we covered and how fast we were going, on the fly! AWESOME instructor.

Name one aspect of this class that NEEDS IMPROVMENT?
    * needs to be longer
    *
    * wants more time to work in.
    * pacing, which material to emphasis. I thought the first part of the material could have been covered a bit faster, and more time on the meatier issues
    * The class seemed to detail a lot of 'differences between ipv6 and ipv4' and protocol internals in favor of 'how do I actually deal with migration issues'. A better mix would be nice, but I understand that unless you know of the differences, it can be hard to concentrate on implementation.
    *
    * Access to a Lab for a demo might add to the session.
    * nothing, run this class again. MAYBE, talk him into making another class focused on migrating your organization from v4 to v6... so people can do 'intro to ipv6' if they need, then another session on migration strategies.

Should LOPSA offer this class in the furture?
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes

DNSSEC Course Feedback


==> SA4: DNSSEC (DNS Security Extensions) / Huque

Rate this training session: [Description matched the contents of the class]
    * Met Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Class material was useful to my job]
    * Met Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was knowledgeable]
    * Met Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was able to answer students questions]
    * Met Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Course material quality]
    * Met Expectations
    * Missed Some Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

What was the single BEST part of this class?
    * DNSSEC appears do-able
    * My impression of DNSSEC went from theoretically possible to practical in a very short time. Something that is alluded me for quite some time.
    *
    * Best part was seeing the live application of theory in an enterprise environment.
    * Shumon's depth of knowledge; he adapted what we covered and how fast we were going, on the fly! AWESOME instructor. He's so good, we had wonky wifi, and he just ran demos through Bind on his Mac. nice.

Name one aspect of this class that NEEDS IMPROVMENT?
    *
    * Materials were not but will be posted to the presenters site
    *
    * nothing negative to say.
    * nothing, don't change anything, run it again!

Should LOPSA offer this class in the furture?
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes

Wednesday, April 10, 2013

ISOC ION Panel: Advancing the Network

"I tend to think of IPv6 & DNSSEC both a little bit like global warming ... something that is developing kind of slowly ... they're both inevitable, it's a just a question of how long it's going to take"
   - Paul Mockapetris.

The Internet Society has posted a video (1 hour 7 minutes) of the ISOC ION panel that I moderated on "Advancing the Network - Where We've Been, Where We're Headed" in San Diego on December 11th:

    http://www.internetsociety.org/deploy360/blog/2013/04/video-advancing-the-network-where-weve-been-where-were-headed-ion-san-diego/

The panelists were Ron Broersma (DREN), Paul Ebersman (Infoblox), John Spence (nephos6), and Paul Mockapetris (Nominum; and inventor of the DNS), and the main topics of discussion were IPv6 and DNSSEC. All the presentations and the subsequent Q&A were quite informative and worth listening to.

One correction I should make to something I said in my introductory remarks. My last slide showed some statistics from the SecSpider DNSSEC zone monitoring project. Since that project relies on user submissions and some amount of crawling, by now, it vastly underestimates the amount of DNSSEC deployment. There aggregate numbers are roughly 275,000 signed zones, whereas the actual number is a lot higher. The netherlands Top Level Domain (.NL) for example has more than 1.4 million signed zones underneath it:

    http://xs.powerdns.com/dnssec-nl-graph/

--Shumon.


Tuesday, March 26, 2013

IPv6 and DNSSEC at LOPSA-East Conference

I'm teaching 1/2 day courses on IPv6 and DNSSEC at this year's LOPSA-East conference again, being held in New Brunswick, New Jersey, May 3rd-4th 2013.

The IPv6 course is an updated version of the one I did last year at PICC (PICC has since been renamed LOPSA-East) and elsewhere.

Last year I also did a combined course on DNS and DNSSEC. This year's course is focussed on DNSSEC specifically. This will allow me to go into much more detail on how DNSSEC works, how to configure and deploy it (probably with live examples using BIND), etc. I'll also have more time to discuss DANE and application uses of DNSSEC.

Early bird registration discounts for the conference end April 1st. The full schedule of talks and training programs can be seen at:

http://lopsa-east.org/2013/talks/
http://lopsa-east.org/2013/lopsa-east-training/

--Shumon Huque

Sunday, January 6, 2013

USENIX LISA 2012 courses

I taught two courses at last month's USENIX LISA 2012 conference in San Diego, California. One on IPv6 and another on DNS and DNSSEC. Each had about 60 attendees and they both went very well.

Slides for the IPv6 course are available at:
http://www.huque.com/~shuque/doc/2012-12-IPv6-Tutorial-huque.pdf

Slides for the DNS and DNSSEC course are available at:
http://www.huque.com/~shuque/doc/2012-12-DNS-DNSSEC-Tutorial-huque.pdf

I also participated in the Internet Society's ION conference, where I moderated a panel on "Advancing the Network: Where We've Been and Where We're Headed". Participants on my panel were Ron Broersma (DREN), Paul Ebersman (InfoBlox), John Spence (Nephos6), and Paul Mockapetris (Nominum, and the inventor of the DNS).

A few photos from my trip are available on my Google Plus page.

--Shumon Huque

Monday, November 19, 2012

Internet2 IPv6 Panel recap

A few notes from last month's IPv6 deployment panel at the Fall Internet2 Member Meeting in Philadelphia, which I moderated (October 2nd 2012). Watch the entire video of the session (1 hour 15 minutes) for full details.

I opened the session with a brief review of World IPv6 Launch and some its measurement data, and a mention of other IPv6 deployment measurement projects, including the NIST survey of universities.

ARIN - Mark Kosters, CTO

Mark began his presentation by talking about the current state of IPv4 address depletion. ARIN has 2.87 /8 equivalent address blocks left (note: it's down to 2.79 as of Nov 18th). RIPE and APNIC are almost out, below the one /8 threshold at which a real address rationing stage has already begun - they will give out a maximum of only /20 or /22 sized blocks regardless of how much IPv4 address space you really need. With current projections, ARIN is scheduled to exhaust in August 2013, but events could dramatically change the timeline. For example there are some ISPs in the ARIN region that could easily qualify for /9 allocations. ARIN is in phase 2 of a 4-phase runout plan - details can be seen at https://www.arin.net/resources/request/ipv4_countdown.html. Today, 58% of ARIN's membership has only IPv4 address space, 6% has only IPv6, and 36% have both IPv4 and IPv6 space. Legacy address blocks (ie. allocated to organizations prior to ARIN's existence - this is quite common in the US higher ed community) comprise 45% of the ARIN address space.

The second part of Mark's talk was about IPv6 deployment activites at ARIN itself. He went through a short history of their IPv6 implementation, which dates back to 2003, at which time they had a somewhat creaky, segregated (ie. non dualstack) implementation. By 2008 they migrated to a robust dualstack implementation. All ARIN services today are native, dualstack. Meeting networks are a bit more challenging, where they often have to rely on tunnels (due to hotel carrier limitations).

Comcast - John Brzozowski, Chief IPv6 Architect

Comcast now has IPv6 enabled on 50% of their broadband network footprint. However only 2.5% of the customer base is currently dualstack - of these dualstack users about 65% are using a computer directly connected to the cable modem, and 35% have an IPv6 enabled home router to which devices are attached. Comcast expects this ratio will eventually flip over to 80/20 in favor of home routers as it is in IPv4. To date, their focus has been on residential broadband, but they have pilots for business and commercial service. The Comcast metro ethernet service is IPv6 ready and they'd love to have more customers using it. In terms of traffic, Comcast has seen a 375% increase in IPv6 compared to World IPv6 day (June, 2011), with the majority of the increase occurring between January and June 2012, in the run up to World IPv6 Launch. The majority of the traffic is composed of services like Youtube and Netflix. The 2012 Olympics (streamed by Comcast/NBC in the US) had a noticeable impact - about 6% of this traffic to Comcast customers used IPv6. Comcast continues to work on content and services. Xfinity and comcast.net are IPv6 enabled and they use Akamai as a content delivery network.

They've put in place an extensive measurement and metrics platform to proactively detect any adverse affects on customer experience and take action if needed. Comcast sees advantages to IPv6 beyond the usual address space depletion concerns. An example cited was the fact that once they've allocated an IPv6 subnet, they don't have to be concerned about resizing it again, in marked contrast to their IPv4 deployment - this adds up over time to a significant operational benefit.

Marist College - Eric Kenny

Marist College is located in upstate network with 5,500 students on a 180-acre campus with 50 buildings. They started IPv6 deployment in 2010, and by June 2012 the wired network was mostly done. They plan to deploy IPv6 to the wireless network in the winter of 2012. One big item for them was getting provider independent address space from ARIN (mostly internal process issues). They started with a provider allocated block from NySERNET, but eventually got a /48 from ARIN, and are now wondering if they should have applied for a larger block. They also have a native IPv6 BGP peering with their commercial ISP, Lightower, which was described as an interesting experience, since they were their first IPv6 customer. They use stateless address autoconfiguration, and most devices on their network support IPv6. They haven't yet made much progress with IPv6 enabled network services, apart from DNS. At the current time about 4% of the traffic crossing their border is IPv6 - they expect this number to go up considerably after wireless deployment. One of their biggest concerns has been address tracking and accountability. They've developed their own application to track IPv6 address and MAC address associations.

Louisiana State University - Allie Hopkins

Allie focussed more on the application and process side of things. As part of their rollout process, LSU has had a great dialog with the user community, with extensive outreach and communications via message boards and e-mail lists, and close contact with application developers and deployers. And despite a set of issues (described later), they rate their deployment a success and as can be seen from World IPv6 Launch measurements, they generate a substantial amount of traffic. One of the issues they've had is with their DNS based network registration system which was incapable of working properly with IPv6 enabled hosts. Another issue they encountered was connectivity issues between tagged and untagged VLANs on interfaces, due to the Cisco routers using the same IPv6 link local addresses on them - this was fixed by manually configuring unique addresses on the VLAN interfaces. They've also had ongoing issues with being put on Google's AAAA blacklist.They are aware that they have some pockets of poor IPv6 connectivity within their campus network, but so far they've gotten no help from Google about measurement data that could help them more easily track down these cases.

I was scheduled to do a short presentation about IPv6 at Penn, but I decided to skip it (I'll post the slides later) in the interests of providing more time for audience discussion and questions.

Q&A Portion

Richard Machida (U of Alaska) asked if anyone was looking into deploying IPv6-only networks for any purpose. I answered that we've deployed them only in the lab for testing purposes (longer term, we would investigate whether it's possible to run specific applications that may not need IPv4, like VoIP on them). John says Comcast has no IPv6-only networks, but they do have plenty of IPv6-only devices that sit on dual-stack networks.

There was a question about more details of LSU's network registration system (from what I gathered it's a Netreg or Netreg like system), and whether 802.1x based network access control would be a solution (802.1x authentication happens at the link layer and is IP protocol independent). It probably would, but LSU was not quite prepared to deploy it yet.

There was discussion about what folks were doing about measuring IPv4 vs IPv6 traffic. Comcast went into some more detail about their work. I showed some data from Penn, where over the summer we were approaching 11% inbound and 4% outbound traffic composed of IPv6. We've since dropped down substantially, because we had to take IPv6 off the wireless network (I'm planning to write another blog article detailing why, but the quick summary is that deploying v6 broke IP address mobility, and we're working with our wireless equipment vendor on some possible solutions).

Dan Magorian (Johns Hopkins) asked what cable modem data specifications support IPv6 and what is the state of IPv6 support in the currently deployed field. John's answer is that DOCSIS 3, the current spec supports it pretty completely, almost all current equipment implements it, but there are ways to support IPv6 on older specs too.

---

Addendum: John B was one of the recipients of the Internet Society's Itojun Service Award recently, and presented at IETF'85, for his "for his tireless efforts in providing IPv6 connectivity to cable broadband users across North America and evangelizing the importance of IPv6 deployment globally". Congratulations John!

Shumon Huque

Friday, November 16, 2012

IPv6 and DNSSEC in San Diego



I'll be in San Diego, California in early December for the USENIX LISA 2012 conference. As part of the conference's training program, I'm teaching two courses - a full day on IPv6, and a half-day on DNS and DNSSEC.

The early registration deadline (cheaper rates) is November 19th. Various discounts are available for members of USENIX and LOPSA.

The IPv6 course "Using and Migrating to IPv6", is on Monday, December 10th. This is a revised and expanded version of the half-day course I taught at last year's LISA conference in Boston, Massachusetts, which was well received. I had originally proposed to do a full-day course last year too, but the organizers at the time felt that they wouldn't be able to attract enough attendees for day long session on IPv6. It turned out that my session was packed and the audience was quite engaged - I even ran over by 30 minutes to cover some advanced topics and almost everyone stayed the extra time. Hopefully I'll get a good turnout this time also.

The DNS and DNSSEC course, is on Tuesday morning, December 11th. This will cover the basic DNS protocol as well as the DNS Security Extensions, including practical configuration examples.

Attendee comments and feedback on my last IPv6 and DNS courses (PICC 12) are available if you're interested. In courses like these with audiences with potentially diverse backgrounds, it's rarely possible to please everyone, but my courses generally get almost uniformly positive reviews (so far at least).

Incidentally, "IPv6 and DNSSEC" is one of the themes of the conference this year. Other sessions in the category include Owen DeLong of Hurricane Electric on "IPv6 Address Planning", Scott Rose of NIST on "Progress of DNSSEC deployment in the federal government", and Roland van Rijswijk of SURFnet on "DNSSEC, what every sysadmin should be doing to keep things working".

Vint Cerf is delivering the keynote on "The Internet of Things and Sensors and Actuators", which will surely discuss IPv6.

There are many other interesting sessions. Check out the agenda for the technical program, the training program, and the workshops.



Co-located with LISA, the Internet Society is also hosting its ION Conference on the afternoon of Tuesday, December 11th.  ION is a conference series organized by ISOC's Deploy360 programme, which provides deployment information on advanced technologies like IPv6, DNSSEC, secure routing, etc.

I'm moderating a panel session titled "Advancing the Network: Where We've Been, Where We're Headed" - joining me on the panel are Ron Broersma (DREN), Paul Ebersman (Infoblox), John Spence (Nephos6), and Paul Mockapetris (inventor of the DNS). Another panel focussed on DNSSEC is being run by Dan York. The full agenda is available.

Registration for the ION conference is free, but seats are limited.

Hope to see some of you in San Diego ..

Shumon Huque