Showing posts with label DNSSEC. Show all posts
Showing posts with label DNSSEC. Show all posts

Wednesday, July 30, 2014

Key Transparency for DNSSEC?

At the recent IETF meeting in Toronto, there was an interesting discussion in the trans working group on DNSSEC certificate transparency, and there is a (very) preliminary IETF draft (that needs a lot more work):

    http://tools.ietf.org/html/draft-zhang-ct-dnssec-trans

This isn't a new topic. It has been talked about off and on for a number of years. The first time I ran across this was on the "The Right Key" email list in 2012 when measures to detect and counter fraudulently issued PKI (X.509) certificates were being proposed. This ultimately led to the creation of the trans working group, whose main goal is to produce and standardize a transparency system for X.509 certificates, based on the mechanism described in RFC 6962.

Does DNSSEC need a similar transparency system? For X.509 certificates, the threats are well known and documented (I wrote about them a bit in an earlier blog article) - any of the many root certificate authorities, or their intermediates, are capable of issuing a certificate for anyone on the Internet, and it is virtually impossible to know for sure if a fraudulent certificate has been issued. A central, cryptographically verifiable audit log of issued certificates might be able to address this issue (assuming that all CAs participated in it, which is by no means a certain proposition).

For DNSSEC it isn't obvious that a similar mechanism is needed, and whenever this topic comes up, there is a lot of head scratching and bewildered looks from DNS engineers wondering what the possible threat model is. I must admit, I didn't foresee the possible attack, until it was described to me in this exchange on The Right Key list by Paul Hoffman and Ben Laurie:

    http://www.ietf.org/mail-archive/web/therightkey/current/msg00470.html

If you'd like to read the entire email thread, it starts here:

    http://www.ietf.org/mail-archive/web/therightkey/current/msg00452.html

In DNSSEC, the keys for a zone are vouched for by a single parent zone (by means of a signed DS record corresponding to the child's keys). A zone operator can query the parent's DS records himself to verify that the correct DS key is being returned. However, what the zone operator cannot do, is to verify that the parent domain is not selectively responding with false DS records to queries from a targeted set of other victims.

To quote from that thread,

"For example, assume the domain name example.newtld. The owner of example has put DS record A in the newtld zone. If the owner of newtld goes rogue and shows DS record B to a limited number of requests (such as to a particular geographic region or set of network addresses), the party with the private key associated with B can spoof example, and the owner of example would not know unless he could see B."

(Note: for this attack to be useful, in addition to showing the fake DS records, newtld would have to show fake NS records and possibly glue records to redirect the victim to alternate DNS servers with the corresponding DNSKEY records)

If the parent zone is doing this on wide scale, they are likely to get caught and face action. But highly targeted attacks will be very hard to detect. Are these scenarios far fetched? Most DNS top level domains are run by reputable organizations and would likely not risk engaging in such security shenanigans. However, even if we assume they are completely forthright, they are vulnerable to "compelled" attacks by government agencies. In April 2010, Chris Soghoian and Sid Stamm in a paper ("Certified Lies: Detecting and Defeating Government Interception Attacks against SSL") describe such attacks against SSL/TLS certificates with evidence suggesting that they are actually in use. In light of Edward Snowden's NSA revelations, these kinds of compelled attacks (legally compelled or otherwise) are more likely than ever. DNSSEC keys and delegation signer records are just as vulnerable to them.

The title of the IETF draft mentioned at the beginning of this article is "Certificate Transparency for DNSSEC", which is probably a misnomer. The data that would be most valuable to enter into a DNSSEC transparency log are not certificates, but secure entry point keys for zones (e.g. DS records and/or Key Signing Keys). So a more appropriate name might be "Key Transparency for DNSSEC". DNS zones can contain certificates, or hashes corresponding to certificates (e.g. TLSA, CERT, etc records), however there are many other record types that might contain cryptographic keying material (SSHFP, IPSECKEY, and proposals for others in the pipeline). And why not have an audit log for more mundane non-crypto records too, eg. name to address mappings? Individually logging all of these data types for every zone will likely prove to be an infeasible task.

If we limit the scope of the transparency log to DS records, there are still some very significant technical challenges that need to be solved. One is scalability to the world wide DNS. The Certificate Transparency log is being implemented as an append-only log with a Merkle tree, and a DNSSEC log will likely follow the same approach. Quoting RFC 6962:

"The append-only property of each log is technically achieved using Merkle Trees, which can be used to show that any particular version of the log is a superset of any particular previous version. Likewise, Merkle Trees avoid the need to blindly trust logs: if a log attempts to show different things to different people, this can be efficiently detected by comparing tree roots and consistency proofs. Similarly, other misbehaviors of any log (e.g., issuing signed timestamps for certificates they then don't log) can be efficiently detected and proved to the world at large."

Even though DNSSEC is still in a fledgeling state of deployment, we need to design a mechanism that can scale to the entire DNS system and accommodate the expected churn of zone keys (e.g. due to key rollovers etc). A single centralized log may not be able to do this, and alternative models may need to be considered (e.g. limiting the depth of zones that the log will hold; implementing a hierarchy of logs, etc).

Another problem is that a rogue or compelled parent zone can not only return fake DS records, but could also answer authoritatively for names inside the child zone without any referrals to the child zone (a fake in-zone answer). This is harder to protect against, but I can think of a number of possible defenses. At the level of TLDs (top level domains) one possible protection might be to have DNS resolvers treat them as delegation-only and reject all subdomain answers that aren't referrals. Another (probably more promising) approach is for resolvers to employ a query-name minimization algorithm that only reveals the needed labels of the query name to authoritative DNS servers as they traverse the delegation hierarchy. In fact, there is active work going on in the DNS engineering community on such qname minimization schemes and other privacy enhancing extensions to the DNS.

A DNSSEC transparency log, if deployed, could be useful as an audit channel to periodically detect attacks, and for forensics. Performing checks of the log inline with the DNS resolution process may not be practical because of the probably high performance penalty (with the currently proposed log structure), which means attacks could not be detected in real time.

So is it really worth deploying such a system? I'm not yet sure. In the end, DNS is a hierarchical system, and there is always the possibility of being victimized by your parent zone either by error, incompetence, malice, or coercion. Even if we deploy centralized sets of transparency logs, we'd have to them think about how to prevent them from being compromised or co-erced. There are decentralized (non-hierarchical) naming systems out there, like gnunet, namecoin, etc. But they have the usual problem that they are only really used by a small group of very technically savvy users. We should probably take a more serious look at them. But I think the compelled attack is a very real threat, and it's probably worth some serious thought about how to deploy practical defenses against it in the global DNS.

The IETF recently published RFC 7258, declaring that pervasive monitoring is an attack for which all IETF protocols should have technical countermeasures. It may also be time for the IETF to standardize mitigations for highly targeted attacks.

Shumon Huque

Friday, May 2, 2014

HUQUE.COM has a signed delegation now

(Off topic: I've been at Verisign Labs over a month now - I'll write a separate post describing my experiences at my new job when I have more time. But the short summary is: no regrets so far; lots of interesting technical work I'm getting involved in; it was probably time for me to move on from Penn; and I think I made the right decision).

A few weeks ago, I created a set of test DNS TLSA records in my personal zone (huque.com) to help out some folks participating in the getdns-api hackathon. To correctly authenticate records in the zone, folks needed to have resolvers configured to use the ISC DLV registry, since I hadn't gotten around to setting up a secure delegation in the parent zone (.COM). This finally motivated me to get that done.

Why didn't I do this earlier? Long story, but I've used my own domain for a lot of experiments with DNSSEC configurations over the years (going back to 2007), including testing configurations that I eventually deployed in other larger, production domains. The bigger stumbling block is that the DNS registrar I was using, Network Solutions, does not support DNSSEC (specifically it provides no way for customers to transmit DNSKEY or DS record contents to the TLDs it registers domains in).

So last week, I started the process of changing to a DNSSEC capable registrar. There are a growing number of them out there, and ICANN maintains a list of them. The one I chose is GKG.net.

This was my first time switching registrars - Network Solutions has been my registrar since 1998 when I first registered the domain. At that time Network Solutions both operated the .com DNS top level domain (among others) and registered domains within it. In 2000, Verisign (where I now work) acquired Network Solutions to get into the DNS business, and in 2003 spun it off as a registrar-only business, retaining operation of the .com registry. Unfortunately, as far as I can tell, Network Solutions has announced no plans to support DNSSEC.

The whole process took about 5 days. On April 24th I logged into my network solutions account, unlocked my registration and requested an authorization code to transfer my domain registration. On the 25th I called them on the telephone to confirm the transfer intention and to expedite the delivery of the authorization code (it normally takes them 3 days), which I got shortly after. I then created an account at gkg.net, and initiated the domain transfer request with the supplied code. The charge was US $8.20, which included a 1-year registration period that extended beyond the current expiration date for my registration with Network Solutions. On the 28th, the transfer was successfully completed.

I then logged into the gkg.net account and used their DNSSEC interface to add a DS record for huque.com, which appeared in the .COM zone more or less immediately.

huque.com.      86400 IN DS 40924 8 2 (
                816524EB1C3B7D1315AE8330652DD17909C95DE0533C
                1F2DC023BFFEDB1F5E9B )
huque.com.      86400 IN RRSIG DS 8 2 86400 (
                20140509035959 20140502024959 56657 com.
                UkLdvaDKUcHmqAe8JQyXrxn+luWRKrkjfNzG4/xd/PXy
                zQr03L1ZXNzJHnVp7PZSau2UVfsfz5BmYGN5jepIScPc
                57zd/CnKXTZgucT9ry7dHvkdmxr+UCGY1Zg4LQ0pDyAY
                2avC9Hd2gJKBNJGWfZlGU/KHa1KvRv8fqlNWWQo= )


Below is a screen capture of their DNSSEC interface. You need to supply the parameters from your zone's DNSSEC key: the keytag, algorithm, digest type, and the digest string.


Here's a graphical view of the current chain of trust for my zone from the DNSviz tool. As you can see, there are two secure entry paths into my zone, one via the just created normal delegation path through .COM, and one via the DLV Registry at dlv.isc.org. I will be removing the DLV entries shortly.




Saturday, February 1, 2014

DNSSEC/DANE/TLSA Browser Add-ons

The folks at CZ.NIC (the operators of the Czech Republic's country-code top level domain: .cz) have created a set of web browser add-ons to perform DNSSEC/DANE/TLSA validation. You can read about them and download them from their website:

        https://www.dnssec-validator.cz/

I installed the Firefox web browser plugin and did some quick tests of them on my own website. The plugin installs two new icons on the right side of the browser's location (URL) bar. The first with a key on it shows information about whether the domain name for the website has a valid DNSSEC signature associated with it. The second icon with a lock on it shows information about whether the TLS certificate of the website can be authenticated with a DANE TLSA record. Here are screenshots with my own website ( https://www.huque.com/ ).

In this first screenshot (below), I clicked on the key icon, and it reports that the 'www.huque.com'  domain name has a valid DNSSEC signature.


In the next screenshow (below), I clicked on the lock icon, and it reports that the certificate for www.huque.com has been successfully authenticated by means of a signed TLSA record.


In this case, since this is an HTTPS connection at the standard port (TCP port 443), the plugin looked for the TLSA record at the domain name "_443._tcp.www.huque.com."

    $ dig _443._tcp.www.huque.com. TLSA +noall +answer
   [...]
    _443._tcp.www.huque.com. 7200 IN TLSA 3 0 1 (
                    7EF4BD014E9A4F302FC1EE74FB2D29718C5B0F4CB23B
                    25B267A1D92F0410890B )
    _443._tcp.www.huque.com. 7200 IN RRSIG TLSA 8 5 7200 (
                    20140217205026 20140118205010 14703 huque.com.
                    NsUKFsBAUD4OxrHQ72iB0Oz9mBoMEqL8wMsks56sp2yz
                    3ksXcqGSddooC3jZvGH/4iF6ssD3KRNQVONJqpK246nX
                    jPhxBhM730TKEwMZRw/NRqYanRKyEMhkUy538suej0Pv
                    rK3w8r6tdNF4gXqIM3sQlz9gPY/WOu0zxjezaIk= )


Below is another screenshot for https://www.ietf.org/. In this case, the second icon has a cross marked on it, meaning that no TLSA record was found for this site. Apparently, the IETF is not yet eating its own dogfood. Although see this short slide deck from IETF'87 - there appears to be a proposal to do so.



There are a few configuration options that can be set for the add-on. Here is a view of the settings window:


The plug-in appears to do its own DNS resolution (and validation) by default. But you can also choose to use DNS resolvers configured for your system, or a customer resolver such as the Google public resolver (8.8.8.8).

If you need help creating a TLSA record for your website, I have a web based tool available here:

    https://www.huque.com/bin/gen_tlsa

One thing I should mention, in case you're looking at the configuration of my website: huque.com does not today have a secure delegation (i.e. DS record) published in its parent zone. This is because the registrar I use, Network Solutions, still cannot process requests to install DS records. I did quick check on their website (again) to see if anything's changed. Doesn't appear so:


Instead, I've had DLV record published in the ISC DLV Registry. But there are several big resolver services, like Google DNS, and Comcast, that do not perform lookaside validation, so it's probably time to switch registrars. If anyone has suggestions for competent DNSSEC enabled registrars (with registrar-lock support), I'd be happy to receive them. I hope to make the switch soon.

Dan York from ISOC also has an article on these addons here. (I started writing this before seeing his!)

--Shumon Huque


Addendum (May 2014): my domain huque.com now has a secure delegation from .COM.

Sunday, December 1, 2013

EDU Top Level Domain statistics

Some DNS Top Level Domain (TLD) operators publish statistics about their DNS zones. Some others have a zone file access program that allows others to examine their data and publish statistics. Frederic Cambus (@fcambus on Twitter) maintains a site called statdns ( http://www.statdns.com/ ) that keeps statistics for several of the TLDs.

The EDU top level domain is conspicuously absent from the statdns site because the operators don't publish any statistics and also don't have a zone file access program in place. The EDU domain has a very complicated operational policy arrangement. It is managed by Educause (a higher education IT consortium), but operated by Verisign, under a contract with the United States Department of Commerce. I recently spoke with colleagues at Educause about current prospects for publishing some statistics or making the zone data available. The good news is that a zone file access program request is in fact in the queue to be approved by the Dept of Commerce. But it's stuck behind a few other requests, so it may still take some time to come to fruition.

In the meantime, to satisfy my own curiosity, I've been looking at other ways to obtain some statistics. In particular I'm interested in seeing how much DNSSEC deployment has happened so far, and how EDU compares with some of the other TLDs in this respect. One way to gain visibility into zone contents is to examine passive DNS databases. A number of folks and organizations run such databases that collect historical information seen from DNS responses at collections of resolvers. By searching records over a period of time in these databases, it's possible to reconstruct a substantial portion of the active records in a zone. I did this for EDU and analyzed the results recently.

The passive DNS database search managed to find about 7,158 second level domains under EDU. Of these, 6955 domains turned out to be valid (the others probably existed at one point but don't any more). EDU is known to have in the neighborhood of 7,000 delegations, so this is most probably a pretty good approximation of the active contents of the zone.

EDU Zone Statistics:

Number of Domains from passive DNS db: 7158
Number of Valid Domains: 6955

Total NS records: 19527
Unique NS records: 9757
Number of (glue) IPv4 address records: 4555
Number of (glue) IPv6 address records: 246

DNSSEC Specific Stats for EDU:

Number of DNSSEC Signed Zones: 94 (1.37%)
Number of NSEC3 Zones: 29 (30.1% of the signed zones)
Number of Zones with DS records: 76
Number of Zones with DLV records at dlv.isc.org: 7

As expected, only a very small fraction (1.37%) of domains have deployed DNSSEC. This compares with about 0.25% in .COM, 0.41% in .NET, and 0.30% in .ORG.

The 94 zones in EDU signed with DNSSEC are:

acadiana.edu
baker.edu
beloit.edu
berkeley.edu
bucknell.edu
cameron.edu
carnegiemellon.edu
catc.edu
chattanoogastate.edu
cltc.edu
cmu.edu
coloradomesa.edu
cookman.edu
csupomona.edu
cuhk.edu
desales.edu
drake.edu
example.edu
fhsu.edu
fhtc.edu
gfcmsu.edu
gsu.edu
gtc.edu
hfg.edu
highlands.edu
indiana.edu
indianatech.edu
internet2.edu
iu.edu
iub.edu
iup.edu
iupui.edu
jhuapl.edu
kestrel.edu
kiropraktik.edu
k-state.edu
ksu.edu
kutztown.edu
lctcs.edu
lsu.edu
ltc.edu
ma.edu
mansfield.edu
mcpherson.edu
merit.edu
mesa.edu
millikin.edu
mnsfld.edu
monmouth.edu
monterey.edu
mst.edu
nau.edu
northcentral.edu
northshorecollege.edu
nwltc.edu
okstate.edu
pacificu.edu
penn.edu
pitt.edu
psc.edu
richland.edu
rockefeller.edu
rose-hulman.edu
scl.edu
sdsmt.edu
sfcollege.edu
shoreline.edu
suu.edu
tbu.edu
tiasnimbas.edu
tilburguniversity.edu
tiss.edu
truman.edu
uaa.edu
ualr.edu
ucaid.edu
ucb.edu
ucberkeley.edu
ucdavis.edu
ucr.edu
uiowa.edu
umbc.edu
uni-stuttgart.edu
unt.edu
untsystem.edu
upenn.edu
upf.edu
usnwc.edu
uwm.edu
uwstout.edu
valencia.edu
waketech.edu
washjeff.edu
weber.edu

The 29 zones that use the NSEC3 variety of DNSSEC are:

csupomona.edu
cuhk.edu
gfcmsu.edu
internet2.edu
jhuapl.edu
kestrel.edu
kiropraktik.edu
k-state.edu
ksu.edu
lsu.edu
ma.edu
mansfield.edu
mcpherson.edu
millikin.edu
mnsfld.edu
pitt.edu
richland.edu
rose-hulman.edu
sdsmt.edu
suu.edu
tiasnimbas.edu
tilburguniversity.edu
ualr.edu
ucaid.edu
ucr.edu
uni-stuttgart.edu
unt.edu
untsystem.edu
washjeff.edu

There are 18 zones that do not have DS records published (not sure why):

beloit.edu
cameron.edu
cookman.edu
iup.edu
kiropraktik.edu
kutztown.edu
mansfield.edu
merit.edu
mnsfld.edu
okstate.edu
shoreline.edu
tbu.edu
tiasnimbas.edu
uaa.edu
usnwc.edu
uwm.edu
uwstout.edu
waketech.edu

There are also 7 zones with DLV records published at ISC's DLV registry, but this set is disjoint with the set that doesn't have DS records:

bucknell.edu
internet2.edu
k-state.edu
ksu.edu
ualr.edu
ucaid.edu
ucr.edu

-- Shumon Huque

Wednesday, November 20, 2013

New DNS Top Level Domains

If you follow DNS news, you may know that ICANN has put in place a program to introduce many new generic top level domains (GTLD) into the DNS. I haven't been a fan. ICANN says there is market demand for GTLD expansion (perhaps), and that it allows innovation in the DNS ecosystem (how?). It probably will have an effect of diluting the entrenched market power of the big TLD operators (.com, .org etc), which may be a good thing. But the system may end up being primarily a significant financial windfall for ICANN. Even Esther Dyson (original ICANN chair) has spoken out against the program.

There appear to be some trademark protection mechanisms built in to the new system. But it seems clear that many organizations will rush to defensively register their names under some of the new TLDs. Strictly speaking, DNS domain names have no intended or actual relation to trademarks, but we have to deal with the real world. My university's upper administration has already contacted the IT department to discuss the topic. A while back, we defensively registered "upenn.xxx" to protect against possible reputational damage (and no, I wasn't involved in that decision).

On a more technical note, one interesting and welcome feature of the new GTLDs, is that they must be deployed with DNSSEC. This should significantly increase the proportion of signed top level domains in the DNS. My dnsstat DNS monitoring site has been monitoring the TLDs for a while now, and I just updated it with the latest list of TLDs.

    http://www.huque.com/app/dnsstat/category/tld/

Since late August, 32 new TLDs have been introduced, 27 normal GTLDs, 5 IDN (Internationalized domains) TLDs. But 11 IDN TLDs have also disappeared. That's a net gain of 21 TLDs, bringing the total count to 339.

Some DNSSEC specific stats: 143 (or 42.2%) of the TLDs are signed with DNSSEC. Here's a breakdown of type key and zone signing algorithms in use for the signed TLDs:

Key Signing Keys (KSK):
RSASHA256 (8) = 119 (63.0%)
RSASHA512 (10) = 6 (3.2%)
RSASHA1 (5) = 16 (8.5%)
RSASHA1-NSEC3-SHA1 (7) = 48 (25.4%)

Zone Signing Keys (ZSK):
RSASHA256 (8) = 133 (62.4%)
RSASHA512 (10) = 8 (3.8%)
RSASHA1 (5) = 17 (8.0%)
RSASHA1-NSEC3-SHA1 (7) = 55 (25.8%)

Note: new GTLDs continue to be added, so the numbers in this article might be out of date soon.

Here are the added TLDs so far (as of November 20th 2013):

+ bike
+ camera
+ clothing
+ construction
+ contractors
+ diamonds
+ directory
+ enterprises
+ equipment
+ estate
+ gallery
+ graphics
+ guru
+ holdings
+ kitchen
+ land
+ lighting
+ photography
+ plumbing
+ sexy
+ singles
+ tattoo
+ technology
+ tips
+ today
+ ventures
+ voyage

Here are the new IDN TLDs:

+ xn--80asehdb
+ xn--80aswg
+ xn--mgba3a4f16a
+ xn--ngbc5azd
+ xn--unup4y

Here are the deleted IDN TLDs:

- xn--0zwm56d
- xn--11b5bs3a9aj6g
- xn--80akhbyknj4f
- xn--9t4b11yi5a
- xn--deba0ad
- xn--g6w251d
- xn--hgbk6aj7f53bba
- xn--hlcj6aya9esc7a
- xn--jxalpdlp
- xn--kgbechtv
- xn--zckzah

Note: one IDN TLD (xn--l1acc) has had a severely busted DNSSEC deployment for a while. My monitoring system detects that its DS records in the root of the DNS do not match any DNSKEY records in the zone, and furthermore, the signatures on the DNSKEY records have expired. I hope they get their act together soon.

--Shumon Huque

Wednesday, October 23, 2013

TLSA Record Generator

Last year I wrote a blog article on DNSSEC and Certificates. Occasionally I get questions from folks who've tried to follow my instructions to create the content of TLSA records, but have failed because they are using a version of openssl that is too old to generate SHA-256 and SHA-512 hashes.

I've written a small web application to help generate TLSA records. I hope this is of use to some folks:

        https://www.huque.com/bin/gen_tlsa

(I apologize in advance for my rather primitive webpage design skills!)

Here is a screenshot of it in action to generate the TLSA record for my own website:


And the resulting TLSA record that was generated:



-- Shumon Huque

Tuesday, October 22, 2013

IPv6 and DNSSEC at LISA in DC

LISA '13

Once again, I'm teaching a couple of courses at the USENIX LISA conference, this time in Washington, DC. The first is a half day course on DNSSEC on Sunday, November 3rd. And the second is a full day course on IPv6 on Monday, November 4th. I hope to see you there if you're interested in learning or talking about these topics. Early bird registration discounts for the conference end on October 22nd (sorry for the short notice).

Matt Simmons (@standaloneSA) interviewed me about both classes: DNSSEC and IPv6.

-- Shumon Huque

Sunday, September 8, 2013

DNSSEC Validation in the Internet2 community

As a follow-up to my examination of the ISC DLV registry, I conducted an informal poll of some of my peers in the Internet2 community to find out 1) who is using DNSSEC validation on their resolvers, and 2) who additionally uses the ISC DLV.

A while back I setup a small project to monitor the status of DNS signed zones in Internet2 and few other selected communities. There is no easy way to programmatically determine who is using DNSSEC validation though, so the easiest way is to ask others [1]. I got responses from a number of universities and regional R&E networks. Here's a summary:

Institution                             Uses ISC DLV?
University of Pennsylvania              Yes      
Virginia Tech                           Yes
Univ of California, Los Angeles         No
Univ of Massachusetts, Amherst          No
Kansas Research & Education Network     Yes

Kansas State University                 <unknown>
Fort Hays State University              <unknown>
Louisiana State University              No
Univ of California, Berkeley            Yes
Energy Sciences Network (ESNet)         Yes
Lawrence Berkeley National Lab (LBNL)   <unknown>
North Dakota State University           Yes
Univ of Delaware                        Yes

3ROX (3 Rivers Optical Exchange)        Yes
Pittsburgh Supercomputing Center (PSC)  <some resolvers>
University of Idaho                     No

I'm sure I'm missing others - I'll add to this list as I discover them. If you know of anyone, feel free to let me know!

Footnotes:
[1] Although Geoff Huston and others have conducted some large scale studies of validation use, using a method of buying and analyzing ad impressions at popular websites, directing clients to carefully constructed URLs located in zones with differing DNSSEC signature statuses.

-- Shumon Huque

Sunday, September 1, 2013

ISC DLV registry usage

On a LinkedIn forum, Dan York of the Internet Society recently asked a question about who still uses the ISC DNSSEC Lookaside Validation (DLV) registry. While commenting on the discussion, I decided to take a look at the contents of the registry, and I'm sharing some of my findings in this article.

DLV is a method to locate DNSSEC public keys off-path. See RFC 5074 and RFC 4431 for details. It is meant to be an early deployment aid until full deployment of DNSSEC happens. It's useful in situations where the DNSSEC keys for a target zone cannot be obtained by the normal top down traversal of the DNS delegation hierarchy, typically because one or more zones between the target zone and the root aren't signed. Another situation is where a parent zone may be signed but it was not possible for the child zone to have a Delegation Signer (DS) record installed in the parent for some reason - a common one is that the DNS registrar in use did not support the ability to do it.

Internet Systems Consortium (ISC) runs a DLV registry at dlv.isc.org. The basic idea is that if you can't find a DS record for a zone, say "example.org", you append the name of the DLV registry and look for DLV record at "example.org.dlv.isc.org" - the contents of the record are the same as would have been found at the DS record. Validating resolvers are pre-configured with the public key of the dlv.isc.org zone and use it to authenticate the signature associated with the DLV record.

It appears that some large DNS resolver services like Google DNS and Comcast do not use any DLV registries for validation, so only zones that have an intact chain of trust can have their data validated. I'm not sure if ISC publishes any usage statistics for their DLV registry, but from casual discussion with colleagues in the US R&E community over the years, I know quite a number of universities that do have their campus resolvers configured to use it. We use it at the University of Pennsylvania too.While upenn.edu is signed and has a secure delegation in its parent, there are some auxiliary zones that we run, like magpi.net that don't have a secure delegation, and we make use of the ISC DLV registry to publish keys there. In MAGPI's case, the reason is that the registrar we use, Network Solutions, still doesn't support DS records. I suppose it's time to switch registrars, and it's on my todo list!

In modern versions of  resolvers like ISC BIND and Unbound, a mere one line addition to the configuration file will turn this feature on. In fact, some OS distributions, like Fedora Linux already have it turned on in their default configuration.

The ISC DLV zone by design uses NSEC, so it's trivial to write a short program to fully enumerate its contents and look at the data. Here's what I see from a snapshot of the zone taken on August 29th 2013:

  Number of distinct zones:    2760
  Total number of DLV records: 6020

The number of DLV records is higher because most zones have multiple DLV records - their key digests are published with mutiple hashing algorithms (SHA1 and SHA256), and in some cases mutiple keys are published (perhaps key rollovers are in progress). Here's a breakdown of the number of DLV records per zone, and the number of zones with that many records:

  #DLV recs   #Zones
  8                1
  6                3
  4              241
  2             2515


The zone with 8 DLV records (!) incidentally is hysh.jp (4 keys, 2 digests/key).

Looking at the distribution of zones across Top Level Domains (TLD), we see:

  Number of TLDs represented: 111

There are 318 total TLDs at the current time, 116 of which appear to be signed, so that leaves 202 that aren't. I maintain some more detailed statistics of the TLDs at http://www.huque.com/app/dnsstat/category/tld/

Here's the full list of the 111 TLDs represented, sorted by descending order of the number of zones within them that are in the ISC DLV registry.

  arpa 487
  com 456
  org 270
  net 263
  de 185
  info 75
  eu 67
  uk 66
  ch 50
  hu 49
  ro 34
  us 34
  cz 32
  za 31
  pl 31
  fr 29
  ru 28
  ca 28
  it 26
  biz 25
  be 25
  au 25
  nl 24
  jp 22
  id 22
  name 20
  me 20
  mx 19
  tv 18
  at 17
  edu 16
  tw 13
  tk 12
  es 12
  mobi 11
  br 10
  cx 10
  co 8
  is 8
  nu 8
  fi 8
  sk 8
  dk 7
  se 7
  gov 6
  im 6
  ua 6
  am 6
  asia 5
  ws 5
  cc 5
  in 5
  nz 5
  xn--p1ai 5
  pt 4
  gs 3
  do 3
  bz 3
  cn 3
  hr 3
  ms 3
  ve 3
  mil 3
  nf 3
  gm 2
  lc 2
  la 2
  li 2
  th 2
  ph 2
  hn 2
  mu 2
  pro 2
  ar 2
  io 2
  ni 2
  gr 1
  gp 1
  lv 1
  to 1
  tl 1
  lu 1
  tj 1
  tg 1
  ec 1
  rs 1
  re 1
  jobs 1
  cm 1
  int 1
  tm 1
  pe 1
  pn 1
  aero 1
  hk 1
  md 1
  mg 1
  uy 1
  mw 1
  ug 1
  vc 1
  ae 1
  ai 1
  al 1
  vn 1
  as 1
  xxx 1
  kg 1
  sr 1
  st 1
  kr 1


Interestingly of the 2760 zones, 653 of them (almost a quarter!) also have DS records in their parent zones, so technically they don't need to be in the DLV registry at all. This includes three TLDs: th, ua, and kg. I wonder what the motivation for additionally maintaining keys in a DLV registry is. One theoretical reason might be to have an off-path database of keys that could be audited in case of an attack in the normal delegation chain.

Below are the sixteen zones inside .EDU:

  bucknell.edu                 DS exists
  internet2.edu                DS exists
  k-state.edu                  DS exists
  cs.kent.edu                  kent.edu not signed
  ksu.edu                      DS exists
  ai.mit.edu                   mit.edu not signed
  csail.mit.edu                mit.edu not signed
  dlp.mit.edu                  mit.edu not signed
  lcs.mit.edu                  mit.edu not signed
  npitest.psu.edu              psu.edu not signed
  ualr.edu                     DS exists
  ucaid.edu                    DS exists
  cse.ucdavis.edu              DS exists
  math.ucdavis.edu             DS exists
  ucr.edu                      DS exists
  maf.wisc.edu                 wisc.edu not signed


The EDU TLD is signed and has single registrar (Educause) that has supported DNSSEC for a long time. All the second level domains in the list above also have DS records in EDU, so they don't really need to also have DLV records. Most of the third level domains (one at Kent State U, four at MIT, one at Penn State, and one at U of Wisconsin) have parents that are not yet signed, so that makes sense. However, the two third level domains, cse.ucdavis.edu and math.ucdavis.edu have DS records in ucdavis.edu, so don't need DLV records either.
 
Shumon Huque

Friday, June 14, 2013

LOPSA East Class reviews - IPv6 & DNSSEC

I just received the reviews and attendee feedback for the IPv6 and DNSSEC classes I taught at the recent LOPSA-East conference. So far my recent stint as a technical course instructor at various conferences has been going well. Students are generally very pleased with the courses, and the positive feedback often results in invitations to teach at other venues.

The DNSSEC class is new. At past conferences, I've taught a combined DNS and DNSSEC class. But I've received feedback that many folks would like to see a course focussed on DNSSEC, so I created one. I also incorporated some live demos of setting up DNSSEC, which attendees found to be very useful.

I'll most likely be teaching these classes again at the USENIX LISA conference in Washington, DC later this year.

The possible responses for each question in the feedback survey are "Unsatisfactory", "Missed Some Expectations", "Met Expectations", "Exceeded Expectations", and "Greatly Exceeded Expectations". The data below is only for the (small) subset of the class that offered feedback of course.

IPv6 Course Feedback


==> SA1: Using and Migrating to IPv6 / Huque

Rate this training session: [Description matched the contents of the class]
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Met Expectations
    * Met Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Class material was useful to my job]
    * Greatly Exceeded Expectations
    * Met Expectations
    * Greatly Exceeded Expectations
    * Met Expectations
    * Met Expectations
    * Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was knowledgeable]
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was able to answer students questions]
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Course material quality]
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

What was the single BEST part of this class?
    * good material, awesome presenter
    * Excellent balance of technical detail with beginner introduction
    * instructor clear experience and ability to communicate topic
    * He has done the work, and could provide many answer from experience
    * Quality of the instructor - good speaker and very knowledgable.
    *
    * The instructor was able to explain a very complex topic clearly & in terms that are directly applicable to my future use of the material.
    * Shumon's depth of knowledge; he adapted what we covered and how fast we were going, on the fly! AWESOME instructor.

Name one aspect of this class that NEEDS IMPROVMENT?
    * needs to be longer
    *
    * wants more time to work in.
    * pacing, which material to emphasis. I thought the first part of the material could have been covered a bit faster, and more time on the meatier issues
    * The class seemed to detail a lot of 'differences between ipv6 and ipv4' and protocol internals in favor of 'how do I actually deal with migration issues'. A better mix would be nice, but I understand that unless you know of the differences, it can be hard to concentrate on implementation.
    *
    * Access to a Lab for a demo might add to the session.
    * nothing, run this class again. MAYBE, talk him into making another class focused on migrating your organization from v4 to v6... so people can do 'intro to ipv6' if they need, then another session on migration strategies.

Should LOPSA offer this class in the furture?
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes

DNSSEC Course Feedback


==> SA4: DNSSEC (DNS Security Extensions) / Huque

Rate this training session: [Description matched the contents of the class]
    * Met Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Class material was useful to my job]
    * Met Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was knowledgeable]
    * Met Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Instructor was able to answer students questions]
    * Met Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations
    * Greatly Exceeded Expectations

Rate this training session: [Course material quality]
    * Met Expectations
    * Missed Some Expectations
    * Greatly Exceeded Expectations
    * Exceeded Expectations
    * Greatly Exceeded Expectations

What was the single BEST part of this class?
    * DNSSEC appears do-able
    * My impression of DNSSEC went from theoretically possible to practical in a very short time. Something that is alluded me for quite some time.
    *
    * Best part was seeing the live application of theory in an enterprise environment.
    * Shumon's depth of knowledge; he adapted what we covered and how fast we were going, on the fly! AWESOME instructor. He's so good, we had wonky wifi, and he just ran demos through Bind on his Mac. nice.

Name one aspect of this class that NEEDS IMPROVMENT?
    *
    * Materials were not but will be posted to the presenters site
    *
    * nothing negative to say.
    * nothing, don't change anything, run it again!

Should LOPSA offer this class in the furture?
    * Yes
    * Yes
    * Yes
    * Yes
    * Yes

Saturday, April 13, 2013

DNS Amplification Attacks

There has been a lot of talk recently about DNS amplification attacks (with prominent news reports of high bandwidth attacks targeted at anti-spam services, cloud providers, financial institutions, etc). These are a class of denial of service attack that use DNS servers to emit large amounts of traffic onto unsuspecting victims. The attackers use the forged source addresses of their victims to send a large stream of queries to the DNS servers, and this results in a much larger stream of responses being sent from those DNS servers back to the victim computers - with the aim of overwhelming them or their network connections. Although the DNS servers employed in the attack can become adversely impacted, the primary target of the attacks are the computers whose addresses are being forged.

Attacks that employ IP source address forgery are effective on the Internet today, because the countermeasures that would prevent this forgery are not very widely deployed. BCP 38 (a 13 year old document!) and BCP 84 describe network ingress filtering as a way for ISPs to block forged traffic from their customer networks, but many ISPs fail to employ it. Organizations should also ideally configure their networks not to permit internally generated traffic with forged addresses from crossing their borders - typically a border Access Control List or more granular techniques like unicast reverse path forwarding (URPF, a per-subnet level antispoofing method) can be used to do this, but once again, these are not in very widespread use.

In the past, the attacks commonly employed 'open recursive DNS resolvers' - these are DNS recursive resolvers that will resolve names for any client computer without restriction. When such servers are available, the attacker can use a DNS record of their choosing (possibly under the attackers control) that intentionally generates a very large response. To prevent such abuse, many organizations these days lock down their recursive resolvers so that they answer requests only from clients on their own networks. Penn does this also. There are however public DNS resolver services, like Google DNS, and OpenDNS, that by design are open to the world, and so need to have effective countermeasures in place to deal with these potential attacks. Both Google and OpenDNS say that they do.

There are still a huge number of open recursive DNS resolvers on the Internet, the vast majority of which are probably unintentionally so. The Open DNS Resolver Project has been cataloging them and reports a number in the neighborhood of 25 million!

But authoritative DNS servers (which need to be open to the world) are also quite vulnerable to amplification attacks. In this case, the attacker cannot choose an arbitrary DNS record, but instead must use only records that already exist in the authoritative server's zones. However, DNS responses are usually always amplifying (ie. the size of the response is a lot larger than the size of the request), so it's only a question of the scale of attack that can be achieved.

Interestingly, DNSSEC, a security enhancement to the DNS protocol, makes the amplification problem significantly worse, since DNS responses with cryptographic signatures are much bigger than normal, unsigned DNS responses. Hence lately, we've been seing a lot of these attacks target authoritative servers with zones that are signed with DNSSEC.

Penn was one of the earliest organizations to deploy DNSSEC (August 2009, well before the root and most of the TLDs were signed). We first noticed such attacks on our authoritative servers in July of last year (2012) - at that time we didn't have a good way to counteract these, but our server and network infrastructure was easily able to absorb the attacks, so we didn't take any action - the attacks continued for a few weeks and then disappeared. In late January 2013 though, a new round of amplification attacks happened on a much larger scale that did negatively affect our infrastructure, causing high load on two of our servers and almost saturating the outbound bandwidth on their network connections. By this time, code to implement an experimental response rate limiting countermeasure was available (see Vernon Schryver and Paul Vixie's Response Rate Limiting (RRL) -- implementations are available for popular DNS server software such as BIND, NSD, etc). We deployed these enhancements shortly after the new attacks, and they have effectively addressed the problem for the time being. The RRL code works by keeping track of client requests, and for repeated requests for the same records from the same client addresses it rate limits the responses, either by silently ignoring some requests, or providing small 'truncated' responses. The working assumption is that well behaved DNS resolvers cache responses for the advertised TTL of the record and so should not be making repeated queries for the same record in a short period of time. The truncated responses when encountered will cause well behaved DNS resolvers to retry their query over TCP, which cannot be effectively used in forged address attacks. More details of how this works are available in this technical note describing the operation of RRL. I've heard that the RRL extensions are planned to be incorporated officially into BIND 9.10, although from the recent announcement about ISC's new DNSco subsidiary, it isn't clear whether this feature will be available only to commercial customers.

Any record that produces a large response can be effectively employed in these attacks. Most of the attacks to date though have been using the DNS resource record type 'ANY' (RR type 255). This query, when directed to an authoritative server's zone name, returns all records at the apex of the zone. With DNSSEC, you'll additionally get DNSKEY, RRSIG, and NSEC/NSEC3 records. To give an idea of the scale of the amplification, a DNSSEC-enabled "upenn.edu, ANY" query generates a response that is roughly 88 times as large as the request (a query of about 38 bytes, and a response of 3,344 bytes). The actual amplification ratio of the bits on the wire is less than this because we have to consider the encapsulating headers (L2, IP, and UDP). With Ethernet (14 bytes) and IPv4 (20 bytes) and UDP (8 bytes), a 38-byte DNS query occupys an 80-byte Ethernet frame. The 3,344 byte DNS response packet exceeds the Ethernet MTU and is fragmented into 3 ethernet frames, totalling 3,470 bytes. This yields an amplification ratio of about 40x, so in the absence of rate limiting countermeasures, a 1Mbps stream of query traffic (about 1,500 queries/second) would have produced a 40Mbps stream of traffic directed towards the victim.

Even non-DNSSEC zones produce a quite substantial amplification though, and often it's easily sufficient for an effective attack. From discussion with a number of colleagues at  other institutions, it's clear that non-DNSSEC sites have also been undergoing the same types of attacks and have had to deploy rate limiting countermeasures.

Some people have proposed not answering requests for ANY (and after all ANY was only meant to be a diagnostic tool and not intended for production uses). This might buy time until attackers adapt to using other records. But it could cause collateral damage also. It turns out there is a variety of software that uses ANY for different purposes. For example, the PowerDNS recursor uses it to obtain A and AAAA records from authority servers in one query and response (normally two query/responses would be required).

So, what can be done in the long term? The RRL implementations appear to be working very well at many sites, but attackers will undoubtedly adapt their methods - perhaps by performing more highly distributed attacks across many authoritative servers using a larger set of record types. Some folks at NLnet Labs have written a good paper that discusses some of the issues: Defending against DNS reflection amplification attacks. Also, although RRL has been designed very carefully to minimize collateral damage, there will still be situations in which it might not work very well, eg. when dealing with resolvers behind large scale NATs and CGNs - a problem which might be increasingly common as we approach IPv4 depletion.

There doesn't seem to be much hope (or incentive) for widescale deployment of BCP38 and other methods to reduce the scope of source address forgery.

Ideas have also been proposed in the IETF to enhance the DNS query/response conversation with lightweight authentication cookies, which might thwart most forgery based attacks. See http://tools.ietf.org/html/draft-eastlake-dnsext-cookies-03 for example. But they would require widescale updates to a lot of DNS software to have an effect, and have thus far not gained much traction.

Forcing DNS queries to use TCP is probably a much too heavyweight solution that will impose large costs in terms of DNS response latency and server resource requirements, although the experimental TCP cookie transactions extension (see RFC 6013 - http://tools.ietf.org/html/rfc6013 and this USENIX paper) aims to address some of the issues. It may be necessary to consider a TCP based solution in light of some operational problems observed with UDP and large DNS packets - for example firewalls and other middle boxes that do not pass through IP fragments, or that botch up handling of extension mechanisms like EDNS0 that negotiate the use large UDP/DNS payloads.

Response amplification survey at Internet2 schools


I was interested in knowing what size amplification the ANY query would produce at some other sites, so I wrote a quick program to do this and tabulate the results. I chose the set of 210 Internet2 universities that I already monitor at my dnsstat website. For each zone, I ran an EDNS0 (DO=1) DNS query for ANY at the zone apex via each of the authoritative servers for the zone, and measured the query and response size and the resulting amplification (response_size/query_size). The full set of data can be seen here, but I'll just excerpt some of the entries from the beginning of the file, sorted by descending order of response amplification. As expected the largest amplifications (in the neighborhood of 100x for DNS payloads, 50x for full packets) are all from DNSSEC signed zones. But many non DNSSEC zones produce large amplifications too. The very low amplification ratios for the zones towards the end of the dataset are mostly due to DNS servers that don't understand EDNS0 and return FORMERR (format erorr) responses.

#########################################################################
# DNS response amplification results for ANY query at zone apex.
# Data collected 2013-04-10
# Sorted by descending order of response amplification ratio (last column)
# Amplification is the ratio of the DNS response and request payloads only,
# it doesn't include the encapsulated UDP/IP/L2 etc headers.
# Line Format:
# zone ns_name ns_address query_size response_size amp1 amp2
# amp1 is the ratio of DNS response payload to DNS query payload
# amp2 is the estimated ratio of the entire response packet(s) to request
# packets, assuming an ethernet path MTU.
#########################################################################
ksu.edu nic.kanren.net. 164.113.192.242 36.0 4085.0 113.47 53.99
ksu.edu kic.kanren.net. 164.113.92.250 36.0 4085.0 113.47 53.99
umbc.edu UMBC3.umbc.edu. 130.85.1.3 37.0 4093.0 110.62 53.41
lsu.edu phloem.uoregon.edu. 128.223.32.35 36.0 4016.0 111.56 53.10
lsu.edu bigdog.lsu.edu. 192.16.176.1 36.0 4016.0 111.56 53.10
umbc.edu UMBC5.umbc.edu. 130.85.1.5 37.0 4045.0 109.32 52.80
umbc.edu UMBC4.umbc.edu. 130.85.1.4 37.0 4045.0 109.32 52.80
sdsmt.edu ns5.gratisdns.dk. 85.17.221.46 38.0 4095.0 107.76 52.76
sdsmt.edu ns4.gratisdns.dk. 87.73.3.3 38.0 4095.0 107.76 52.76
sdsmt.edu ns2.gratisdns.dk. 208.43.238.42 38.0 4095.0 107.76 52.76
sdsmt.edu ns1.gratisdns.dk. 109.238.48.13 38.0 4095.0 107.76 52.76
uiowa.edu dns3.uiowa.edu. 128.255.1.27 38.0 4093.0 107.71 52.74
uiowa.edu dns2.uiowa.edu. 128.255.64.26 38.0 4093.0 107.71 52.74
uiowa.edu dns1.uiowa.edu. 128.255.1.26 38.0 4093.0 107.71 52.74
lsu.edu otc-dns2.lsu.edu. 130.39.254.30 36.0 3972.0 110.33 52.54
lsu.edu otc-dns1.lsu.edu. 130.39.3.5 36.0 3972.0 110.33 52.54
ucr.edu adns2.berkeley.edu. 128.32.136.14 36.0 3965.0 110.14 52.45
ucr.edu adns1.berkeley.edu. 128.32.136.3 36.0 3965.0 110.14 52.45
ualr.edu ns4.ualr.edu. 130.184.15.85 37.0 3979.0 107.54 51.96
ualr.edu ns3.ualr.edu. 144.167.5.50 37.0 3979.0 107.54 51.96
ualr.edu ns2.ualr.edu. 144.167.10.1 37.0 3979.0 107.54 51.96
ualr.edu ns.ualr.edu. 144.167.10.48 37.0 3979.0 107.54 51.96
uiowa.edu sns-pb.isc.org. 192.5.4.1 38.0 4013.0 105.61 51.74
sdsmt.edu ns3.gratisdns.dk. 194.0.2.6 38.0 3963.0 104.29 51.11
berkeley.edu ns.v6.berkeley.edu. 128.32.136.6 41.0 4040.0 98.54 50.19
berkeley.edu adns2.berkeley.edu. 128.32.136.14 41.0 4040.0 98.54 50.19
berkeley.edu adns1.berkeley.edu. 128.32.136.3 41.0 4040.0 98.54 50.19
upenn.edu noc3.dccs.upenn.edu. 128.91.251.158 38.0 3866.0 101.74 49.90
berkeley.edu sns-pb.isc.org. 192.5.4.1 41.0 3996.0 97.46 49.66
berkeley.edu phloem.uoregon.edu. 128.223.32.35 41.0 3996.0 97.46 49.66
ksu.edu ns-3.ksu.edu. 129.130.139.150 36.0 3651.0 101.42 48.42
ksu.edu ns-2.ksu.edu. 129.130.139.151 36.0 3651.0 101.42 48.42
ksu.edu ns-1.ksu.edu. 129.130.254.21 36.0 3651.0 101.42 48.42
indiana.edu dns1.iu.edu. 134.68.220.8 40.0 3671.0 91.78 46.30
indiana.edu dns1.illinois.edu. 130.126.2.100 40.0 3671.0 91.78 46.30
indiana.edu dns2.iu.edu. 129.79.1.8 40.0 3655.0 91.38 46.11
mst.edu dns02.srv.mst.edu. 131.151.245.19 36.0 3433.0 95.36 45.63
okstate.edu ns2.cis.okstate.edu. 139.78.200.1 40.0 3599.0 89.97 45.43
okstate.edu ns.cis.okstate.edu. 139.78.100.1 40.0 3599.0 89.97 45.43
mst.edu ns-2.mst.edu. 131.151.247.41 36.0 3417.0 94.92 45.42
mst.edu ns-1.mst.edu. 131.151.247.40 36.0 3417.0 94.92 45.42
mst.edu dns01.srv.mst.edu. 131.151.245.18 36.0 3417.0 94.92 45.42
mst.edu dns03.srv.mst.edu. 131.151.245.20 36.0 3385.0 94.03 45.01
mst.edu ns1.umsl.com. 134.124.31.136 36.0 3369.0 93.58 44.81
ucr.edu ns2.ucr.edu. 138.23.80.20 36.0 3356.0 93.22 44.64
ucr.edu ns1.ucr.edu. 138.23.80.10 36.0 3356.0 93.22 44.64
ksu.edu nic.kanren.net. 2001:49d0:2008:f000::5 36.0 4085.0 113.47 43.58
ksu.edu kic.kanren.net. 2001:49d0:2003:f000::5 36.0 4085.0 113.47 43.58
upenn.edu dns2.udel.edu. 128.175.13.17 38.0 3344.0 88.00 43.38
upenn.edu dns1.udel.edu. 128.175.13.16 38.0 3344.0 88.00 43.38
upenn.edu adns2.upenn.edu. 128.91.254.22 38.0 3344.0 88.00 43.38
upenn.edu sns-pb.isc.org. 192.5.4.1 38.0 3312.0 87.16 42.98
lsu.edu phloem.uoregon.edu. 2001:468:d01:20::80df:2023 36.0 4016.0 111.56 42.88
lsu.edu bigdog.lsu.edu. 2620:105:b050::1 36.0 4016.0 111.56 42.88
[ ... rest of data omitted ...]

Link to full data set.

-- Shumon Huque

Wednesday, April 10, 2013

ISOC ION Panel: Advancing the Network

"I tend to think of IPv6 & DNSSEC both a little bit like global warming ... something that is developing kind of slowly ... they're both inevitable, it's a just a question of how long it's going to take"
   - Paul Mockapetris.

The Internet Society has posted a video (1 hour 7 minutes) of the ISOC ION panel that I moderated on "Advancing the Network - Where We've Been, Where We're Headed" in San Diego on December 11th:

    http://www.internetsociety.org/deploy360/blog/2013/04/video-advancing-the-network-where-weve-been-where-were-headed-ion-san-diego/

The panelists were Ron Broersma (DREN), Paul Ebersman (Infoblox), John Spence (nephos6), and Paul Mockapetris (Nominum; and inventor of the DNS), and the main topics of discussion were IPv6 and DNSSEC. All the presentations and the subsequent Q&A were quite informative and worth listening to.

One correction I should make to something I said in my introductory remarks. My last slide showed some statistics from the SecSpider DNSSEC zone monitoring project. Since that project relies on user submissions and some amount of crawling, by now, it vastly underestimates the amount of DNSSEC deployment. There aggregate numbers are roughly 275,000 signed zones, whereas the actual number is a lot higher. The netherlands Top Level Domain (.NL) for example has more than 1.4 million signed zones underneath it:

    http://xs.powerdns.com/dnssec-nl-graph/

--Shumon.


Tuesday, March 26, 2013

IPv6 and DNSSEC at LOPSA-East Conference

I'm teaching 1/2 day courses on IPv6 and DNSSEC at this year's LOPSA-East conference again, being held in New Brunswick, New Jersey, May 3rd-4th 2013.

The IPv6 course is an updated version of the one I did last year at PICC (PICC has since been renamed LOPSA-East) and elsewhere.

Last year I also did a combined course on DNS and DNSSEC. This year's course is focussed on DNSSEC specifically. This will allow me to go into much more detail on how DNSSEC works, how to configure and deploy it (probably with live examples using BIND), etc. I'll also have more time to discuss DANE and application uses of DNSSEC.

Early bird registration discounts for the conference end April 1st. The full schedule of talks and training programs can be seen at:

http://lopsa-east.org/2013/talks/
http://lopsa-east.org/2013/lopsa-east-training/

--Shumon Huque

Sunday, January 6, 2013

USENIX LISA 2012 courses

I taught two courses at last month's USENIX LISA 2012 conference in San Diego, California. One on IPv6 and another on DNS and DNSSEC. Each had about 60 attendees and they both went very well.

Slides for the IPv6 course are available at:
http://www.huque.com/~shuque/doc/2012-12-IPv6-Tutorial-huque.pdf

Slides for the DNS and DNSSEC course are available at:
http://www.huque.com/~shuque/doc/2012-12-DNS-DNSSEC-Tutorial-huque.pdf

I also participated in the Internet Society's ION conference, where I moderated a panel on "Advancing the Network: Where We've Been and Where We're Headed". Participants on my panel were Ron Broersma (DREN), Paul Ebersman (InfoBlox), John Spence (Nephos6), and Paul Mockapetris (Nominum, and the inventor of the DNS).

A few photos from my trip are available on my Google Plus page.

--Shumon Huque

Friday, November 16, 2012

IPv6 and DNSSEC in San Diego



I'll be in San Diego, California in early December for the USENIX LISA 2012 conference. As part of the conference's training program, I'm teaching two courses - a full day on IPv6, and a half-day on DNS and DNSSEC.

The early registration deadline (cheaper rates) is November 19th. Various discounts are available for members of USENIX and LOPSA.

The IPv6 course "Using and Migrating to IPv6", is on Monday, December 10th. This is a revised and expanded version of the half-day course I taught at last year's LISA conference in Boston, Massachusetts, which was well received. I had originally proposed to do a full-day course last year too, but the organizers at the time felt that they wouldn't be able to attract enough attendees for day long session on IPv6. It turned out that my session was packed and the audience was quite engaged - I even ran over by 30 minutes to cover some advanced topics and almost everyone stayed the extra time. Hopefully I'll get a good turnout this time also.

The DNS and DNSSEC course, is on Tuesday morning, December 11th. This will cover the basic DNS protocol as well as the DNS Security Extensions, including practical configuration examples.

Attendee comments and feedback on my last IPv6 and DNS courses (PICC 12) are available if you're interested. In courses like these with audiences with potentially diverse backgrounds, it's rarely possible to please everyone, but my courses generally get almost uniformly positive reviews (so far at least).

Incidentally, "IPv6 and DNSSEC" is one of the themes of the conference this year. Other sessions in the category include Owen DeLong of Hurricane Electric on "IPv6 Address Planning", Scott Rose of NIST on "Progress of DNSSEC deployment in the federal government", and Roland van Rijswijk of SURFnet on "DNSSEC, what every sysadmin should be doing to keep things working".

Vint Cerf is delivering the keynote on "The Internet of Things and Sensors and Actuators", which will surely discuss IPv6.

There are many other interesting sessions. Check out the agenda for the technical program, the training program, and the workshops.



Co-located with LISA, the Internet Society is also hosting its ION Conference on the afternoon of Tuesday, December 11th.  ION is a conference series organized by ISOC's Deploy360 programme, which provides deployment information on advanced technologies like IPv6, DNSSEC, secure routing, etc.

I'm moderating a panel session titled "Advancing the Network: Where We've Been, Where We're Headed" - joining me on the panel are Ron Broersma (DREN), Paul Ebersman (Infoblox), John Spence (Nephos6), and Paul Mockapetris (inventor of the DNS). Another panel focussed on DNSSEC is being run by Dan York. The full agenda is available.

Registration for the ION conference is free, but seats are limited.

Hope to see some of you in San Diego ..

Shumon Huque

Friday, October 19, 2012

DNSSEC and Certificates

DNSSEC is a system to verify the authenticity of DNS data using public key signatures. With increasing deployment of DNSSEC comes the possibility of applications using the DNS to store and retrieve TLS/SSL certificates in an authenticated manner. And possibly obviating the need for public/global certification authorities (CA), and empowering domain owners to issue their own certificates instead.

For those of you who know me a little better, you're probably aware that I've been griping about the inherent deficiencies of the public CA model for years. Certainly my colleagues at Penn have been subjected to many a diatribe on this topic, and occasionally I've done so more publicly (no pun intended).

In brief, the problems are that applications (such as web browsers) are pre-configured to trust a very large number of global CAs. These CAs have absolutely no constraints on the namespace for which they can issue certificates. Any of them can issue a certificate for any of your services (whether you have a business relationship with them or not). And thus, we have what is tantamount to least common denominator security - the collective security of the system is equivalent to the CA with the weakest security and laxest policies. It gets worse - many of these CAs in turn issue subordinate CA certificates for their customers, again with no namespace constraints.

Another issue is that most of these CAs are capable only of issuing certificates with the most basic capabilities. If you need to deploy a certificate with some more exotic feature (eg. an alternate name form or extension), you're basically out of luck.

Incidentally the PKIX specifications do have the capability of constraining the namespace (see RFC 5280 Section 4.2.1.10: Name Constraints extension), but deploying it in the existing public CA model will require a huge amount of work (both technical and political), including deployment of a hierarchical global CA architecture, as well as enhancing client software to validate name constraints (which practically none do today).

Introducing DANE and TLSA

Recent work in the IETF holds some promise to improve things. RFC 6698 (DANE) defines the new DNS record type TLSA, which holds what is called Certificate Association data. In conjunction with DNSSEC signatures, this will permit better and more secure ways for applications to authenticate certificates. The record can be used in 4 different ways (defined by a "Usage" field):
  • (Usage 0) Specify authorized public CAs - this can prevent certificates signed by CAs that you didn't explicitly specify from being trusted by client software.
  • (Usage 1) Specify which specific server certificate can be trusted. The public CA certificate validation chain all the way to the server certificate must still be validated.
  • (Usage 2) Authorize a new non-public CA (eg. a CA that the domain owner operates itself)
  • (Usage 3) Directly specify the server certificate in the DNS - no certificate chain from a public CA needs to be used. This is called a "Domain issued certificate"
If you're looking for a way to not use public CAs, Usage 3 is probably the one for you.

The owner name of the TLSA record defines the port, transport protocol, and server host name. Here's an example for the HTTPS service (port 443, TCP) at www.example.com:

  _443._tcp.www.example.com. IN TLSA ( 0 0 1
                                d2abde240d7cd3ee6b4b28c54df034b9
                                7983a1d16e8a410e4561cb106618e971 )


The right hand side ("rdata" or resource data) of the TLSA record contains 3 numbers and the raw certificate data in hexadecimal. The 3 numbers are the usage field (previously described), the selector field (which says whether the certificate data takes into account the full certificate or only the subject's public key), and the matching-type field (which says whether the certificate data is the full content that the selector field specifies, or whether it is a cryptographic hash of it). See RFC 6698 for a more detailed description.

Deploying a TLSA record for my website

After seeing several prominent mentions of DANE at this week's ICANN'45 DNSSEC workshop, (see Dan York and Paul Wouters' presentations for example) I felt motivated to install a TLSA record for my website.

Now billions of users with TLSA capable web browsers and DNSSEC validating resolvers will be able to securely reach my website! Okay, that's a slight exaggeration, but perhaps one day, in the not too distant future!

I wanted to see if I could do this using standard tools already available on most UNIX systems. It wasn't too hard. My site already has an SSL certificate issued by StartCom Ltd, but I want to specify that the server certificate can also be authenticated directly without a CA (TLSA Usage type 3). I'll also use a SHA256 hash (match-type 1) of the full certificate (selector 0) as the certificate association data (so the 3 numbers in the rdata will be 3 0 1).

OpenSSL can be used to easily generate the certificate association data portion. We first convert my PEM encoded certificate (in file huque.crt) into the binary DER encoding (openssl x509 -outform DER) and then pipe the output to "openssl sha256". This generates the 256-bit SHA hash represented as a string of 64 hex digits (each hex digit is 4 bits):

  $ openssl x509 -in huque.crt -outform DER | openssl sha256
  (stdin)= 8cb0fc6c527506a053f4f14c8464bebbd6dede2738d11468dd953d7d6a3021f1


The resulting TLSA DNS record will thus look like:

  _443._tcp.www.huque.com. IN TLSA 3 0 1 ( 
       8cb0fc6c527506a053f4f14c8464bebbd6dede2738d11468dd953d7d6a3021f1 )

One quick nsupdate command  (details omitted) and the record was installed in my DNSSEC signed zone. We can check that the record (and its signature) is present with dig:

$ dig +dnssec +noall +answer +multi _443._tcp.www.huque.com. TLSA

_443._tcp.www.huque.com. 893 IN TLSA 3 0 1 (
                                8CB0FC6C527506A053F4F14C8464BEBBD6DEDE2738D1
                                1468DD953D7D6A3021F1 )
_443._tcp.www.huque.com. 893 IN RRSIG TLSA 8 5 900 (
                                20121117202722 20121018192722 14703 huque.com.
                                OJ4b/O7J+Fh3KVDGG8nH9X5dRSXgl3j7/S/PbB1RczcY
                                fYxdu5C9xZSALCz0MgVFW6Kcne74ou5R/Wd+CDKm7mYY
                                Ga28MrtE1boNLuOTa6kOpFcNjW+UYnMFQuc6S8zhU1G7
                                LH0n3TbM3rJS0wH0u6J4NgtiowZS2VlkwPez3D8=
)


(If you're using an older version of BIND/dig that doesn't understand the TLSA record type, you can replace TLSA in the command line above with TYPE52.)

Note: the huque.com zone does not currently have a secure delegation from the .COM TLD. This is mainly because my current DNS registrar (Network Solutions) is incapable of processing DS records, sigh, and I've been too lazy to switch. In the meantime, it has a DLV record published in the DLV registry operated by ISC, so any validating resolver configured to use that registry (and many are) should be able to authenticate records in huque.com.

Not surprisingly, there are already tools out there that make it easy to generate TLSA records. Paul Wouter's hash-slinger package can do it (supposedly integrated into newer Fedora Linux releases). With the "tlsa" program included in that package, we can do this via:

  $ tlsa --create --output rfc --usage 3 --certificate huque.crt www.huque.com
  _443._tcp.www.huque.com. IN TLSA 3 0 1

       8cb0fc6c527506a053f4f14c8464bebbd6dede2738d11468dd953d7d6a3021f1

Obviously DANE needs to be widely adopted by the Internet community for it to be successful, and it's not yet clear how long that will take. And obviously browsers and other application sofware will have to evolve to make use of TLSA records. But preliminary work appears to already be in progress - there are a few browser extensions that try to validate TLSA records for HTTPS websites. I plan to look into those next ..

Incidentally, I'm teaching a half-day DNS and DNSSEC course at the upcoming USENIX LISA conference in San Diego, December 9th-14th. I hope to talk more about DANE and TLSA there.

Shumon Huque

-------------------------------------------------------------------------
Updates:

* (June 2013)  I thought I'd mention I have a small program that generates the resource data for a TLSA record from a certificate file and given parameters. You can find it on github: tlsa_rdata ]

* (September 2013) Randy Bush pointed out to me that OpenSSL versions on a number of operating systems like current versions of Mac OS X and FreeBSD are too old to support sha256. It appears that the sha256 digest command in OpenSSL is fairly new, appearing in version 1.0.1, which is newer than many current operating system distributions provide.  So I'll provide a few alternatives to "openssl 256" for them:

On Mac OS X, you can use shasum (/usr/sbin/shasum):
  openssl x509 -in certfile.crt -outform DER | shasum -a 256
On FreeBSD, you can use sha256 (/sbin/sha256):
  openssl x509 -in certfile.crt -outform DER | sha256
On Redhat Linux, you can use sha256sum (/usr/sbin/sha256sum):
  openssl x509 -in certfile.crt -outform DER | sha256sum

Or you can use Python's hashlib module (going back as far as Python 2.5). I've put up a small program on github called dgst that provides a simple command line utility to generate various cryptographic hashes of files or whatever is fed on standard input:

    https://github.com/shuque/dgst