Wednesday, February 3, 2016

Enhanced getdns data access

getdns is a modern, improved DNS API which provides ease of access to advanced DNS features, including flexible transport options, roadblock (firewall and NAT) avoidance, one-step DNSSEC validation, and a host of other features either not available elsewhere or available through a complicated programming interface that often requires detailed knowledge of DNS internals.  The API was originally specified by Paul Hoffman and is being implemented by a team from NLNet Labs, Verisign Labs, Sinodun Internet Technologies, and No Mountain software.  In addition to the C API, language bindings are also available for Python, PHP, and Node.js.

While the query interface is extremely easy-to-use for the programmer, up until the newest release the returned data could be onerous to process, with the data being represented as dictionaries, lists, and (often) bindatas, none of which are native to C, and which can require a lot of repetitive code to drill down into deeply-nested data structures.  Because of this, we've specified and implemented a new interface in which data elements can be accessed directly using a syntax similar to JSON pointers and to Unix filesystem pathnames.

For example, rather than getting individual answer records by pulling out replies_tree then looping through each reply in the list and for each of those pull out the answer and then go through that list to pull out answer data, getting the answer data is now as simple as 
getdns_dict_get_int(response, "/replies_tree/0/answer/0/type", &type);  

(to retrieve the record type).  The reduction in programmer effort and code size becomes particularly apparent when looping through lists, which you'd do by getting the list length and using snprintf to format the string containing the element to retrieve:

   (void)getdns_list_get_length(replies_tree, &nanswers);  
   for ( i = 0 ; i < (int)nanswers ; i++ ) {  
     snprintf(element, MAXELEM, "/replies_tree/%d/answer/0/type", i);  
     (void)getdns_dict_get_int(response, element, &type);  
          /* do something with the data */  
   }  
I've put up a demo program with complete code on Github, here.

By the way, another extremely handy new feature is pretty-printers for both getdns dicts and getdns lists.  If you ever find yourself scratching your head over the structure of getdns returned data, using these functions to dump the contents to your screen or to a file can be a terrific time-saver.  the functions are getdns_pretty_print_dict() and getdns_pretty_print_list().  For example, I sometimes like to use it as follows:

 printf("%s\n", getdns_pretty_print_dict(response));  

which will print a well-formatted and easy-to-read representation of the data to my screen.

We've added a number of other features not in the original specification.  Some of these provide improvements to the API itself, while others implement new DNS features and transport options.  All of these contribute to the API's appeal to those who need or want access to advanced DNS services to implement application features or as a platform for other DNS-related services.  I'll be posting more about those in the coming weeks.

Sunday, April 27, 2014

Dane-ish

As we were moving getdns towards release we started talking about what would be an interesting thing to do with the library and language bindings to show what could be possible.  With the DANE work in the IETF moving along smartly, Allison suggested doing something sort of dane-ish, by pulling down TLSA records, extracting the X.509 certificate and then the public key from the certificate, and using that key to encrypt some text.  So, that's what we did.

[Please note that we assume some basic familiarity with DNS.]

Sample Python code is here.  We were pretty surprised by how little code was required to do something this fundamentally complicated, but the getdns api and the M2Crypto module take care of the really labor-intensive pieces of work and allow the programmer to remain focused on the application.

First things first: you will need the Python getdns and M2Crypto modules.  (We also import sys for utility reasons).

import getdns
import M2Crypto as m2
from M2Crypto import RSA
import sys

In our example, we've got a known good TLSA certificate coded in, but you can use another:

    tls_name = '77fa5113ab6a532ce2e6901f3bd3351c0db5845e0b1b5fb09907808d._smimecert.getdnsapi.org'

    if len(sys.argv) == 2:
        tls_name = sys.argv[1]

Then, we perform the DNS query.  The first thing we do is get a getdns context.  We set up our extensions, which is just a Python dictionary, to tell the getdns library that we would like DNSSEC validation of the query response.  Note also that rather than using getdns.address,as we did in our previous post, we're using getdns.general.  That allows us to request an arbitrary query (or resource record) type, in this case TLSA.

    c = getdns.context_create()
    extensions = { 'dnssec_return_status' : getdns.GETDNS_EXTENSION_TRUE }
    results = getdns.general(c, tls_name, getdns.GETDNS_RRTYPE_TLSA, extensions=extensions)

The results are returned in the form of a Python dictionary.  A future post will pull that dictionary apart for you, but in the meantime there are a few key elements in the dictionary that you'll need to examine.
The first thing you'll want to do is to make sure that you received a response and that the DNS resolver library considered the query successful.  That information is returned in the 'status' element of the results dictionary.  getdns.GETDNS_RESPSTATATUS_GOOD indicates success.  Other possible values include getdns.GETDNS_RESPSTATUS_NO_NAME (queries returned negative responses),
getdns.GETDNS_RESPSTATUS_ALL_TIMEOUT (there was no answer within the time limit),
and getdns.GETDNS_RESPSTATUS_NO_SECURE_ANSWERS (the programmer requested that only DNSEC-secured records be returned, and at least one response was received but no response was determined to be secured by DNSSEC).

    if results['status'] != getdns.GETDNS_RESPSTATUS_GOOD:
        print 'query status is {0}'.format(results['status'])
        sys.exit(1)

DANE is predicated on the availability of DNSSEC-protected records, to provide assurance that the certificate (or other credential) being returned in the DNS data is legitimate.  So, we look for the first "secure" response in the response dictionary, and we've written a small function to do that for us.  The first thing we do is to make sure that we did indeed receive at least one record.  Because the response dictionary is so deeply nested we make some assignments to make the code both easier to type and easier to read.  Once we've determined that we did receive a response and that the record is valid under DNSSEC, we look for our  certificate by pulling out a record of type getdns.GETDNS_RRTYPE_TLSA (the RR type in our initial query).  When we find it, we return it.

def get_first_secure_response(results):
    replies_tree = results['replies_tree']
    if (not replies_tree) or (not len(replies_tree)) or (not replies_tree[0]['answer']) or (not len(replies_tree[0]['answer'])):
        print 'empty answer list'
        return None
    else:
        reply = replies_tree[0]
        if reply['dnssec_status'] != getdns.GETDNS_DNSSEC_SECURE:
            print 'insecure reply'
            return None                      
        answer = replies_tree[0]['answer']
        record = [ x for x in answer if x['type'] is getdns.GETDNS_RRTYPE_TLSA ]
        if len(record) == 0:
            print 'no answers of type TLSA'
            return None
        return record[0]
   

The next step is to pull out the certificate.  The certificate is returned as a binary blob - not human-readable and not immediately usable by the M2Crypto module.  However, we can load it into a M2Crypto x509 object quite easily, using m2.X509.load_cert_der_string.  From there we pull out the public key (get_pubkey().get_rsa()) and use it to encrypt the string "A chunk of text".  The result of the encryption is another binary blob, so we make it human-readable for printing by base64-encoding it before writing it to the console.

        record = get_first_secure_response(results)
        cert = record['rdata']['certificate_association_data']
        try:
            x509 = m2.X509.load_cert_der_string(cert)
            rsakey = x509.get_pubkey().get_rsa()
            encrypted = rsakey.public_encrypt("A chunk of text", RSA.pkcs1_oaep_padding)
            print encrypted.encode('base64')
        except:
            print 'Error: ', sys.exc_info()[0]
            sys.exit(1)

(If you're not familiar with RSA and/or would like an explanation of OAEP padding, this blog post provides a good overview.)

So that's basically it.  The actual DNS part of this exercise, retrieving a TLSA record, is made pretty trivial through the use of the getdns Python module, and the M2Crypto module also manages to hide an astonishing amount of OpenSSL complexity from the programmer.

Our next post on getdns will take a closer look at what's returned from a query.

Friday, April 25, 2014

Getdns - easy access to advanced DNS features

For quite some time now IETF participants have been slaving over hot keyboards, specifying important DNS features, such as DNSSEC, which aren't seeing the hoped-for uptake.  Part of the problem has been out-of-date or difficult-to-use APIs ("The IETF doesn't do APIs").  So, several years ago Paul Hoffman started working with several application developers to come up with a modern, flexible C API that might help jumpstart DNS-related application development, and ended up publishing getdns.  Allison Mankin, Director of Verisign Labs, put together and funded an open source implementation project, bringing in NLNet Labs, and me.

The C getdns library was published in February of this year, and over the past few months we've been working on getdns bindings for Node.js and Python.  Those bindings were used in a Verisign challenge in the hack battle at The Next Web Europe conference earlier this week, and it seemed to go pretty well.

As a quick introduction, here's what's involved with doing a basic IP address lookup from Python:

import getdns
c = getdns.context_create()
ext = { "return_both_v4_and_v6" :  getdns.GETDNS_EXTENSION_TRUE }
ret  = getdns.address(c, "www.google.com", getdns.GETDNS_RRTYPE_A, ext)

Simply import the module, create a getdns context (an opaque data structure describing the environment within which the resolution will take place, such as root servers, timeouts, whether the resolution should be done recursively or as a stub resolver, etc).  We've added an extension saying that we'd like both IPv4 and IPv6 addresses returned (this is the default with getdns.address() but would need to be set for other query types).  The call returns a somewhat complex dictionary containing everything returned in the query, plus some additional information to make the developer's life a little easier, and I plan to walk through this dictionary in later posts.

But just doing an address resolution isn't that interesting.  Here's an example of checking the DNSSEC status of a given domain name:


#!/usr/bin/python

import getdns, pprint, sys

dnssec_status = {
    "GETDNS_DNSSEC_SECURE" : 400,
    "GETDNS_DNSSEC_BOGUS" : 401,
    "GETDNS_DNSSEC_INDETERINATE" : 402,
    "GETDNS_DNSSEC_INSECURE" : 403,
    "GETDNS_DNSSEC_NOT_PERFORMED" : 404
}


def dnssec_message(value):
    for message in dnssec_status.keys():
        if dnssec_status[message] == value:
            return message

def main():
    if len(sys.argv) != 2:
        print "Usage: {0} hostname".format(sys.argv[0])
        sys.exit(1)

    ctx = getdns.context_create()
    extensions = { "return_both_v4_and_v6" : getdns.GETDNS_EXTENSION_TRUE,
                   "dnssec_return_status" : getdns.GETDNS_EXTENSION_TRUE }
    results = getdns.address(ctx, name=sys.argv[1], extensions=extensions)
    if results["status"] == getdns.GETDNS_RESPSTATUS_GOOD:
        sys.stdout.write("Addresses: ")
        for addr in results["just_address_answers"]:
            print " {0}".format(addr["IPSTRING"])
        sys.stdout.write("\n")

        for result in results["replies_tree"]:
            if "dnssec_status" in result.keys():
                print "{0}: dnssec_status: {1}".format(result["canonical_name"],
                                                       dnssec_message(result["dnssec_status"]))

    if results["status"] == getdns.GETDNS_RESPSTATUS_NO_NAME:
        print "{0} not found".format(sys.argv[1])


if __name__ == "__main__":
    main()

This is a lot more interesting.  The real work takes place in just a few lines.  We add an extension asking for the DNSSEC status to be returned, do the lookup just as we did in the previous example, and then check for the presence of the "dnssec_status" key in the results.  This makes it trivial to check to see whether or not a given DNS record is DNSSEC-protected.

One of the hack battle teams used the equivalent functionality in the Node.js bindings to implement a web interface to a DNSSEC status checker and called it "DNSSEC Name and Shame!"



We're hopeful that making it easy for application developers to check DNSSEC status might help provide some incentive for folks running DNS servers to get their their records signed.

My next post will look at using getdns to implement DANE functionality.  But if you'd like a teaser, check out example code, here.

Monday, April 29, 2013

httpsnow.org has a certificate problem

I'm still undecided on whether or not the Electronic Frontier Foundation are doing this to be funny.  You may know that the EFF have a project to advocate for universal use of TLS for web traffic, in support of user privacy.  They've also released a tool, HTTPS Everywhere, to provide a means for encrypted access to unencrypted websites.  One of their particularly interesting projects is the SSL Observatory, which looks at issued certificates in use on the web and evaluates them for vulnerabilities and for certificate issuance practice at certification authorities.

If you go to their TLS advocacy website, https://www.httpsnow.org, you may see something like this:



Given what the EFF is doing with HTTPS advocacy and its investigations of shoddy CA practices, I found this very surprising.  Unfortunately, however, it's common for there to be problems with web server certs, and that's the case here (i.e. it's not that there was a compromise).

What happened here is that the subject name/subject alt name is ... "*.eff.org".  So, aside from points lost for the use of a wildcard cert, the EFF are using a certificate from what's essentially, for the purpose  of certificate validation, an unrelated and incorrect certificate.  


Sunday, December 23, 2012

Not quite getting the point on trusted third party authentication

I recently learned that a former co-worker is being treated for cancer and has been communicating with friends through Caring Bridge.  So, I went to leave a note in her guest book and was given the option to log in through Facebook Connect or to create a local account.  I opted for the Facebook route (so sue me), and was taken here:


I can see where they may want to allocate local resources, but they don't seem to have quite grasped the delegated authentication thing.  It seems clear to me that there's no benefit to using Facebook to authenticate (unless you see it as a positive to have Caring Bridge post your activities to your wall [incidentally, I always set the visibility of those posts to "me, only"]).

I wonder how common it is to have app developers completely misconstrue the purpose of third-party authentication.

Sunday, May 13, 2012

Crypto tokens/smart cards

It's been awhile.

Anyway, I'm starting a project with smart cards and was quite startled to find that if they're for sale in the US it's hard to find those vendors.  I ended up ordering a couple of Feitian ePass tokens from Gooze.  Gooze is based in France and Feitian is headquartered in China.  I also ordered an Athena PKI token, and they're based in Japan (the token shipped from Japan, as well).

It's not that hard to find smart card readers in the US (to read DOD CAC cards and similar), which suggests that the problem here really is crypto export issues.  But still, why isn't someone in the US selling to American customers?

Wednesday, January 25, 2012

Indoor temperature monitoring

This is not particularly technical but might interest someone.  I live in interior Alaska and during the winter it's not that unusual for the outside temperatures to drop to -40, or lower.  Needless to say, if the indoor heating fails some really terrible things could happen to the plumbing and other household systems,  so using the software that pulls data out of my weather station and uploads it to Weather Underground I was able to throw together a trivial script to send email and an SMS message if the indoor temperature drops below 60F.

I have an Ambient Weather WS-1080, which was the predecessor to this and substantially similar.  It's pretty unsophisticated and not that accurate but it's (mostly) sufficient for my purposes and is absolutely an excellent value.  I use Tee-Boy's WeatherSnoop to pull data off the weather station console (there's a USB port) and upload it to wunderground.com.

WeatherSnoop has an option to write the data out to a sqlite database, and being curious, of course, I turned it on.  You can look at the contents of a database from the sqlite command line, and this is what I saw:

sqlite> .tables
barometricPressure  extraHumidity8      extraTemperature9   rainRate          
barometricTrend     extraHumidity9      forecast            solarRadiation    
dayRain             extraTemperature1   indoorDewPoint      uvIndex           
extraHumidity1      extraTemperature10  indoorHeatIndex     windChill         
extraHumidity10     extraTemperature2   indoorHumidity      windDirection     
extraHumidity2      extraTemperature3   indoorTemperature   windGust          
extraHumidity3      extraTemperature4   monthRain           windSpeed         
extraHumidity4      extraTemperature5   outdoorDewPoint     yearRain          
extraHumidity5      extraTemperature6   outdoorHeatIndex  
extraHumidity6      extraTemperature7   outdoorHumidity   
extraHumidity7      extraTemperature8   outdoorTemperature
sqlite>

The table "indoorTemperature" popped right out as something I could use to monitor the house when I was away from home, so I took a look at the schema and found that there are two values: time and value:

sqlite> .schema indoorTemperature
CREATE TABLE indoorTemperature ( 'time' INTEGER, 'value' FLOAT );
sqlite>

So, I just wrote a script to dump the indoor Temperature table
#!/bin/ksh

WEATHERFILE=/Users/melinda/Documents/weather.db

sqlite3 $WEATHERFILE <<EOF 
.separator " "
select strftime('%Y:%m:%d:%H:%M', time, 'unixepoch', 'localtime'),value from indoorTemperature;
EOF

What this does is use a Unix "here document" to provide scripted commands to the sqlite command line.  .separator " " changes the character used to separate fields on output to a space (by default it's a vertical bar, or pipe character).

The select statement is actually pretty straightforward and kind of inefficient and brainless, to be honest.  Basically I'm selecting every record and formatting the time for printing.

This little routine is invoked by the real script, which looks like this:

#!/bin/ksh

CURTMP=`/Users/melinda/lib/curindoortemp | tail -1 | cut -d" " -f2`
ADDR=0000000000@msg.acsalaska.com,xxxxxxx@gmail.com
THRESHHOLD=60

if [ $CURTMP -lt $THRESHHOLD ]
then
        Mail $ADDR <<EOF
        Temperature is $CURTMP
EOF
fi

The curindoortemp script is the one described above.  I grab the last record and print the second field, which is the temperature.  If it's less than the thresshold value (in this case, 60 degrees Fahrenheit) I send off email to the addresses listed in the ADDR field.  Obviously it would be far better programming practice to have the curindoortemp script just return the temperature from the last record.