Welcome to OpenINTEL

OpenINTEL is a joint project of the University of Twente, SURFnet and SIDN Labs. The goal of our project is to perform a comprehensive active measurement of the global Domain Name System (DNS), to track long-term developments on the Internet.

Through this website, we provide information on our project, how our measurement works and how you can reach us if you have questions about using the data for your research.

Latest news

More news items can be found on the News tab.


Contact in case of problems

In case you are a DNS operator and you think our measurement is impacting your infrastructure, please go to our Problems page, where you can read more about what traffic you can expect to see from us, how you can contact us and if necessary, how you can block our traffic.

Background

Introduction

On this page, we provide concise technical background information about our measurement. This information is targeted at DNS operators and academic researchers.

Measurement Goal

The goal of the OpenINTEL measurement platform is to capture daily snapshots of the state of large parts of the global Domain Name System. Because the DNS plays a key role in almost all Internet services, recording this information allows us to track changes on the Internet, and thus its evolution, over longer periods of time. By performing active measurements, rather than passively collecting DNS data, we build consistent and reliable time series of the state of the DNS.

What do we measure?

The measurement system will send a fixed set of DNS queries, which it sends once every 24 hours for every second-level domain in a TLD. This set of queries currently consists of the following queries:

  • SOA
  • NS(3)
  • A (1)
  • AAAA (1)
  • MX(3,4)
  • TXT
  • DNSKEY
  • DS
  • NSEC3(2)
  • CAA
  • CDS(5)
  • CDNSKEY(5)

(1) These queries are also sent for the www label (e.g. www.example.com).
(2) We send a query for a non-existent domain name to record authenticated denial-of-existence data, but only for DNSSEC-signed domains.
(3) We resolve the A and AAAA records for these records in a separate infrastructure measurement.
(4) We resolve associated TLSA records for ports 25, 465 and 587 in a separate infrastructure measurement.
(5) We only resolve these records for DNSSEC-signed domains for which at least a DNSKEY or DS record exists.

All response records, including full CNAME expansions and RRSIG signature records are stored.

What can DNS operators expect?

As a DNS operator, you can expect that our measurement system will send you 11 queries per day per domain you operate, and in case the domain is DNSSEC-signed, 12 queries. The system has been designed to distribute measurements over time, to reduce the impact the measurement has on busy name servers that are authoritative for large numbers of domains. We regularly inspect flow information about our measurements to monitor query rates. In general, only if you operate a very busy name server in terms of numbers of domains, should our measurement be visible in statistics. If you are a DNS operator and you feel our measurement is impacting your infrastructure, please read on.

What should I do if the measurement impacts my infrastructure?

Please go to our Problems page for information on how to contact us and how you can block our traffic if necessary.

What will the data be used for?

The data collected by the OpenINTEL measurement platform will only be used for academic research and may not be used for commercial purposes. Read more about getting access to data for research on our Data Access page.

Where can I find more detailed information?

If you would like to learn more about how OpenINTEL works, please refer to our Papers page, which contains links to academic papers on the design and use of OpenINTEL.

Data access

The data collected by the OpenINTEL platform has many applications in network and network security research. For this reason, we make our data available to academic researchers upon request. As using the data requires specific knowledge and may require specific analysis infrastructure, we invite fellow researchers to contact us about this, so we can discuss your research needs.

We note that there are restrictions on the use of the data collected by the OpenINTEL platform, which are partly a result of the contractual obligations we have with the gTLD and ccTLD operators who have provided access to the zone files for the top-level domains for which they are responsible. One key restrictions is that we cannot provide data for commercial purposes, so please do not contact us if you want to make commercial use of the data we collect. We may also have to impose restrictions about third party release of research data derived from data collected by the OpenINTEL platform. This will be discussed on a per request basis.

Open data

We release two of our datasets as open data. The first dataset is for the Alexa top 1M, the second dataset is for those ccTLDs that release their DNS zone files as open data (currently those are .se and .nu). You can download these datasets below. By downloading the data, you agree to keep to our terms of use.

(click here to open our open access data portal in a separate tab or window)

Current coverage

Generic TLDs

Our measurement currently covers the following generic top-level domains (TLDs):

Generic TLD Measurement start date Status
.com 2015-02-20 Measurement active
.net 2015-02-20 Measurement active
.org 2015-02-20 Measurement active
.info 2016-04-06 Measurement active
.mobi 2016-04-06 Measurement active
new gTLDs (*) 2016-04-01 Measurement active
.gov and .fed.us (**) 2017-05-01 Measurement active

(*) Our measurement covers almost all new gTLDs available through ICANN's Centralized Zone Data Service, which is currently around 1200 new gTLDs.
(**) The measurement only covers US Federal domain names included in the list obtained from the US Federal Government open access API.

Country-code TLDs (ccTLDs)

Our measurement currently covers the following country-code top-level domains (ccTLDs):

Country-code TLD Country or territory Registry operator Measurement start date Status
.nl The Netherlands SIDN 2016-02-09 Measurement active
.se Sweden IIS 2016-06-07 Measurement active
.nu Niue IIS 2016-06-07 Measurement active
.ca Canada CIRA 2016-07-07 Measurement active
.fi Finland Ficora 2016-11-23 Measurement active
.at Austria NIC.at 2017-01-04 Measurement active
.dk Denmark DK Hostmaster 2017-02-06 Measurement active
.ru Russian Federation TCI 2017-06-18 Measurement active
.рф (.xn--p1ai) Russian Federation TCI 2017-06-18 Measurement active

This information was last updated on 2017-06-18

In case of problems

If you are a DNS operator and you believe your authoritative name servers may be experiencing problems due to our measurements, please consult the information below on what you can expect to see from us, how you can contact us, and as a last resort how you can block our traffic.

We thank you in advance for working with us to resolve any issues, and apologise in advance if we have inadvertently caused problems through our measurement.

What can DNS operators expect?

As a DNS operator, you can expect that our measurement system will send you 11 queries per day per domain you operate, and in case the domain is DNSSEC-signed, 12 queries. The system has been designed to distribute measurements over time, to reduce the impact the measurement has on busy name servers that are authoritative for large numbers of domains. We regularly inspect flow information about our measurements to monitor query rates. In general, only if you operate a very busy name server in terms of numbers of domains, should our measurement be visible in statistics. If you are a DNS operator and you feel our measurement is impacting your infrastructure, please read on.

Note: our traffic originates from a single IPv4 /24 prefix and from a single IPv6 /64 prefix. This information may prove helpful if you decide you need to block our traffic (see below).

What should I do if the measurement impacts my infrastructure?

As we believe in responsible network measurement practices, we have taken care in designing our measurement such that it should not impact DNS servers adversely. Nevertheless, it is not up to us to judge if our measurement impacts your infrastructure as a DNS operator. Therefore, if you think our measurement is impacting your infrastructure adversely, please contact us so we can try to resolve the problem. We would prefer to include the domains you operate in our measurement if possible, but of course without causing problems. If you decide to contact us, it would help if you can provide us with the following information:

Blocking our traffic

If you decide you need to block our traffic, we would highly appreciate it if you contact us about this. As mentioned above, our traffic always originates from a single IPv4 /24 prefix and a single IPv6 /64 prefix. If the traffic that is causing you problems does not follow this pattern it is most likely not coming from us. Upon request, we can disclose the specific prefixes from which we send traffic through private communication. We ask for your understanding for not publishing this information publicly.

Response time

We are in the Central European Timezone (UTC+1 in winter, UTC+2 in summer). We try to answer any request concerning problems within one business day. If you need our urgent attention, please include "[URGENT]" in the subject line of any e-mail you send us.

More information

You can read more about how our measurement works on our Background page.

OpenINTEL is a collaboration between the University of Twente, SURFnet and SIDN.

For more information on the OpenINTEL project, please contact one of the following people:

Roland van Rijswijk-Deij <> (University of Twente, SURFnet)
Mattijs Jonker <> (University of Twente)
Anna Sperotto <> (University of Twente)

Listed below are papers that have used data from the OpenINTEL project. Papers are list in order of publication from most recent to oldest.

2018

2017

2016

2015

For more detailed information on the OpenINTEL platform, please consult one of the references below.

Last updated on January 8, 2018