Blog

How to Extract Domain Names from VeriSign’s .COM Zone File

For those of you not familiar with it, the zone file is a file generated daily that lists all of the active domain names and corresponding name servers for a particular Top Level Domain (TLD) like .COM, .NET, etc.

VeriSign managers the .COM zone file and in the last post I explained how to obtain access to download VeriSign’s .COM zone file. In this tutorial I’ll explain how to parse the zone file to extract just the domain names and strip out everything else.

Working with the zone file

The size of the uncompressed zone file varies by day but you can figure it will be somewhere around 8.5 GB:

Any attempt you make to open the entire zone file in a text editor will likely crash your computer, so you have to resort to working with it via the command line (the rest of this tutorial assumes you are using a Unix-like machine such as Mac OS X or Linux).

For this tutorial we’re going to work with only the first 100 lines of the zone file because it makes it easy to show you what’s going on step by step. These steps would also work on the entire zone file, though it would take significantly longer to execute each of the commands.

1. Navigate to the location of the zone file

For this tutorial we’re only going to work with the first 100 lines of the zone file because it’s easier to show you what’s going on with each command we run. If you’d like to follow along, you can download the truncated zone file here.

After you’re in the folder with the zone file (or zone file example) you should be able to run ls -l and see the zone file:

terminal. $ ls l // rw-r—r 1 matt staff 4973 Oct 4 12:06 com.zone.example.txt

You can confirm the number of lines in the zone file by running wc -l:

terminal. $ wc -l com.zone.example.txt // 100 com.zone.example.txt

You can view the contents of the example zone file by opening it up in a text editor (again, don’t try this on the full zone file) or by running cat:

terminal. $ cat com.zone.example.txt

Here’s what the contents of the example zone file look like:

textfile. ; The use of the Data contained in Verisign Inc.‘s aggregated//; .com, and .net top-level domain zone files (including the checksum//; files) is subject to the restrictions described in the access Agreement//; with Verisign Inc.////$ORIGIN COM.//$TTL 900//@ IN SOA a.gtld-servers.net. nstld.verisign-grs.com. (// 1349069930 ;serial// 1800 ;refresh every 30 min// 900 ;retry every 15 min// 604800 ;expire after a week// 86400 ;minimum of 15 min// )//$TTL 172800// NS A.GTLD-SERVERS.NET.// NS G.GTLD-SERVERS.NET.// NS H.GTLD-SERVERS.NET.// NS C.GTLD-SERVERS.NET.// NS I.GTLD-SERVERS.NET.// NS B.GTLD-SERVERS.NET.// NS D.GTLD-SERVERS.NET.// NS L.GTLD-SERVERS.NET.// NS F.GTLD-SERVERS.NET.// NS J.GTLD-SERVERS.NET.// NS K.GTLD-SERVERS.NET.// NS E.GTLD-SERVERS.NET.// NS M.GTLD-SERVERS.NET.//COM. 86400 DNSKEY 256 3 8 AQOl6m3T1q7fQhwJYFfWJO/IvVmtfI2Eg2wX4UR9jcl/qaTiMp+7Kx7baGOsPvZwX4lVGYWif955l4lLh/VnnNJvjDxBWVQcDrH3cHzFAaq9QXZPcEk7UyTOBL1piVpB2dqJzbO2bH9XGFiOXPUj3nhQ7mxvW0bgRiKv9Qah/7NH2w//COM. 86400 DNSKEY 257 3 8 AQPDzldNmMvZFX4NcNJ0uEnKDg7tmv/F3MyQR0lpBmVcNcsIszxNFxsBfKNW9JYCYqpik8366LE7VbIcNRzfp2h9OO8HRl+H+E08zauK8k7evWEmu/6od+2boggPoiEfGNyvNPaSI7FOIroDsnw/taggzHRX1Z7SOiOiPWPNIwSUyWOZ79VmcQ1GLkC6NlYvG3HwYmynQv6oFwGv/KELSw7ZSdrbTQ0HXvZbqMUI7BaMskmvgm1G7oKZ1YiF7O9ioVNc0+7ASbqmZN7Z98EGU/Qh2K/BgUe8Hs0XVcdPKrtyYnoQHd2ynKPcMMlTEih2/2HDHjRPJ2aywIpKNnv4oPo///COM. 86400 NSEC3PARAM 1 0 0 -//COM. 86400 RRSIG NSEC3PARAM 8 1 86400 20121006041758 20120929030758 47783 COM. chypT55+F8iGvkLS2TVSiqonms7mRNjDe2g560bqSulngD7z4y+4qsz13ZOGY4yrM9lBOfGYtNxpai3Q9TAT9n0X2L/3cJDY/xyA5LFSC0ilRvm+zr41d5TRwuf/GMdj7pfN2w6IoSxYckgPczqWHG2yOpX6EPXuIPW5E2L8GZU=//COM. RRSIG NS 8 1 172800 20121006041758 20120929030758 47783 COM. hfdySh/hHeA0zNLcbLQMNtRsXcOVKzH7vGED0t8IbkdaOTeuSFi0E8vXMVUJDjK9hlVYsCa4bE5wh5X61pIKkI9SjyCDjUK92ZpG/2+rtHeYWRbREAMpgcZ4FAySSknskHOnkUa4c/0tA9ZOJ0AkNzxztUr+KinlC+Co8rp5aGg=//COM. 900 RRSIG SOA 8 1 900 20121008053850 20121001042850 47783 COM. odeDdoJS/JVKOMNcdDd4Oh8MnY2DoKobagNU44AKjYE9GuQ3sBgbXmyH3JOrS6a7iBmFexN6UAdLSNcCozOO0Ta51WQFcuJhbvZwhXNrjOH50pkcG7Xw9pzwlOrftj9R7pHwCDEagZp20GGtbGATf946D6CCUJSBmtZ8pqoEu7s=//COM. 86400 RRSIG DNSKEY 8 1 86400 20121005182533 20120928182033 30909 COM. nPBzPp1A3EBgwjf3IrrYVgVh0YcVqdd6YKQ4CeraP5vK8nIyUMqGMnLc2ykA/BWb8AtAdg6KiOVsXl+4dkkqijccbt8mEzUZ6aD3Gd1IT13K5uDq4tjhxaQTRkloZU1TC4FfRhe5DHQSHzTmOWn9ClqonMa2FeNaf9rlsaNCaWq4fctndbPhuhuN0m9EKSh0So8WhM/5wZqjsie9+S2yBPsxakXWTA3zwxR7y9sqfabfmH+KmrQRF2lCXxhF//of4zp3VLpG9UK1kS/4mQTdm8kNRzfgNgCKo1ejS4uMj5g0rS6n5aZvk8PfeVbBlhnVb3oDRImz/RIhZJ1×0w3kzA//ENERCONTECHNOLOGIES NS NS1.BIZ.RR//ENERCONTECHNOLOGIES NS NS2.BIZ.RR//SELFDRIVECARRENTAL NS NS9.IZP//SELFDRIVECARRENTAL NS IZA.HOSTING.DIGIWEB.IE.//SELFDRIVECARRENTAL NS NS8.FOR-SALE-IF-THEPRICE-IS-RIGHT//NANCYVRAINE NS NS1.IMCONLINE.NET.//NANCYVRAINE NS NS2.IMCONLINE.NET.//SELFDRIVECARRENTAL NS NS9.IZP//SELFDRIVECARRENTAL NS IZA.HOSTING.DIGIWEB.IE.//SELFDRIVECARRENTAL NS NS8.FOR-SALE-IF-THEPRICE-IS-RIGHT//WORLDDATASOURCE NS NS01.DOMAINCONTROL//WORLDDATASOURCE NS NS02.DOMAINCONTROL//SAUDIPHOTOGRAPHERS NS NS1.R4L//SAUDIPHOTOGRAPHERS NS NS2.R4L//MERCKCHOICE NS CBRU.BR.NS.ELS-GMS.ATT.NET.//MERCKCHOICE NS CMTU.MT.NS.ELS-GMS.ATT.NET.//ENVIRONMENTALSCHOOLS NS PSNS01.PAULSMITHS.EDU.//ENVIRONMENTALSCHOOLS NS PSNS02.PAULSMITHS.EDU.//EASTHAMPTONHOMES NS BUY.INTERNETTRAFFIC//EASTHAMPTONHOMES NS SELL.INTERNETTRAFFIC//AMERICASHOMEBUILDER NS BUY.INTERNETTRAFFIC//AMERICASHOMEBUILDER NS SELL.INTERNETTRAFFIC//BOVINUS NS C3P0.CBFENTERPRISES//BOVINUS NS R2D2.CBFENTERPRISES//CONSTELLATIONCOLLEGE NS NS1.SEDOPARKING//CONSTELLATIONCOLLEGE NS NS2.SEDOPARKING//DOCHERTYCONSULTING NS NS1.VERINOTE.NET.//DOCHERTYCONSULTING NS NS3.VERINOTE.NET.//SONOMETRICS NS NS35.WORLDNIC//SONOMETRICS NS NS36.WORLDNIC//UNLIMITEDDISCOUNTPHONECALLS NS DNS1.NAME-SERVICES//UNLIMITEDDISCOUNTPHONECALLS NS DNS2.NAME-SERVICES//UNLIMITEDDISCOUNTPHONECALLS NS DNS3.NAME-SERVICES//UNLIMITEDDISCOUNTPHONECALLS NS DNS4.NAME-SERVICES//UNLIMITEDDISCOUNTPHONECALLS NS DNS5.NAME-SERVICES//FREILAND NS NS1.FABULOUS//FREILAND NS NS2.FABULOUS//KUMANET NS UNS01.LOLIPOP.JP.//KUMANET NS UNS02.LOLIPOP.JP.//SANGYOSHIEN NS NS55.WORLDNIC//SANGYOSHIEN NS NS56.WORLDNIC//JONATHANCHARLESNOVAK NS DNS077.A.REGISTER//JONATHANCHARLESNOVAK NS DNS030.B.REGISTER//JONATHANCHARLESNOVAK NS DNS030.C.REGISTER//JONATHANCHARLESNOVAK NS DNS010.D.REGISTER//HQSINGAPORE NS NS41.DOMAINCONTROL//HQSINGAPORE NS NS42.DOMAINCONTROL//PANASOURCE NS F1G1NS1.DNSPOD.NET.//PANASOURCE NS F1G1NS2.DNSPOD.NET.//PRIVATESAUNAS NS NS.BUYDOMAINS//PRIVATESAUNAS NS THISDOMAINFORSALE//BARBARASTREISAND NS NS1.LAMEDELEGATION.NET.//BARBARASTREISAND NS NS2.LAMEDELEGATION.NET.//MONICAMAGNETTI NS NS21.DOMAINCONTROL//MONICAMAGNETTI NS NS22.DOMAINCONTROL//IGUANAWORLD NS DNS1.TNIB.DE.//IGUANAWORLD NS DNS2.TNIB.DE.//IGUANAWORLD NS DNS3.TNIB.DE.//PERFECTDAYSTUDIOS NS NS2.DYNADOT//PERFECTDAYSTUDIOS NS NS1.DYNADOT//SVCROSS NS NS1.ZONEEDIT//SVCROSS NS NS5.ZONEEDIT//EBEIJING NS NS1.PEER1.NET.//EBEIJING NS NS2.PEER1.NET.//NASHSATTERFIELD NS HOME.GIS.NET.

For the purposes of this tutorial you can ignore the first 35 lines or so. The first domain name the file contains is EnerconTechnologies:

textfile. ENERCONTECHNOLOGIES NS NS1.BIZ.RR // ENERCONTECHNOLOGIES NS NS2.BIZ.RR // SELFDRIVECARRENTAL NS NS9.IZP // SELFDRIVECARRENTAL NS IZA.HOSTING.DIGIWEB.IE. // SELFDRIVECARRENTAL NS NS8.FOR-SALE-IF-THEPRICE-IS-RIGHT // …

Notice that there’s one entry for each name server associated with the domain name.

You can confirm the name server’s are correct by running whois on a domain name and checking whether the name servers are are identical:

terminal. $ whois ENERCONTECHNOLOGIES.COM////Whois Server Version 2.0////Domain names in the .com and .net domains can now be registered//with many different competing registrars. Go to http://www.internic.net for detailed information.//// Domain Name: ENERCONTECHNOLOGIES.COM// Registrar: NETWORK SOLUTIONS, LLC.// Whois Server: whois.networksolutions.com// Referral URL: http://www.networksolutions.com/en_US//Name Server: NS1.BIZ.RR.COM// Name Server: NS2.BIZ.RR.COM// Status: clientTransferProhibited// Updated Date: 03-mar-2012// Creation Date: 03-mar-1999// Expiration Date: 03-mar-2022

Extracting just the domain names

In order to extract just the domain names we’ve got to run a series of commands so that all that’s left when we’re done is a list of the domain names. We’ll do this step by step, though you could easily pipe (|) these commands together to achieve the same result.

Notice that the line with domain name and name server is formatted consistently: it’s the domain name, then a space, then the name server. If we want to extract just the domain names, then we can run a command that will extract everything before the first space. To do this, we use the awk command and send the output to first.com.zone.example.txt:

terminal. $ awk ‘{print $1}’ com.zone.example.txt > first.com.zone.example.txt

If you examine the resulting first.com.zone.example.txt you’ll notice a much cleaner output:

textfile. ;//;//;//;////$ORIGIN//$TTL//@//1349069930//1800//900//604800//86400//)//$TTL//NS//NS//NS//NS//NS//NS//NS//NS//NS//NS//NS//NS//NS//COM.//COM.//COM.//COM.//COM.//COM.//COM.//ENERCONTECHNOLOGIES//ENERCONTECHNOLOGIES//SELFDRIVECARRENTAL//SELFDRIVECARRENTAL//SELFDRIVECARRENTAL//NANCYVRAINE//NANCYVRAINE//SELFDRIVECARRENTAL//SELFDRIVECARRENTAL//SELFDRIVECARRENTAL//WORLDDATASOURCE//WORLDDATASOURCE//SAUDIPHOTOGRAPHERS//SAUDIPHOTOGRAPHERS//MERCKCHOICE//MERCKCHOICE//ENVIRONMENTALSCHOOLS//ENVIRONMENTALSCHOOLS//EASTHAMPTONHOMES//EASTHAMPTONHOMES//AMERICASHOMEBUILDER//AMERICASHOMEBUILDER//BOVINUS//BOVINUS//CONSTELLATIONCOLLEGE//CONSTELLATIONCOLLEGE//DOCHERTYCONSULTING//DOCHERTYCONSULTING//SONOMETRICS//SONOMETRICS//UNLIMITEDDISCOUNTPHONECALLS//UNLIMITEDDISCOUNTPHONECALLS//UNLIMITEDDISCOUNTPHONECALLS//UNLIMITEDDISCOUNTPHONECALLS//UNLIMITEDDISCOUNTPHONECALLS//FREILAND//FREILAND//KUMANET//KUMANET//SANGYOSHIEN//SANGYOSHIEN//JONATHANCHARLESNOVAK//JONATHANCHARLESNOVAK//JONATHANCHARLESNOVAK//JONATHANCHARLESNOVAK//HQSINGAPORE//HQSINGAPORE//PANASOURCE//PANASOURCE//PRIVATESAUNAS//PRIVATESAUNAS//BARBARASTREISAND//BARBARASTREISAND//MONICAMAGNETTI//MONICAMAGNETTI//IGUANAWORLD//IGUANAWORLD//IGUANAWORLD//PERFECTDAYSTUDIOS//PERFECTDAYSTUDIOS//SVCROSS//SVCROSS//EBEIJING//EBEIJING//NASHSATTERFIELD

Not bad, but there’s a lot of duplicates because domain names are listed once for each name server, so let’s clean the file up a bit by removing duplicates and sorting the results alphabetically:

terminal. $ sort -u first.com.zone.example.txt > sorted_and_unique.com.zone.example.txt

Here we use the sort command with the u switch to sort the file and remove the duplicates.

The new sorted_and_unique.com.zone.example.txt is looking pretty good:

textfile. //$ORIGIN//$TTL//)//1349069930//1800//604800//86400//900//;//@//AMERICASHOMEBUILDER//BARBARASTREISAND//BOVINUS//COM.//CONSTELLATIONCOLLEGE//DOCHERTYCONSULTING//EASTHAMPTONHOMES//EBEIJING//ENERCONTECHNOLOGIES//ENVIRONMENTALSCHOOLS//FREILAND//HQSINGAPORE//IGUANAWORLD//JONATHANCHARLESNOVAK//KUMANET//MERCKCHOICE//MONICAMAGNETTI//NANCYVRAINE//NASHSATTERFIELD//NS//PANASOURCE//PERFECTDAYSTUDIOS//PRIVATESAUNAS//SANGYOSHIEN//SAUDIPHOTOGRAPHERS//SELFDRIVECARRENTAL//SELFDRIVECARRENTAL//SONOMETRICS//SVCROSS//UNLIMITEDDISCOUNTPHONECALLS//WORLDDATASOURCE

The last problem is that there are a number of lines left over that can’t possibly be domain names because they contain invalid characters such as $ORIGIN and COM.. We’ll use egrep (or grep -e) to extract only the lines that are valid domain names:

terminal. $ egrep ‘^[A-Z0-9]([A-Z0-9-]{0,61}[A-Z0-9])?$’ //sorted_and_unique.com.zone.example.txt > domains.com.zone.example.txt

In case you’re curious, that regular expression finds strings that:

  1. Start and end with a letter of a number (we can just look at uppercase letters because that’s how all the domain names in the zone file are formatted)
  2. Contain letters, numbers, or dashes in between
  3. Are 1 to 63 characters in length

At last, we have a file we can work with:

textfile. 1349069930//1800//604800//86400//900//AMERICASHOMEBUILDER//BARBARASTREISAND//BOVINUS//CONSTELLATIONCOLLEGE//DOCHERTYCONSULTING//EASTHAMPTONHOMES//EBEIJING//ENERCONTECHNOLOGIES//ENVIRONMENTALSCHOOLS//FREILAND//HQSINGAPORE//IGUANAWORLD//JONATHANCHARLESNOVAK//KUMANET//MERCKCHOICE//MONICAMAGNETTI//NANCYVRAINE//NASHSATTERFIELD//NS//PANASOURCE//PERFECTDAYSTUDIOS//PRIVATESAUNAS//SANGYOSHIEN//SAUDIPHOTOGRAPHERS//SELFDRIVECARRENTAL//SELFDRIVECARRENTAL//SONOMETRICS//SVCROSS//UNLIMITEDDISCOUNTPHONECALLS//WORLDDATASOURCE

The one thing that’s inaccurate about this list are the numbers which are not actually domain names though they are listed as such at the top of the list. You could remove these, though the resulting list is more than accurate enough to analyze and use at this point (and all of those numbers do have corresponding .coms except for 1349069930 anyway).

Hope you found this tutorial useful. If you have any questions or notice anything that can be done more efficiently please drop me a note matt@leandomainsearch.com.

Thanks!

Shaun Gilchrist on Naming His jQuery Plugin “Bling”

Shaun Gilchrist, a Lisp programmer who also works with Javascript, recently released a popular jQuery plugin called Bling which makes it easier to dynamically create HTML fragments via JavaScript.

Why’d he choose the name Bling? Here’s Shaun:

I wanted a short name which related to jQuery. The $ prefix is well known in the context of jQuery/web programming as well as the hip hop lexicon, but there it would be known as bling. This was the swagger I desired. I also wanted something playful/non serious to indicate it is clearly a hack and not to be taken too seriously.

When you’re trying to come up with a name it often helps to look at words that are associated with your topic. In this case, Shaun wanted a name that related to jQuery, jQuery uses the dollar sign symbol heavily, a dollar sign is associated with money, which is related to the term bling.

“Bling” also helps describe the personality of his plugin: it was a quick project that’s not intended to take over the world or anything like that. Contrast that with a name like “Fortune”, a word that’s also related to money but carries a much heavier mental association.

Not bad for a playful hack 🙂

How to Get Access to the Official VeriSign .COM Zone File

Did you know that VeriSign publishes a list of all the active .COM and .NET domain names and their respective name servers in what’s known as a zone file? They do and actually offer access to the public so that folks can download and analyze the contents on their own.

While the application process to obtain access might seem daunting at first, it’s actually not that bad. I recently applied and was approved for access only five days after I faxed in the paperwork. This post is about the application process and what you need to do to download the zone files for yourself.

The TLD Zone File Access Program

VeriSign has a program where you can apply for FTP access to download the zone files for the Top Level Domains (TLD) that they oversee. There’s one access program for accessing the .COM and .NET zone files and another for accessing the .NAME zone file. In this post I’ll focus on .COM/.NET access, though the .NAME application process is identical.

You can read the details of the program at the TLD Zone File Access Program homepage, though I’ll go step by step through actual process below.

1. Download the appropriate forms

There are two forms that you have to fill out and fax over to VeriSign in order to get access to the zone files:

  1. “ .com .net TLD Zone File Access Agreement”:http://www.verisigninc.com/assets/com-tld-zonefile-access-agreement.doc
  2. com .net TLDZ User Request

Download both of these onto your computer.

2. Complete the Access Agreement

The Access Agreement, at 5 pages, is the longer of the two forms though the majority of it is just terms and conditions which you must agree to so that VeriSign knows you’re going to use your zone file access responsibly.

Here’s an example of how to fill out the user information section:

A few notes:

  • Take some time to read the terms and make sure you understand them so you avoid trouble down the road.
  • If you don’t have your own company I would just put N/A and submit it without a company name. If you get disapproved because of this please let me know.
  • If you don’t have your own fax number just put N/A (I did and my application was still approved).
  • For host machine name I used my computer name and my public IP address, though changing both after the fact did not seem to affect my ability to access the zone file FTP site. If you’re not sure what your computer name is you can read up on it on Google and to find out your IP address check out the aptly name MyIPAddress.com

Here’s an example of how to fill out the last section:

When you’re done filling out the agreement electronically, print it out and sign it.

2. Complete the User Request

The second form is short and is where you explain why you want zone file access:

When you’re done print out the completed user request.

3. Fax both documents to VeriSign

If you have access to a fax machine, send both documents to 1-703-948-3978.

If you do not have a fax machine, I highly recommend checking out HelloFax, which lets you upload and fax up to 5 documents for free.

4. Wait while your application is processed

I faxed both documents via HelloFax on a Monday and received a call from a friendly female VeriSign employee that Saturday. She gave me my FTP user name and password and told me I was good to go (and I also received a signed hardcopy of my user agreement about a week after that).

5. Accessing the zone file FTP server

With your user name and password, you can now log in to VeriSign’s zone file FTP server at rz.verisign-grs.com. I use Transmit on my Mac, but any FTP software should do.

Once you log in you should see something like this:

The .COM zone file is named com.zone.gz which you are now free to download and unzip using gunzip. Be forewarned though, the compressed .com zone file 2.04 GB which uncompressed is a whopping 8.48 GB.

Here’s what the beginning of the zone file should look like:

In the next post I’ll talk about how to go about intelligently analyzing this gargantuan file.

If you run into any snags during your application process please drop me a note (matt@leandomainsearch.com) and let me know so I can keep this tutorial updated.

Enjoy!

Lean Domain Search Launches New Business Name Generator

I’m thrilled to announce that Lean Domain Search now has a domain name generator specifically geared towards professional businesses. This is premium (paid) product and it’s not for everyone so its important to understand what it is and whether or not it’s something you can benefit from.

Business names vs websites names

Let’s pretend for a moment that you’re interested in starting your own construction company.

If you use the normal Lean Domain Search domain name generator, the results would include Constructionly, Construction Fusion, Construction Bit, and Construction Jet. Would you name your construction company any of these names? Probably not, and it’s not just a matter of needing more results. These are fundamentally not the right structure for naming something like a construction company. If you wanted to create a website about construction, then maybe something like Constructionly would work, but it’s not something you’d want to throw on the side of your office building or business cards if you were running an actual construction company.

If you look at how actual construction companies name themselves the names look more like Turner Construction, Holder Construction, Walsh Construction, and Gotham Construction.

Like the majority of other offline businesses, these domain names follow a common format: <word or phrase> <industry>.

The new business name generator creates business names in just this format. If you search construction it would return names like Advantage Construction, Blue Earth Construction, and Brightstone Construction (all results have available exact match .coms so you don’t have to pay thousands procuring it).

What other industries could use this? Design, engineering, photography, medical, security, venture capital, gardening, cycling, and virtually any other business that primarily operates in the real world instead of online (for a long list of example industries, check out the Business Name Generator homepage and scroll three quarters of the way down the page).

If you’re building a web-based product (like most tech startups), a blog, a forum, or an informational site, this new business name generator is probably not for you. However if you’re looking for a name for the software company behind your web-based products, you could definitely use this tool to find available software company names (that’s how I came up with Lightstone Software for my company).

If you’d like to read more, please check out the Business Name Generator homepage — we offer a 100% money back guarantee so there’s no risk to you. If you find a business name you like, great, and if not, we’ll send you your money back.

If you don’t need to name an offline business, you might like our website name generator which is great for naming startups.

As always if you have any questions please don’t hesitate to email me: matt@leandomainsearch.com.

Release: User Accounts, Design Updates

Hey everybody!

One major update today and a few minor ones:

You can now sign up for a free Lean Domain Search account so that your favorites and search history will be stored online instead of in your browser’s cookies. This is important for two reasons:

  1. If your cookies are deleted you won’t lose all of your favorites or search history because they’ll be stored online
  2. You can switch computers and still have access to your favorites and search history

When you sign up your existing favorites and search history will be automatically imported into your new account. If you do happen to add a few favorites while you’re not signed in, have no fear, they will be saved to your account next time you sign in.

To sign up for a free account, head over to our account creation page page.

Today also brings quite a few design tweaks which I think you’ll agree improve the aesthetics a lot:

Here’s the before and after of the search results and the blog (click to view full image):

The changes include:

  • The header is now slightly thicker and slightly lighter. It also includes sign up and sign in links so you can quickly access your account from any page on the site.
  • The blue background no longer has a gradient; it’s now solid and a few shades darker.
  • You can’t see it on the screenshot, but the Ajax “searching…” spinner is back when you make a search.
  • The left sidebar no longer includes a user survey and no longer includes a newsletter sign up. You can optionally sign up for our newsletter when you create an account.
  • The blog and all of the supporting pages (About, FAQs, Testimonials, Terms and Privacy, Contact) now have a solid white background with dark text which makes them a hell of a lot easier to read.

There a lot of updates planned over the next few weeks. As always, if you run into any snags or have any recommendations please don’t hesitate to drop me a note.

Thanks!
Matt

Lean Domain Search Now Lets You Check the Availability of Any Domain Name Without Leaving the Search Results

I love using Lean Domain Search to brainstorm domain name ideas, but I’ve found that I often come up with other ideas while browsing the search results that I want to check. There are a lot of ways to do that but none through Lean Domain Search itself. That changes today.

There’s now a section on the right side of the search results that lets you check the availability of any domain name without leaving the page:

For those of you whose minds run wild while you’re browsing the results page this should be a welcome addition.

Enjoy!

Matt

Lean Domain Search Six Months After Its HackerNews Launch

On Monday, January 16, 2012 I launched Lean Domain Search by posting a link to it on HackerNews, making today it’s six month anniversary (woohoo!). In this post I’ll go over the major changes to the site since its launch, review some of its key metrics, and end with a glimpse of where its headed next.

Interface and features: then and now

To get an appreciation for just how far the site has come in six months, you need look no further than the search results page.

Here’s what it looked like at launch vs. what it looks like today.

Click to view full size:

A few things to look out for:

  • The color scheme will be the first major change that jumps out at you. When it launched, I was trying to differentiate the site from all the light-themed domain search tools out there. Quite a few people complained that the shade of green it used was too intense, so I went through several iterations trying to adjust the shade of green before deciding to move away from the dark design and over to the blue design that’s in use today.
  • The original version lacked sorting and filtering options. I didn’t think these weren’t necessary to launch the initial version (which is probably true), but they were so highly requested right after the launch that I wound up adding rudimentary versions within hours of launching.
  • There weren’t any share buttons on the original version. While not critical in terms of whether or not the product works, I probably missed out on a lot of traffic because the site was not easier for the initial wave of visitors to share. For future apps, having social sharing buttons built in on launch day is a must.
  • The original version lacked a search history. This was also not necessary to launch, so I held off on implementing until a few weeks later.
  • On the right in the new version you can see the “Popularity Index”, which is a measure of how popular your keyword is based on how often it appears in other registered domain names. I ran an A/B test which showed that users tended to search for more results when they were presented with this tidbit.
  • Also, in the original launch the results were not sorted at all — not just that you didn’t have the ability to sort them, but the order of the search results was more or less random. I eventually figured out a way to gauge the popularity of the prefixes and suffixes used in the results and order them that way by default, which is why the search results these days seem much better.
  • One other obvious change is that the search results are broken up into sections to make them easier to browse without losing your place.
  • Also not shown here, but when you double-check that a domain name is still available by clicking on it in the search results, Lean Domain Search will automatically check whether the Twitter name is still available and whether or not there is an existing trademark that it might infringe upon.

I’m happy with these changes because almost every one contributed to making the core product better. It’s so easy to start tacking on features when you should be improving the features that are the essence of your product.

Metrics, metrics, and more metrics

Number of searches per week

The spike on April 2 was a blog post where I announced that I had been adding 100 search results to Lean Domain Search per week since the launch and it now checked 2,000 domain names per search.

The spike in mid-April was due to another blog post, this one about calculating the top domain name prefixes and suffixes, and a mention several days later on a The Next Web article about naming your startup.

Finally, the spike in mid-June was due to a mention by Cyrus Shepard in an article he wrote for SEOmoz about picking a domain name, where he listed Lean Domain Search as the top domain name search tool.

For the curious, if you add up each month the total number of searches is just over 200,000.

Average number of searches per user

When I launched Lean Domain Search, I thought the average number of searches per user would be an important metric for gauging how useful the tool was. After all, if the average number of searches per user is low then it likely means that folks perform their first search don’t like what they see and leave never to come back. The problem with this line of thinking is that if the search results are really, really good then you’d also expect to see this number low because it means folks are finding domain names that they like and are done searching. I include this only to emphasize that you can’t just blindly look at metrics; you have to understand how to correctly interpret them.

Average number of available domain names returned per search

When Lean Domain Search launched, it checked the availability of 1,000 domain names per search. Almost immediately, I set out to add 100 new search results per week — that’s why the average number of search results rises so steadily in the beginning. As you’ll see in the next graph, these extra search results came at a cost in terms of time though, so after hitting 2,000 I put the breaks on and started looking for ways to improve the search speed.

As I made improvements in the speed, I continued adding search results; it now checks 2,500 domain names per search.

The funny thing is that even though it can now handle more in terms of capacity, I can’t do it without degrading the quality of the results because I’ve already added almost all of the “good” prefixes and suffixes.

Average amount of time it takes to return the search results

As noted above, as I added more search results, the amount of time it took to perform those searches steadily rose as well. I eventually started caching the search results so that if someone searches for a term that has already been searched for, it looks those results up in a database instead of performing the search the time-consuming algorithmic way.

I also spent a lot of time trying to devise new ways of performing the searches that would decrease the amount of time it took for non-cached queries, the results of which you can see starting at the end of June.

Percentage of searches that resulted in an error

For a variety of reasons, sometimes a user’s search would not finish successfully. For example, some queries took upwards of 30 seconds to complete, causing the search to fail. At it’s peak, about 4% of searches or 1 in 25 resulted in an error. Over time, I developed ways of reducing the number of errors — today, the error rate stands at about 0.3%, or 1 in 333.

Percentage of double-checked domain names that are actually registered

As noted above, the search results are calculated using Verisign’s .com zone file, which supposedly contains all of the registered .com domain names but doesn’t. As a result, some of the domain names that show up as available actually aren’t. Also, as people register domain names (both Lean Domain Search users and everyone else), some of the domain names that were available no longer will be.

When you click on an available search result in Lean Domain Search, it automatically double-checks that that domain name is still available. The graph above shows what percentage of domain names that are double-checked are actually registered. As you can see, early on almost 3 in 10 “available” domains were no longer available. These days, it’s less than 1 in 10 (and since Lean Domain Search tracks which are registered and no longer shows them as available, this number will keep on improving over time).

By the way, all of these metrics are courtesy of MixPanel, which makes it really easy to keep manage analytics in JavaScript-heavy apps.

Onward!

Lean Domain Search is my fourth and most successful web app in terms of both revenue and popularity.

(One good decision I made early on was to immediately add nice things folks were saying about Lean Domain Search to its testimonials page. Since launching, I’ve collected 160 unique testimonials, a testament to how much people seem to enjoy it.)

Looking forward, I see three primary areas to focus on:

  1. Continue improving the search interface to make it easier to find keyword-based domain names
  2. Explore methods for generating quality generic domain names
  3. Invest more time marketing the site in order to educate people about its existence

The last point is especially important because no matter how good or how fast the site is, it doesn’t matter one bit to the 99.9% of folks struggling to find a domain name because they don’t know Lean Domain Search exists.

Here’s to the next six months and beyond —

Matt

Lean Domain Search Now Automatically Checks Whether Domain Names Are Trademarked

A few days ago I added a feature to Lean Domain Search that automatically checks a domain name’s Twitter availability when you confirm that the domain name is still available. Today, I’m launching a similar update: Lean Domain Search now automatically checks whether there’s a registered trademark for domain name’s that you’re interested in.

For example, if you search for “app” and like and see that “appcyber.com” is still available, you can click it and Lean Domain Search will automatically check that the domain name is still available, that the Twitter name @appcyber is still available, and that there aren’t any trademarks containing “app cyber”:

There’s also a link included to the source on Trademarkia so you can double-check the results on your own. If there are similar trademarked names, Lean Domain Search will tell you there “may be” a registered trademark, which you can review and determine whether it conflicts with your name and business. When in doubt, check with a lawyer. 🙂

Hope it helps —

Lean Domain Search Now Automatically Checks a Domain Name’s Twitter Availability

Hey everybody!

Quick update that I think you’ll like: when you click on an available search result, Lean Domain Search now automatically checks whether that Twitter name is also available:

If you’re hunting for a domain name where it’s important to have a matching Twitter username, this new feature should save you a bit of work.

Enjoy —

Lean Domain Search Adds 450 New Search Results; Now Checks 2,500 Domain Names Per Search

I’m happy to announce that we’ve added 450 new search results to Lean Domain Search, putting the total domain names checked per search at 2,500. This should give you an even wider range of domain names to choose from for all but the most popular subjects areas.

Getting from 2,050 to 2,500 was not as trivial as it might sound. The 25% increase required a significant amount of architectural changes to the way Lean Domain Search performs the searches. Not only did we upgrade our search server, but we reworked our entire search algorithm so that these new results did not cause the average search time to jump as well. It’s still too early to say for sure, but it looks like even with the 450 new results the average search time for non-cached results is about 30% less than it used to be.

Here’s the full list of prefixes and suffixes added to the results:

+affiliate, +age, +air, +alert, +all, +angel, +anywhere, +architects, +architecture, +ask, +awards, +benefits, +bits, +black, +bliss, +boy, +brand, +broker, +bucks, +buy, +buyer, +buys, +byte, +cave, +centre, +chart, +charts, +com, +community, +consultancy, +credit, +culture, +db, +deals, +discount, +distribution, +doctors, +dollars, +download, +downloads, +dr, +earth, +education, +entertainment, +equipment, +ex, +family, +farm, +forest, +forlife, +fx, +gateway, +geeks, +generation, +gifts, +girls, +guy, +hd, +heart, +history, +ify, +images, +industries, +ink, +internet, +investments, +ist, +jam, +kids, +kingdom, +learn, +lens, +lighting, +limited, +listing, +llc, +mail, +maps, +marketing, +markets, +match, +men, +merchant, +meter, +method, +money, +monster, +networking, +new, +nexus, +ondemand, +pit, +planning, +plans, +player, +price, +program, +programs, +properties, +publishing, +quality, +ranch, +rates, +reach, +rewards, +rules, +runner, +sales, +savvy, +scan, +screen, +seller, +service, +shine, +sight, +silver, +squared, +ster, +stores, +stories, +summit, +surfer, +table, +teacher, +techs, +themes, +tours, +tower, +tracks, +tube, +tv, +university, +update, +usa, +video, +village, +visions, +wealth, +wholesale, +wiz, +woman, advance+, adventure+, after+, air+, america+, american+, angel+, anti+, article+, ask+, atlantic+, atlas+, bar+, bargain+, base+, basic+, beautiful+, bio+, block+, body+, bonus+, boom+, box+, brain+, bright+, brilliant+, broker+, buddy+, budget+, byte+, cafe+, camp+, campus+, capital+, cash+, celebrity+, center+, central+, certified+, change+, charity+, cheap+, choose+, circle+, clever+, client+, coach+, coastal+, commerce+, community+, company+, consumer+, contact+, content+, corporate+, country+, crown+, crystal+, data+, day+, db+, design+, dev+, diamond+, directory+, discount+, doc+, doctor+, dollar+, dr+, dragon+, eagle+, earth+, east+, edge+, edu+, education+, empire+, enjoy+, enter+, enterprise+, equi+, ex+, exchange+, fan+, final+, finance+, financial+, flip+, fly+, for+, forum+, fox+, freedom+, friendly+, front+, galaxy+, garden+, geek+, gem+, giant+, gift+, globe+, golden+, goto+, grand+, grey+, guide+, gulf+, hard+, hd+, healthy+, hello+, hollywood+, howto+, hub+, ideal+, impact+, industrial+, industry+, information+, ink+, inner+, innovation+, insight+, intelligent+, international+, internet+, intra+, invest+, iron+, island+, jet+, joy+, jump+, kid+, king+, lady+, land+, launch+, leader+, lean+, liberty+, lifestyle+, lime+, lion+, list+, little+, logic+, lucky+, mad+, major+, marketing+, maximum+, medi+, media+, mighty+, mind+, mission+, mob+, mobil+, monkey+, motion+, mountain+, mr+, multi+, national+, news+, nice+, noble+, nordic+, north+, northern+, ocean+, on+, organic+, our+, outdoor+, over+, pacific+, pack+, page+, panda+, para+, paradise+, partner+, patent+, peak+, peer+, people+, performance+, phoenix+, pin+, pink+, plan+, point+, pop+, portal+, post+, pre+, premier+, private+, profit+, promo+, push+, rank+, re+, realtime+, research+, retail+, review+, rich+, robo+, robot+, rock+, royal+, ruby+, sale+, savvy+, send+, senior+, service+, shadow+, shared+, shop+, short+, side+, skill+, slick+, slim+, smooth+, snow+, solo+, solution+, south+, southern+, speed+, spin+, spirit+, splash+, spring+, stage+, standard+, start+, stock+, stone+, store+, storm+, story+, street+, strong+, study+, sub+, success+, summit+, sunshine+, superior+, support+, supreme+, surf+, swap+, swift+, swiss+, system+, talent+, tap+, target+, thunder+, tiger+, time+, tiny+, titan+, tour+, trader+, traffic+, travel+, triple+, turbo+, tutor+, two+, uber+, union+, universal+, university+, up+, usa+, valley+, vertical+, victory+, video+, vintage+, vista+, vita+, wave+, we+, website+, west+, western+, wholesale+, wise+, wish+, wolf+, wow+, write+, young+, youth+, zap+, zone+, zoom+

Enjoy!