Build your own Geo-Locating solution | 16bytes

Apr

2

Build your own Geo-Locating solution

Thu, 04/02/2009 - 09:29

Many websites use Geo-Locating services to provide location aware content, so users will get the content in their language and currency, and will "enjoy" from some targeted adverts.

Geo-location is also required when providing content that is protected by DRM (Digital Rights Management) and has some geographical constraints.
On top of that, webmasters can benefit from Geo-Locating logs to analysis their web traffic.

This article explains how to build your own GeoLocating solution by translating IP address to Country.
At the end of this article, I'll show an example of simple Java implementation of IP-to-Country Geo-Locating tool.

Who manage the IP address allocation space?

There are five Regional Internet Registries (RIR) operating worldwide. Each one of these not-for-profit organisations is responsible to manage all internet number resources in its region, as specified at the following table:

Registry IPv4 Allocations* Region
AfriNIC 1% Africa
APNIC 32% Asia Pacific
ARIN 30% Canada, United States and several islands in the Caribbean Sea and North Atlantic Ocean
LACNIC 4% Latin America, Caribbean
RIPE NCC 33% Europe, the Middle East and parts of Central Asia

* As stated on NRO (Number Resource Organisation) report from September 2006. NRO is the organisation which is responsible for the joint activities of all Regional Internet Registries.

Historical assignments which are not under Regional Internet Registry management are still maintained by IANA

How do I get RIR database?
Each one of the Regional Internet Registries is managing its own database, and providing access to the database through a dedicated whois server.
Since hitting the whois servers every time our application is willing to get some IP information is not a good idea, it makes more sense to download their entire database.
Each RIR has a different format and a different way to download their entire database which contains many details as company's name. In addition, that detailed data is protected by some restricted terms and conditions.
However, since all we want is to get the country for a given IP addresses, we can just download files in a simplified format called "Standard Statistics Exchange Format". These files are used by all regional internet registries, produced daily, and can be downloaded by saving the following links:

ftp://ftp.afrinic.net/pub/stats/afrinic/delegated-afrinic-latest
ftp://ftp.apnic.net/pub/stats/apnic/delegated-apnic-latest
ftp://ftp.arin.net/pub/stats/arin/delegated-arin-latest
ftp://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-latest

ftp://ftp.ripe.net/ripe/stats/delegated-ripencc-latest
ftp://ftp.apnic.net/pub/stats/iana/delegated-iana-latest

How do I use RIR Statistics Exchange files?
The full specification of the RIR statistics exchange file format can be found here.

The following Java class provides geo-locating solution by using RIR statistics exchange files.
Take a look at the main() method to see an example of how to use this class.


/*
 * @(#)GeoLocator.java    1.00 15/10/06
 *
 * Copyright 2006 16Bytes.
 *  
 */
package com.sixteenbytes.geolocator;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.net.InetAddress;
import java.util.List;
import java.util.NoSuchElementException;
import java.util.StringTokenizer;
import java.util.TreeMap;
import java.util.Vector;

/**
 * The GeoLocator class allows applications to support
 * Real Time Geo-Locating solution by using IP allocation database.
 * Before calling the getCountry() method, be sure you loaded 
 * all IP ranges by calling addIPRanges().
 * 
 * @author  Alon Schreibman
 * @version 1.00, 15/10/06
 */

public class GeoLocator {

    private TreeMap<Long, List<Object>> map = new TreeMap<Long, List<Object>>();

    /**
     * Parse the ipAddress string, and transform it into a single long value
     *
     * @param    ipAddress     a string contains the ip address.
     * @return    the ipAddress in a long format
     */
    protected long ipToLong(String ipAddress)
    {
        StringTokenizer st1 = new StringTokenizer(ipAddress,".");
        long byte1 = Short.parseShort(st1.nextToken());
        long byte2 = Short.parseShort(st1.nextToken());
        long byte3 = Short.parseShort(st1.nextToken());
        long byte4 = Short.parseShort(st1.nextToken());

        return byte1 * 256 * 256 * 256 
                + byte2 * 256 * 256 
                + byte3 * 256 
                + byte4;
    }

    /**
     * Convert IPv4 InetAddress into a single long value
     *
     * @param    ipAddress     a string contains the ip address.
     * @return    the ipAddress in a long format
     */
    protected long ipToLong(InetAddress addr)
    {
        
        byte[] addrBytes = addr.getAddress();
        long byte1 = (addrBytes[0] < 0 ? addrBytes[0] + 256 : addrBytes[0]);
        long byte2 = (addrBytes[1] < 0 ? addrBytes[1] + 256 : addrBytes[1]);
        long byte3 = (addrBytes[2] < 0 ? addrBytes[2] + 256 : addrBytes[2]);
        long byte4 = (addrBytes[3] < 0 ? addrBytes[2] + 256 : addrBytes[2]);
        
        
        return byte1 * 256 * 256 * 256 
                + byte2 * 256 * 256 
                + byte3 * 256 
                + byte4;
    }
    
    /**
     * Load RIR Statistics Exchange Format file into memory.
     * It is possible to load multiple RIR Statistics Exchange
     * from different registries.
     *
     * @param   fileName     a string to be parsed.
     * @exception FileNotFoundException
     * @exception IOException
     */
    public void addIPRanges(String fileName) throws FileNotFoundException, IOException
    {
        File f = new File(fileName);
        BufferedReader r = new BufferedReader(new FileReader(f));

        r.readLine();
        List<Object> list;
        // for each line in the file

        for (String str = r.readLine(); str != null; str = r.readLine()) {

            // get all tokens

            StringTokenizer st = new StringTokenizer(str, "|");

            if (st.countTokens() == 7) {

                list = new Vector<Object>(7);
                list.add(st.nextToken());
                list.add(st.nextToken());
                list.add(st.nextToken());
                // if it's IPv4 record
                if (list.get(2).equals("ipv4")) {
                    // Create a long which represents the beginning of the IP range
                    Long lIp = new Long(ipToLong(st.nextToken()));
                    list.add(lIp);
                    list.add(new Long(Long.parseLong(st.nextToken())
                            + ((Long) list.get(3)).longValue()));
                    list.add(st.nextToken());
                    list.add(st.nextToken());
                    // use the IP as the index
                    map.put(lIp, list);
                }
            }
        }
        r.close();
    }
    
    /**
     * Returns a two-letter country code for a givven IPv4 InetAddress
     *
     * @param   addr     string to be parsed.
     * @return the two-letter country code.
     */
    public String getCountry(InetAddress addr) {
        long ipAddress = ipToLong(addr);

        try {
            Object key = map.headMap(new Long(ipAddress + 1)).lastKey();
            List resultRecord = (List) map.get(key);
            if (((Long) resultRecord.get(4)).longValue() >= ipAddress)
                return (String) resultRecord.get(1);
        } catch (NoSuchElementException e) {
//            e.printStackTrace();
        }
        return null;
    }

    public static void main(String[] args) {
        try {

            GeoLocator geoLocator = new GeoLocator();
            geoLocator.addIPRanges("/RIR/delegated-arin-latest.txt");
            geoLocator.addIPRanges("/RIR/delegated-apnic-latest.txt");
            geoLocator.addIPRanges("/RIR/delegated-lacnic-latest.txt");
            geoLocator.addIPRanges("/RIR/delegated-ripencc-latest.txt");
            geoLocator.addIPRanges("/RIR/delegated-afrinic-latest.txt");
            geoLocator.addIPRanges("/RIR/delegated-iana-latest.txt");
            
            System.out.println(geoLocator.getCountry
               (InetAddress.getByName("80.178.75.155")));
            System.out.println(geoLocator.getCountry
               (InetAddress.getByName("16bytes.com")));

        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

»