Friday, 17 February 2012

Diving in to Perl with GeoTags and GoogleMaps


Girl, Unallocated recently posted a guide to plotting geotag data using exiftool and Google Earth here.

GoogleMaps also has some info about how to plot lat / long coordinates along with an info box on a map here.
The example URL given looks like:
http://maps.google.com/maps?q=37.771008,+-122.41175+%28You+can+insert+your+text+here%29&iwloc=A&hl=en
You can see that the latitude parameter comes first in floating point (eg 37.771008) followed by the longitude and then a text field ("You can insert your text here") to be printed in the information popup. The "iwloc" parameter shows the GoogleMaps point label ("A"). The "hl" parameter is the language to use (en = English).

So ... I'd thought I'd build on (*cough RIPOFF *cough) those posts  and write a basic Perl script to automatically extract the EXIF location data from a photo and then formulate an URL the user can then paste into their Internet Browser. The SIFT VM already has the exiftool Perl module and Perl 5 installed so it should be simple eh?

Method

At first, I thought I would filter/convert the output text of "exiftool" but after hunting around at www.cpan.org (the Perl package repository), I discovered the Image::ExifTool::Location module which provides some easy to use functions to retrieve lat / long coordinates from EXIF data.

In order to use this though, we first have to install it on SIFT. Here's the guide.
Its pretty simple though, you just type:
"sudo cpan Module::Name"
to get the cpan installer to automagically retrieve and install the module you're after. I added in the "sudo" so the installations aren't hindered by file permission issues.

Looking at the documentation for the Image::ExifTool::Location module shows that it requires/depends on the Image::ExifTool and the Geo::Coordinates::DecimalDegrees modules. The Image::ExifTool module is already installed on SIFT - so we will install the Geo::Coordinates::DecimalDegrees module before installing the Image::ExifTool::Location module just in case.

At a terminal window type:
"sudo cpan Geo::Coordinates::DecimalDegrees"
followed by
"sudo cpan Image::ExifTool::Location"

At this point, I had a bit of of an epiphany / scope creep when I thought "Wouldn't it be cool if you could also output the URLs for multiple files as links in a single HTML file?". The user could then open the HTML file and click on the various links to see the corresponding Google Map. No cutting and pasting of URLs and if there's a lot of pics with geotag information, it could make things a little easier for our fearless forensicators.

Fortunately, there's already a Perl module to create an HTML table - its called HTML::QuickTable. We can install it by typing "sudo cpan HTML::QuickTable" at the command line.

After we've completed the initial setup, we can now start writing our script (finally!). I called it "exif2map.pl" and put it in "/usr/local/bin/". Note: Don't forget to to make the file executable by typing "chmod a+x /usr/local/bin/exif2map.pl".


The Code

Here's the code I mangled came up with:

#START CODE
#!/usr/bin/perl -w

# Perl script to take the output of exiftool and conjure up a web link
# to google maps if the image has stored GPS lat/long info.

use strict;

use Image::ExifTool;
use Image::ExifTool::Location;
use Getopt::Long;
use HTML::QuickTable;

my $help = '';
my $htmloutput = '';
my @filenames;
my %file_listing;

GetOptions('help|h' => \$help,
    'html' => \$htmloutput,
    'f=s@' => \@filenames);

if ($help||@filenames == 0)
{
    print("\nexif2map.pl v2012.02.16\n");
    print("Perl script to take the output of exiftool and conjure up a web link\n");
    print("to google maps if the image has stored GPS lat/long info.\n");

    print("\nUsage: exif2map.pl [-h|help] [-f filename] [-html]\n");
    print("-h|help ....... Help (print this information). Does not run anything else.\n");
    print("-f filename ... File(s) to extract lat/long from\n");
    print("-html ......... Also output results as a timestamped html file in current directory\n");

    print("\nExample: exif2map.pl -f /cases/galloping-gonzo.jpg");
    print("\nExample: exif2map.pl -f /cases/krazy-kermit.jpg -f/cases/flying-piggy.jpg -html\n\n");
    print("Note: Outputs results to command line and (if specified) to a timestamped html file\n");
    print("in the current directory (e.g. exif2map-output-TIMESTAMP.html)\n\n");
   
    exit;
}


# Main processing loop
print("\nexif2map.pl v2012.02.16\n");
foreach my $name (@filenames)
{
    ProcessFilename($name);
}

# If html output required AND we have actually retrieved some data ...
if ( ($htmloutput) && (keys(%file_listing) > 0) )
{   
    #timestamped output filename
    my $htmloutputfile = "exif2map-output-".time.".html";

    open(my $html_output_file, ">".$htmloutputfile) || die("Unable to open $htmloutputfile for writing\n");

    my $htmltable = HTML::QuickTable->new(border => 1, labels => 1);

    # Added preceding "/" to "Filename" so that the HTML::QuickTable sorting doesn't result in
    # the column headings being re-ordered after / below a filename beginning with a "\".
    $file_listing{"/Filename"} = "GoogleMaps Link";

    print $html_output_file "<HTML>";
    print $html_output_file $htmltable->render(\%file_listing);
    print $html_output_file "<\/HTML>";

    close($htmloutputfile);
}

sub ProcessFilename
{
    my $filename = shift;

    if (-e $filename) #file must exist
    {
        my $exif = Image::ExifTool->new();
        # Extract all info from existing image
        if ($exif->ExtractInfo($filename))
        {
            # Ensure all 4 GPS params are present
            # ie GPSLatitude, GPSLatitudeRef, GPSLongitude, GPSLongitudeRef
            # The Ref values indicate North/South and East/West
            if ($exif->HasLocation())
            {
                my ($lat, $lon) = $exif->GetLocation();
                print("\n$filename contains Lat: $lat, Long: $lon\n");
                print("URL: http://maps.google.com/maps?q=$lat,+$lon($filename)&iwloc=A&hl=en\n");
                if ($htmloutput) # save GoogleMaps URL to global hashmap indexed by filename
                {
                    $file_listing{$filename} = "<A HREF = \"http://maps.google.com/maps?q=$lat,+$lon($filename)&iwloc=A&hl=en\"> http://maps.google.com/maps?q=$lat,+$lon($filename)&iwloc=A&hl=en</A>";
                }
                return 1;
            }
            else
            {
                print("\n$filename : No Location Info available!\n");
                return 0;
            }
        }
        else
        {
            print("\n$filename : Cannot Extract Info!\n");
            return 0;
        }
    }
    else
    {
        print("\n$filename does not exist!\n");
        return 0;
    }
}

#END CODE

A brief explanation? The first section is getting the command line args (GetOptions) and printing help if the user stuffs up. The next section is the main processing loop which takes the list of filenames and calls our ProcessFilename subroutine for each. The results will be printed AND stored in the %file_listing hash (contains the filename and the URL).
The next section handles writing the HTML file (if required) and uses the HTML::QuickTable module to come up with the HTML for the table. We then print a preceding <HTML> tag,  the table HTML and the trailing </HTML> tag all to a timestamped file in the users current directory called "exif2map-output-TIME.html". Where TIME is the number of seconds since 1970.


Testing

Here's a sample geotagged image from wikipedia:
I've saved it into "/cases/" along with a .jpg file with no EXIF geotag information (Cheeky4n6Monkey.jpg) and a mystery geotagged file (wheres-Cheeky4n6Monkey.jpg). I prepared this mystery file on the sly earlier (using a separate but similar Perl script). Note: Blogger seems to be stripping out the GPS data (maybe I missed setting some other parameters?) of the pic below but I did actually manage to set the GPS info on a local copy. Honest!


Mystery file - "wheres-Cheeky4n6Monkey.jpg"
 
Lets see what happens when we run our script ...

sansforensics@SIFT-Workstation:~$ exif2map.pl -f /cases/wheres-Cheeky4n6Monkey.jpg -f /cases/GPS_location_stamped_with_GPStamper.jpg  -f /cases/Cheeky4n6Monkey.jpg -html

exif2map.pl v2012.02.16

/cases/wheres-Cheeky4n6Monkey.jpg contains Lat: 36.1147630001389, Long: -115.172811
URL: http://maps.google.com/maps?q=36.1147630001389,+-115.172811(/cases/wheres-Cheeky4n6Monkey.jpg)&iwloc=A&hl=en

/cases/GPS_location_stamped_with_GPStamper.jpg contains Lat: 41.888948, Long: -87.624494
URL: http://maps.google.com/maps?q=41.888948,+-87.624494(/cases/GPS_location_stamped_with_GPStamper.jpg)&iwloc=A&hl=en

/cases/Cheeky4n6Monkey.jpg : No Location Info available!
sansforensics@SIFT-Workstation:~$


You can see that the script found/printed GPS positions and URLs for our mystery file wheres-Cheeky4n6Monkey.jpg and GPS_location_stamped_with_GPStamper.jpg but nothing for Cheeky4n6Monkey.jpg.

In Firefox, The outputted HTML file looks like:

Output HTML Table

Notice how we don't have a table entry for the file with no GPS info (Cheeky4n6Monkey.jpg)? I thought it wouldn't make sense as the whole purpose was to plot EXIF GPS data.
Anyhow, when we follow the table link for "GPS_location_stamped_with_GPStamper.jpg" we get:

"GPS_Location_stamped_withGPStamper.jpg" plotted on GoogleMaps

Aha! Chicago eh? No surprise when you look at the picture. Go Cubbies !


 "wheres-Cheeky4n6Monkey.jpg" gives:

 "wheres-Cheeky4n6Monkey.jpg" plotted on GoogleMaps


 36.114763,-115.172811 = Vegas Baby!  Uh-oh, what have you been up to Cheek4n6Monkey?!

Like any good forensicator we should validate our results. We can check our script's output against the "Exif Viewer" Firefox plugin (already installed on SIFT):

"Exif Viewer" of "GPS_Location_stamped_withGPStamper.jpg"


"Exif Viewer" of "wheres-Cheeky4n6Monkey.jpg"

Hooray! Our script output matches Exif Viewer's results for Latitude and Longitude. In fact, I think "Exif Viewer" has rounded off a few digits of precision.

So that's about it folks - all of this was not in vain. There might be some further formatting to be done to make it look prettier but the basic functionality has been achieved. Cool bananas eh? Let me know what your think - comments/suggestions/curses?


8 comments:

  1. Nice. GirlUnallocated's post got me thinking of doing the same. I put together a bash one liner that would take the GPS Lat/Lon coordinates from all of the images in a directory and pull down the corresponding Google map with the coordinates marked. I got it working and was going to blog it (may still), but you've actually engineered something good here and beat me to it. Very nicely done.

    ReplyDelete
  2. Thanks Dave!

    I don't know if "engineered" would be the most apt description for my script lol.

    And I'd definitely be interested in reading more about your solution.
    Having more resources on Bash scripting can only be a good thing eh?

    Thanks for taking the time to read/comment.

    ReplyDelete
  3. Really good stuff. I think a lot of people can
    find this script really useful.
    IMHO a good improvement could be introduce the possibility
    to execute the script agains a folder tree and have it to
    travel around extracting geo infos from pictures to build
    the html file with the google maps links.
    Thanks for sharing.

    ReplyDelete
    Replies
    1. Challenge accepted! It might take me some time but I shall see what my novice Perl skills can come up with.

      Thanks for the suggestion!

      Delete
  4. Hi, excellent post.

    This script could be useful one day or another. Put it in my toolkit.


    Have you see that ?
    http://windowsir.blogspot.com/2012/02/sharing-and-case-studies.html

    Work always pay ;)

    Thierry

    ReplyDelete
    Replies
    1. Thanks - nice to know someone thinks it might be useful (and I haven't wasted my time lol).
      I read Harlan's blog regularly and was pleasantly surprised when he mentioned this blog. Cool bananas eh?
      I wish this blog did pay ;) Maybe someone out there needs a new DFIR analyst?

      Delete
  5. Inspired by you and "Girl, Unallocated," my contribution, such as it is:

    http://trustedsignal.blogspot.com/2012/02/plotting-photo-location-data-with-bing.html

    ReplyDelete
    Replies
    1. Ahhh the plot(s) thicken(s)!
      I just left a comment on your blog.
      Nice work on your blog BTW - I found it highly informative yet still entertaining.

      Delete