Linux Sleuthing

2012-02-14T13:38:00.000-08:00

iOS .sinf Name Calling

In my ever present quest to identify the true owners of stolen iPods, I made discovery in iOS while examining a Touch that may be probative: the app .sinf files found in the /private/var2/Applications sub folders. According to File-Extensions.org:

The SINF file extension is associated with applications for Apple iOS operating system that is used in Apple iPhone, iPad and iPod Touch. File contains information about digital rights that are applied in application. The SINF file is stored in an IPA iOS application archive.

I found that by searching the ../Applications directory for .sinf files, and then grepping for the term "name", the Apple Store real name associated with the app can be discovered. On the Linux command line, this can be accomplished very quickly with:

$ find private/var2/Applications -name "*.sinf" -exec strings -f {} \; | grep name

Modification dates for the files can be used to create a timeline of activity for the device and perhaps demonstrate when new residents moved in, so to speak. The find command can by used with stat to quickly provide a list of date stamps:

$ find private/var2/Applications -name "*.sinf" -exec stat {} \;

But, even better, you can put it all together in a fairly simple command and create csv output for examination and sorting:

$ find private/ -name "*.sinf" | while read i; do name=$(strings "$i" | grep name); date=$(stat -c %y "$i"); echo -e "$i,$name,$date"; done

It appears from content that I have uncovered in a suspected stolen device, that the real name of the Apple Store account used to install the app is embedded in the .sinf file at the time of installation. If this is the case, a stolen device, though it have the device name changed and the true owner's data deleted, may still have applications that were installed with the owners Apple Store account!

Testing still needs to be done for verification, and I don't currently have any test devices to properly test. If you are able to conduct any validation studies, please comment on this post with your findings. I'll amend this post once I'm able to conduct my own studies or receive reliable findings from others.

iPod's, what's in a name?

2012-01-27T13:43:00.000-08:00

iPod Device Names

iPod devices have a name. It's set by the user when they initialize the device through iTune's (there are alternate initialization methods, but that is not the focus of this post). When the focus of the investigation is determining the device owner, the device name is a good place to start. The device name, for example, could be "John Doe" and you happen to know who is John Doe, or how to find out.

Of course, the device name could be 'Pookie', which won't help you out too much. But, don't give up, I've already demonstrated another, even more useful, method for identifying iPod owners through iTune's purchased media. Take a look here if that interests you.

But, I got curious, where in the iPod can you find the device name? It's clearly stored on the device, because, as any iPod owner can tell you, if you navigate from the main menu to the 'About' screen in 'Settings', you'll see something akin to "John Doe's iPod."

Where to Look

The first place to look in a FAT formatted iPod is the volume label of the data volume (aka partition). The current device name is the volume name. You can view it with blkid, or for the forensically inclinded, with the sluethkit at the root level.

I'll use recent 5th gen Nano I recently examined as an example. I am operating as root because I am examining a device directly:

# blkid /dev/sdd1
/dev/sdd1: LABEL="PINK PANTHE" UUID="E0B8-3334" TYPE="vfat"

# fls /dev/sdd1
r/r 3:    PINK PANTHE (Volume Label Entry)
d/d 5:    iPod_Control
...

Now, I'm fairly worldly (all my friends are now rolling their eyes), but I suspected when I check Settings | About, the device name on this Nano, I'd find the device name was 'Pink Panther', not the truncated 'Pink Panthe' that was in the volume, which has a limit of 12 characters. And sure enough, that's what I found: 'pink panther.'

So, if the 'r' in pather isn't in the volume, then the volume is not the source of the data in the About screen. So, what is the source? Turns out, after mounting the device read-only and employing my favorite keyword search utility (more on that one later), the source turns out to be the 'Library.itdb' SQLite database in the 'iPod_Control/iTunes/iTunes Library.itlp/' directory.

I found the table in which the device name resides as follows:

# sqlite3 '/media/iPod/iPod_Control/iTunes/iTunes Library.itlp/Library.itdb' .dump | grep 'pink panthe'

INSERT INTO "container" VALUES(-3226555229562403833,0,333435002,347345556,'pink panther',100,0,1,0,1,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);

What I did there was dump the table contents, which shows the commands that were issued to create the database and populate it. The dump, when saved to a file, can be used to backup and restore a database. For my purpose, I see that a list of values, including 'pink panther' was inserted into the 'container' table.

Now, I can produce a nice query that can be used in future examinations to directly recover the device name from the Library.itdb database:

# sqlite3 -line '/media/iPod/iPod_Control/iTunes/iTunes Library.itlp/Library.itdb' 'select name from container'
 name = pink panther

Now I have two sources for the device name in a FAT formatted device. And, the database query can be used for HFS formatted iPod Classics, presumably. Combine that with the media search for Apple Store account and real name information, and even an unallocated search for MPEG-4 metadata (next post), and you have a robust, though not fool proof methodology for identifying iPod owners.

Whose iPod?

2012-01-19T16:31:00.000-08:00

iPods, iPods, everywhere...

...Which means they are frequently lost or stolen. ...Which means they end up in my office with a request attached stating, "Can you please try to figure out who owns this?"

Device Name

iTunes, the principal means for managing iPod content, allows users to name their device. Usually, owner's put their names, like 'John Doe,' so the the device is reported in iTunes as 'John Doe's iPod.' In a fat32 formatted device, like an iPod Nano, the device name is recorded as the volume label.

When it comes to linking the device to owners, though, the device name is seldom enough. 'John Doe' might be too common a name, not listed in the phone book or in your records management system. Worse, the device name might be 'Wookie'. What then?

What Apple Doesn't Tell You

The Apple Store, commonly known as the iTunes store, sells media for playback on the device. Music and Video are popular purchases, and are sold in the MPEG-4 format with file extensions of .m4p (music) and .m4v (video).

The purchaser of this content has to create and use an Apple Store account. For quite some time, the account name is the user's email address. Users provide there real names as part of the creation process, which is necessary for credit card transactions. Very standard business practices, nothing nefarious here.

The slight of hand comes on download (well, it would have to occur before download for you precise-types). The Apple Store account name and the purchaser's real name are embedded in the media file! MPEG-4 files contain metadata (data about data) such as the Artist, Title, Album, even album cover art. The metadata takes the form of key:value pairs, often referred to as 'atoms.'

Where to look

The atoms 'name' and 'apID' contain the purchasers real name and account name (email address) respectively. However, no tools I am aware of automatically display this content. The excellent exiftool by Phil Harvey will display the account name (apID atom, i.e., email address) but not the purchasers real name.

While Harvey's tool is excellent, it just doesn't do to run exiftool against every media file on an iPod. First, not every media file is an MPEG-4 with Apple Store metadata. iTunes allows users to convert their existing mp3 and CD collections to MPEG-4, for example. These media files take on the .m4a file extension and do not contain purchaser information. Nor do .mp3s, for that matter.

When I'm in a hurry, which is most of the time, I resort to a straight forward, compound command:

$ find /media/iPod/iPod_Control/Music -type f -name "*.m4[pv]" | while read i; do strings -f "$i"|grep -E -m1 'name.+'; strings -f "$i"| grep -E -m1 -A2 'apID'; done

Am I out of my flipping mind? No. That really works, works well, and works really, really fast. Should I explain it? No, not unless you really want me to. The output looks like this:

/media/iPod/iPod_Control/Music/F49/PFLT.m4v: nameJohn Doe
/media/iPod/iPod_Control/Music/F49/PFLT.m4v: 8apID
/media/iPod/iPod_Control/Music/F49/PFLT.m4v: 0data
/media/iPod/iPod_Control/Music/F49/PFLT.m4v: jdoe@email.com

/media/iPod/iPod_Control/Music/F49/QQDN.m4p: nameJohn Doe
/media/iPod/iPod_Control/Music/F49/QQDN.m4p: 8apID
/media/iPod/iPod_Control/Music/F49/QQDN.m4p: 0data
/media/iPod/iPod_Control/Music/F49/QQDN.m4p: jdoe@email.com

Pretty? Maybe not. Does it answer the question of what is the real name and email address of the media purchaser? Yes. And did I mention, really, really fast?

I'm aware that some people might like or need nicer output for a report of some kind. I wrote a bash script , called iphone_music that works with exiftool to produce nice output:

======== /media/iPod/iPod_Control/Music/F49/PFLT.m4v
File Type                       : M4V
Apple Store Account             : jdoe@email.com
Apple Store Account Type        : iTunes
Apple Store Real Name           : John Doe

======== /media/iPod/iPod_Control/Music/F49/QQDN.m4p
File Type                       : M4P
Apple Store Account             : jdoe@email.com
Apple Store Account Type        : iTunes
Apple Store Real Name           : John Doe

Iphone_music is also quite fast and uses the same basic methodology as the find command first demonstrated. Additionally, it can tell you the names, artists, albums, etc. of other media on the device for instances where the owner has no purchased media on the device but can describe the media on board (e.g., .mp3, .m4a). You may have noticed from the paths of the media files demonstrated, iTunes does not name the files after their content.

Iphone_music can be vastly improved, such as sorting by artist, email, owner name, etc, and I'll likely rewrite the tool in python to facilitate implementing these features.

Final Note (pun intended)

The methods detailed here work on mounted file systems and allocated files. It is possible to find the real name and email address on devices where the media files have been deleted by the iPod thief / finder. I won't detail the process here, but it involves using the Sleuthkit to pipe unallocated space to strings and grep for the name and apID atoms. Another method, though slower, would be to use photorec or another file carving tool to recover MPEG-4 files an then use the methods above to search the recovered files.

Unoconv is number one!

2012-01-19T14:04:00.000-08:00

I discussed a recent case where I was seeking Enhanced Metafiles and discovered that LibreOffice could be used to open and view them (see, LibreOffice: An Unlikely Image Viewer). I mentioned at the end of that post that unoconv could be used to automate the process of taking the difficult to view EMF files and convert them something very portable, like PDF.

Today, I actually undertook that task when carving form EMF files produced over 1000 files to be examined. Doing so exposed me to the full capabilities of unoconv, and I'm quite excited about the possibilities.

What, exactly is unoconv?

From the man page: "unoconv is a command line utility that can convert any file format that OpenOffice can import, to any file format that OpenOffice is capable of exporting." This begs the question: what can OpenOffice (or LibreOffice) import and export? Glad you asked:

$ unoconv --show

The following list of document formats are currently available:

bib      - BibTeX [.bib]
doc      - Microsoft Word 97/2000/XP [.doc]
doc6     - Microsoft Word 6.0 [.doc]
doc95    - Microsoft Word 95 [.doc]
docbook - DocBook [.xml]
html     - HTML Document (OpenOffice.org Writer) [.html]
odt      - Open Document Text [.odt]
ott      - Open Document Text [.ott]
ooxml    - Microsoft Office Open XML [.xml]
pdb      - AportisDoc (Palm) [.pdb]
pdf      - Portable Document Format [.pdf]
psw      - Pocket Word [.psw]
rtf      - Rich Text Format [.rtf]
latex    - LaTeX 2e [.ltx]
sdw      - StarWriter 5.0 [.sdw]
sdw4     - StarWriter 4.0 [.sdw]
sdw3     - StarWriter 3.0 [.sdw]
stw      - Open Office.org 1.0 Text Document Template [.stw]
sxw      - Open Office.org 1.0 Text Document [.sxw]
text     - Text Encoded [.txt]
txt      - Plain Text [.txt]
vor      - StarWriter 5.0 Template [.vor]
vor4     - StarWriter 4.0 Template [.vor]
vor3     - StarWriter 3.0 Template [.vor]
xhtml    - XHTML Document [.html]

The following list of graphics formats are currently available:

bmp      - Windows Bitmap [.bmp]
emf      - Enhanced Metafile [.emf]
eps      - Encapsulated PostScript [.eps]
gif      - Graphics Interchange Format [.gif]
html     - HTML Document (OpenOffice.org Draw) [.html]
jpg      - Joint Photographic Experts Group [.jpg]
met      - OS/2 Metafile [.met]
odd      - OpenDocument Drawing [.odd]
otg      - OpenDocument Drawing Template [.otg]
pbm      - Portable Bitmap [.pbm]
pct      - Mac Pict [.pct]
pdf      - Portable Document Format [.pdf]
pgm      - Portable Graymap [.pgm]
png      - Portable Network Graphic [.png]
ppm      - Portable Pixelmap [.ppm]
ras      - Sun Raster Image [.ras]
std      - OpenOffice.org 1.0 Drawing Template [.std]
svg      - Scalable Vector Graphics [.svg]
svm      - StarView Metafile [.svm]
swf      - Macromedia Flash (SWF) [.swf]
sxd      - OpenOffice.org 1.0 Drawing [.sxd]
sxd3     - StarDraw 3.0 [.sxd]
sxd5     - StarDraw 5.0 [.sxd]
tiff     - Tagged Image File Format [.tiff]
vor      - StarDraw 5.0 Template [.vor]
vor3     - StarDraw 3.0 Template [.vor]
wmf      - Windows Metafile [.wmf]
xhtml    - XHTML [.xhtml]
xpm      - X PixMap [.xpm]

The following list of presentation formats are currently available:

bmp      - Windows Bitmap [.bmp]
emf      - Enhanced Metafile [.emf]
eps      - Encapsulated PostScript [.eps]
gif      - Graphics Interchange Format [.gif]
html     - HTML Document (OpenOffice.org Impress) [.html]
jpg      - Joint Photographic Experts Group [.jpg]
met      - OS/2 Metafile [.met]
odd      - OpenDocument Drawing (Impress) [.odd]
odg      - OpenOffice.org 1.0 Drawing (OpenOffice.org Impress) [.odg]
odp      - OpenDocument Presentation [.odp]
otp      - OpenDocument Presentation Template [.otp]
pbm      - Portable Bitmap [.pbm]
pct      - Mac Pict [.pct]
pdf      - Portable Document Format [.pdf]
pgm      - Portable Graymap [.pgm]
png      - Portable Network Graphic [.png]
pot      - Microsoft PowerPoint 97/2000/XP Template [.pot]
ppm      - Portable Pixelmap [.ppm]
ppt      - Microsoft PowerPoint 97/2000/XP [.ppt]
pwp      - PlaceWare [.pwp]
ras      - Sun Raster Image [.ras]
sda      - StarDraw 5.0 (OpenOffice.org Impress) [.sda]
sdd      - StarImpress 5.0 [.sdd]
sdd3     - StarDraw 3.0 (OpenOffice.org Impress) [.sdd]
sdd4     - StarImpress 4.0 [.sdd]
sti      - OpenOffice.org 1.0 Presentation Template [.sti]
stp      - OpenDocument Presentation Template [.stp]
svg      - Scalable Vector Graphics [.svg]
svm      - StarView Metafile [.svm]
swf      - Macromedia Flash (SWF) [.swf]
sxi      - OpenOffice.org 1.0 Presentation [.sxi]
tiff     - Tagged Image File Format [.tiff]
vor      - StarImpress 5.0 Template [.vor]
vor3     - StarDraw 3.0 Template (OpenOffice.org Impress) [.vor]
vor4     - StarImpress 4.0 Template [.vor]
vor5     - StarDraw 5.0 Template (OpenOffice.org Impress) [.vor]
wmf      - Windows Metafile [.wmf]
xhtml    - XHTML [.xml]
xpm      - X PixMap [.xpm]

The following list of spreadsheet formats are currently available:

csv      - Text CSV [.csv]
dbf      - dBase [.dbf]
dif      - Data Interchange Format [.dif]
html     - HTML Document (OpenOffice.org Calc) [.html]
ods      - Open Document Spreadsheet [.ods]
ooxml    - Microsoft Excel 2003 XML [.xml]
pdf      - Portable Document Format [.pdf]
pts      - OpenDocument Spreadsheet Template [.pts]
pxl      - Pocket Excel [.pxl]
sdc      - StarCalc 5.0 [.sdc]
sdc4     - StarCalc 4.0 [.sdc]
sdc3     - StarCalc 3.0 [.sdc]
slk      - SYLK [.slk]
stc      - OpenOffice.org 1.0 Spreadsheet Template [.stc]
sxc      - OpenOffice.org 1.0 Spreadsheet [.sxc]
vor3     - StarCalc 3.0 Template [.vor]
vor4     - StarCalc 4.0 Template [.vor]
vor      - StarCalc 5.0 Template [.vor]
xhtml    - XHTML [.xhtml]
xls      - Microsoft Excel 97/2000/XP [.xls]
xls5     - Microsoft Excel 5.0 [.xls]
xls95    - Microsoft Excel 95 [.xls]
xlt      - Microsoft Excel 97/2000/XP Template [.xlt]
xlt5     - Microsoft Excel 5.0 Template [.xlt]
xlt95    - Microsoft Excel 95 Template [.xlt]

So, what's missing? The newer Microsoft 'x' formats: docx, xlsx, etc. (Microsoft Office XML) are not listed, but conversion is still possible! Let you mantra be "unoconv is a command line utility that can convert any file format that OpenOffice can import, to any file format that OpenOffice is capable of exporting."

Using unoconv

To use unoconv, you first have to start the listener:

$ unoconv -l &  #the '&' backgrounds the process and returns control of the 

                 terminal winodow to your

[1] 9998        #9998 is the process number of the listener.

We can see that the listener is a python program and the killall command to cancel the listener would have to be directed at python. To avoid killing other processes, 'kill 9998' should be used rather than 'killall python':

$ ps 9998
PID TTY      STAT   TIME COMMAND
9998 pts/0    Sl     0:00 /usr/bin/python /usr/bin/unoconv -l

With the listener running, conversion of documents is straight forward, as we can see from the help:

$ unoconv -h
usage: unoconv [options] file [file2 ..]

Convert from and to any format supported by OpenOffice

unoconv options:
-c, --connection=string use a custom connection string
-d, --doctype=type       specify document type
                             (document, graphics, presentation, spreadsheet)
-e, --export=name=value set export filter options
                             eg. -e PageRange=1-2
-f, --format=format      specify the output format
-i, --import=string      set import filter option string
                             eg. -i utf8
-l, --listener           start a listener to use by unoconv clients
-o, --outputpath=name    output directory
      --pipe=name          alternative method of connection using a pipe
-p, --port=port          specify the port (default: 2002)
                             to be used by client or listener
-s, --server=server      specify the server address (default: localhost)
                             to be used by client or listener
-t, --template=file      import the styles from template (.ott)
-T, --timeout=secs       timeout after secs if connections to OpenOffice fail
      --show               list the available output formats
      --stdout             write output to stdout
-v, --verbose            be more and more verbose

So, in its simplest form, conversion takes the following form:

$ unoconv test.docx

The command will finish silently if successful. It creates a .pdf by default in the same directory as the document. Add the -f [fmt] option to convert to a different format, for example:

$ unoconv -f txt test.docx

When your conversion work is done, close the listener with:

$ kill 9998

Now you see why unoconv is number one!

LibreOffice: An Unlikely Image Viewer

2011-11-21T14:44:00.000-08:00

I encountered a series of deleted Enhanced Metafiles (EMF) files during the examination of a Window's-based system the other day. EMF files are second generation Windows Metafiles (WMF), and early on in in most forensics careers, forensics examiners are taught to seek out those files as printer artifacts. And, it just so happens, the path of these deleted files and a time stamp analysis suggested these files were in fact printer artifacts.

Sidebar: EMF is not only a printer artifact. In fact, its not really accurate to call it a printer artifact. Windows applications, like Microsoft Office, use the EMF format to make images portable between applications. In printing from these applications, the document (even text documents) is converted to an EMF image and sent to the printer.

None of my native Linux image viewers was capable of displaying the EMF files. I was considering downloading and running XnView, and excellent image viewer with over 400 supported image formats. The new version being developed, XnViewMP (Multiplatform), is capable but unstable, so I didn't relish using it other than as a last resort. (And, as it turns out, EMF is not on the list of supported formats.)

A little poking around the Internet, and I discovered that OpenOffice supports EMF files. Open Office, for the uninitiated, is and open office sweet with Microsoft Office document compatibility. I have a fork of OpenOffice installed, called LibreOffice. I opened the Draw program (though Writer would have worked as well), and I dragged the EMF into the document window. Voila! I had a perfect representation of the document sent to the printer.

LibreOffice can save the image as a PDF for distribution, if required. While it might seem untenable to process numerous images in this manner (opening one at a time in LibreOffice to covert to another format), you may not be limited to this approach. The unoconv program can be used to convert any document that can be opened by LibreOffice into any format that can be written by LibreOffice. Automation, anyone?

F12 Temptations

2011-11-21T08:33:00.000-08:00

BIOS one-time boot menus are tempting, especially if you are a frequent forensics boot disc user like me. I use boot discs to conduct preview examinations, something many people know as "device triage," and for imaging. In fact, boot discs can be essential tools in certain device imaging scenarios (see Difficult Disk Imaging). One-time boot menus make it easy to select alternate boot devices, like CD-ROM or USB, without wading into the BIOS settings.

I'm in the habit of accessing the BIOS, however, and not using the boot menu. First, it gives me the opportunity to record the BIOS time settings and evaluate detected hardware, but also because one-time boot menus are not always clearly defined in startup messages. They also vary from BIOS to BIOS, computer model to computer model, even from the same manufacturer!

But Dell does a good job of letting you know how to access their menu: F12. And, there are a lot of Dell computers in the world. And, let's admit it, its a lot easier to access the one-time boot menu (meaning, boot from a different device than default this time) than to access the BIOS Settings. But, there is a very good reason not to be tempted by this Boot Device Nirvana, and I encountered it this weekend.

On Friday afternoon, I booted a Dell server with a Linux-based forensics boot disc. It was a hardware-based stripped array. After determining that the server had evidence on board, I chose to image it with Guymager. The process was to take 8 hours, and I went home in the bliss of knowing that when I returned to work on Monday, my image would be waiting for me, along with a hash verification of the source drive (array).

What I discovered upon my return--both from an email and the condition of my equipment--is there had been a power failure during the imaging process. Well, shortly after, in actuality. But, the server had been set to reboot on power failure, and I failed to note that. What I didn't fail to do, was change to boot order to ensure my book disc booted before the internal drives. Therefore, the system was sitting at the boot disc OS desktop and not running the server! In other words, no data had changed. Had I used a one-time boot menu, this would not have been the case.

You might be thinking, "Well, it didn't really matter in this case, because the imaging process had completed." To that I say, "Yes, you're right." But, I didn't have a verified image, because, for an unknown reason, the hash digest of the source device did not match the hash digest of the data in the image. However, because the server booted from the boot disc, and not the internal drives, I was able to hash the source device again and verify the data was the same in the image as on the drive!

Why did the Guymager source hash differ from the image and from my post-power failure verification? I don't know. But I had the opportunity to check the discrepancy, which is the point of this post.

Windows Link Files / Using While Loops

2011-08-01T17:13:00.000-07:00

A colleague asked today about a tool to decode Microsoft Windows link files. Before I begin to discuss the tool I recommended, I'll briefly describe Windows link files. A Windows link file, commonly known as "shortcuts", has the .lnk extension. It is basically a file with information about another file. It is interpreted by the windows shell, and the shell opens the file pointed to by the link file if all goes well. Link files can be user or system generated. You can learn much more about Windows link files in the white paper by Harry Parsonage.

My favorite tool for parsing Windows link files was created by Jake Cunningham who posts tools he creates at the "JAFAT: Archive of Forensic Analysis Tools" website. Jake has created some very well know Safari browser tools, but the tool that has caught my eye is "lnk-parse-1.0.pl". The lnk-parse tool is a perl script that provides the most detail I have ever seen in a link file parser. Here is some sample output:

Link File:  /WINDOWS/system32/config/systemprofile/Start Menu/Programs/Windows Media Player.lnk
Link Flags:  HAS SHELLIDLIST | POINTS TO FILE/DIR | HAS DESCRIPTION | HAS RELATIVE PATH STRING | NO WORKING DIRECTORY | HAS CMD LINE ARGS | NO CUSTOM ICON |
File Attributes: ARCHIVE
Create Time: Wed Jan 30 2008 05:36:33
Last Accessed time: Wed Jan 30 2008 05:39:37
Last Modified Time: Mon Jul 07 2003 05:00:00
Target Length: 520192
Icon Index: 0
ShowWnd: 1 SW_NORMAL
HotKey: 0
Target is on local volume
Volume Type: Fixed (Hard Disk)
Volume Serial: b0a65d8e
Vol Label: 
Base Path: C:\Program Files\Windows Media Player\wmplayer.exe
(App Path:) Remaining Path: 
Description: @%SystemRoot%\inf\unregmp2.exe,-155
Relative Path: ..\..\..\..\Program Files\Windows Media Player\wmplayer.exe
Command Line: /prefetch:1

Lnk-parse takes a single link file as an argument. But anyone who knows Windows knows there are usually hundreds of link files on a system. So, how does one quickly find all the link files in a Windows operating system using Linux tools? I'm going to avoid all the caveats, such a deleted files, renamed files, etc., and make the following assumption: The examiner is interested in allocated link files that have not been obfuscated by renaming or other anti-forensics techniques.

The fastest way to find the link files is to mount the Windows file system and search for link files by name. The search tool to use is the 'find' command. I'll discuss mounting in another post (I recently wrote a tool to read disk/forensic image partition tables an automatically mount all allocated filesystems), but here I want to show a how a simple while loop can make using commands like lnk-parser on multiple files easy.

The following command assumes it is being executed from the root of the Windows file system. Windows file system is a misnomer; I mean a file system in which the Windows operating system is installed.

$ find . -iname "*.lnk" | while read line; do lnk-parse-1.0.pl "$line"; done

I'll break that down by moving the major parts to their own lines:

find . -iname "*.lnk" | \  
while read line  
do 
    lnk-parse-1.0.pl "$line"  
done

Line by line:

find, recursively from the current dir, all files ending with .lnk, regardless of case, and pipe the output to the while loop
"while" there is output from the find command, read each line of output and assign the output (a file name and path) to the variable "line"
a required argument in a while loop, essentially: "do" the following to $line
process $line with lnk-parse-1.0.pl
Finished with this loop cycle, return to while and get another line from "find", if any.

In 1.23 seconds, I processed all 103 link files in a small Windows XP system. I could pipe the output to a text file for a report, if necessary.

$ find . -iname "*.lnk" | while read line; do lnk-parse-1.0.pl "$line"; done > some_path/link_files.txt

I use while loops like this all the time. One gotcha of which to be aware is that variables, assigned within the while loop, don't survive outside of the while loop. I'll explain that sometime in the future, and provide some strategies on how to handle this. But for now, if you mimic this usage, you won't run into that issue, you'll save yourself a lot of work, and you'll increase your productivity tremendously.

BlackBerry Messenger / Google Talk for BlackBerry Save-files

2011-07-26T15:39:00.000-07:00

My last post, "BlackBerry Text Message Parsing, AKA, Why I use Linux for Forensics," had unintended but pleasant consequences: it introduced me to BlackBerry forensics expert Shafik Punja. I really wrote the post to comment on the versatility of the Linux operating system, and fortunately I erred in calling the file that was the subject of my discussion a "BlackBerry Text Message" save file, rather than the BlackBerry Messenger (BBM) save file that it was.

Shafik corrected me and taught me a bit more about the file type that I'll share here in the event that you missed his comments. BBM messenger does not save message content by default. It is a user selected option, and only available in BBM v5 and higher. The files can be saved to the device's internal memory or an installed memory card. Google Talk has the same save options and defaults as BBM, and also saves in the same basic format.

Shafik asked if I could write a program to parse the files. I agreed and quickly converted the basic Bash script in my last post to a more complete program. However, when Shafik tried to run the program in OS X, it failed because OS X and Linux do not utilize the same 'date' command. I was faced with writing a new program for use in OS X, which I wished to support, or make my current script intelligently determine the OS in which it was running and apply the correct date command syntax.

Nothing seems like a bigger waste to me than to write essentially the same thing twice. Isn't there a way to write a program that doesn't have to be altered based on the operating system in which it is run, I asked? Then a little Australian-accented voice of reason reached my conscious thought: why don't you learn python? For the millionth time: Thank you Michael, good idea.

Michael Cohen, a software engineer at Google and author/maintainer of pyFLAG, AFF4, and other tools soon to be released, has suggest many times over that I learn python, but I only recently started to do so. Python is an excellent solution for cross platform programs, and I decided it was time to start.

The BBM / GTalk file format

BBM and GTalk save files are CSV text files with the following basic structure:

Application Name, version[newline]
DateCode, "Sender_ID", "Receiver_ID", Message[newline]
...

The first line of the file is the source application and the application version, separated by a comma. The version is followed by a newline control code.

The remaining lines are formatted thus: the line starts with a DateCode consisting of a 21 digit number. The first 8 digits are the date of the sent message rendered as YYYYMMDD. The remaining 13 digits are Unix epoch time, rendered in milliseconds. In the case of BBM, the sender Hex_ID follows the DateCode, separated by a comma and encapsulated in parenthesis. Similarly, the receiver's Hex_ID follows. For GTalk, the sender and receiver are identified by Google Account ID. Finally, the message, in text, is stored followed by a newline control code.

One minor issue in rendering the file in a spreadsheet centers on the message field. Messages are not encapsulated in commas; therefore, messages with commas will not format well in spreadsheets without additional spreadsheet configuration. This might not be a problem for one file, but would be a major annoyance for many files (one of my colleagues found 23 BBM save files on an SD card). I determined to encapsulate the message field in quotes to overcome this issue.

The python3 code I settled on follows:

import sys
from time import strftime, localtime

def main(csv):
    line_no = 0
    with open(csv) as db_file:
        for line in db_file:
            if line_no == 0:
                print(line)
                print('Date,DateCode,Sender,Receiver,Message')
                line_no += 1
            else:
                datecode, sender, receiver, message = line.split(',', 3)
                date = int(datecode[8:18])
                date = strftime('%Y-%m-%d %H:%M:%S (%Z)', localtime(date))
                print('"{}",{},"{}","{}","{}"'.format(date, datecode, sender, receiver, message.strip()))

if __name__ == '__main__':
    for file in sys.argv[1:]:
        main(file)

I won't try to explain all the code here, because with my rudimentary knowledge of python, I'm likely to misspeak. I will highlight a couple parts of the script and a couple of nice python features, however. First, the "if" condition in line 8 is present to skip over date decoding in the first line of the file. Instead, it prints the line, and adds a line of column headers to help with interpreting the remaining message data.

Line 13 demonstrates a very nice python feature: multiple value assignment. Here, I assign data from line 2 and higher, one line at a time (note the 'for' loop) to the datecode, sender, receiver, and message objects. To work correctly, however, there needs to be four values to assign to the four objects. The split method divides a string object into a list of strings based on a delimiter. Here, I configure split to divide on commas a maximum of three times. This creates a list of four strings. Remaining commas are ignored and included in the last string, which here is assigned to 'message'.

Line 14 converts the 'datecode' string to an integer, and line 15 uses the 'strftime' and 'localtime' functions from the 'time' module to convert the Unix epoch time embedded in the datecode to a human-readable format. I use string slicing, that is, selecting a portion of the datecode string by its position in the string (also called indexing where the starting position is 0), to extract the 10 digits that represent the Unix epoch value. The last three digits of the datecode are ignored since they represent milliseconds.

The final line of which to take note is Line 16, where a few things of import are occurring. First, the interpreted datecode is printed. The original datecode follows for reference and validation purposes, and then the remainder of the line. A format method is applied that replaces the brackets place holders in the print string with the objects that are arguments of the format method. Here, the message object is encapsulated in quotes. Also, the message object has the strip method applied to remove the newline control code at the end of each message to improve formatting.

Though I won't comment on it, I'll include the bash script I wrote first for comparison:

line_no=0 #set counter

cat $1 | while read line
do
    if [ $line_no -eq 0 ]
    then
        echo ${i}  #print first line unaltered
        echo "Date,DateCode,Sender,Receiver,Message"
        line_no=1
    else
        for var in date sent rcvd message
        do
            eval $var="\${line%%,*}"
            [[ "$var" = "message" ]] && message=$line #ignore commas in message
            line="${line#*,}"
        done
        date=$(date +"%Y-%m-%d %H:%M:%S (%Z)" -d @${date:8:10}) #convert date code and store
        len=$((${#message}-1)) #subtract 1 from remainder length for stripping newline control code
        echo "\"$date\",\"$sent\",\"$rcvd\",\"${message:0:$len}\""
    fi
done

BlackBerry Text Message Parsing, AKA, Why I use Linux for Forensics

2011-07-21T22:00:00.000-07:00

A little detour from my usual posts to explain why I use Linux for forensics, though my upbringing was in Windows-based tools like EnCase. A colleague contacted me today with a little issue: He had found a BlackBerry text messaging backup file in CSV format (EDIT: This was actually a BlackBerry Messenger save file) on an external memory card, but the date code for each message was perplexing. He asked me if I could help in interpret the code. It looked like the following:

201010181287467321760

The full format of the CSV was "date, from(hexID), to(hexID), message." It was obvious to my colleague and me that the first 8 digits of the date code was the date of the message in plain text, i.e., "20101018" or "2010-10-18." My Unix roots made the remaining digits of the numeric string easy to identify: unixepoch in milliseconds, i.e., 1287467321760 milliseconds since 1-1-1970 00:00:00.

A quick verification with the date command, but truncating the date at seconds (i.e., dividing by 1000):

$ date -d @1287467321
Mon Oct 18 22:48:41 PDT 2010

The converted unixepoch date matches the plain text date. We seem to have interpreted the code correctly, and now we know the local time of the message as well.

But of course, using the date command is not why I find Linux so valuable. It because of the ease with which I was able to convert the whole file, having discovered the meaning of the date code. Remember the format of the file? It was "date, from(hexID), to(hexID), message."

Consider three lines from the file as an example:

201010181287467321760,"6C31FB2C","0F315216",Hey
201010191287534544913, Oct 19 17:29:04 PDT 2010,"6C31FB2C","0F315216",Hey, you there?
201010191287534602157,"0F315216","6C31FB2C",Yeah, let's meet.

A quick, simple while loop to read each line of the Messenger save file,

$ cat backup.csv | while read line; do date=${line%%,*}; remainder=${line#*,}; echo "$(date -d @${date:8:10}),$remainder"; done
Mon Oct 18 22:48:41 PDT 2010,"6C31FB2C","0F315216",Hey
Tue Oct 19 17:29:04 PDT 2010,"6C31FB2C","0F315216",Hey, you there?
Tue Oct 19 17:30:02 PDT 2010,"0F315216","6C31FB2C",Yeah, let's meet.
...

Let me break that down:

cat backup.csv | while read line  #read a line of the file, assign to variable 'line'
do
  date=${line%%,*}  #read everything up to the first comma, assign to variable 'date'
  remainder=${line#*,}  #read everything beyond the first comma, assign to variable 'remainder'
  echo "$(date -d @${date:8:10}),$remainder" #convert digits 9-18 to local time, print localtime and the remainder of the line to stdout
done

The key to this solution is something I didn't learn in my initial studies of Bash, but I make extensive use of it now: variable expansion. I use the various expansions available in Bash 4 to assign portions each line to variables that I could then operate on and print the result. I won't discuss all available expansions here, but I will explain those I used:

Removing the longest match from the end:

Consider: Each line contained comma separated values. I really only needed to operated on the first value -- the date code. I prefer, as much as possible, to not call external tools, such as cut or awk, so as to not unnecessarily start external processes. Bash variable expansion makes this possible. The syntax ${var%%PATTERN} will remove the longest match to PATTERN from the variable 'var'.

So, in line 3, date=${line%%,*} assigns to the 'date' variable all of the contents of the 'line' variable up to the first comma. Thus, in the case of the first line, date="201010181287467321760".

Removing the shortest match from the beginning:

With the date code isolated in the 'date' variable, we still need to print the rest of the line once we convert the date. The syntax ${var#PATTERN} will remove the shortest match to PATTERN from the variable 'var'.

So, in line 4, remainder=${line#*,} assigns to the 'remainder' variable all of the contents of the 'line' variable after the first comma. Thus, in the case of the first line, remainder=""6C31FB2C","0F315216",Hey"

Returning a substring of a variable:

Finally, we need to isolate the unix epoch time from the plain text date in the date code now stored in the variable 'date'. We do this by indexing. The syntax ${var:OFFSET:LENGTH} will return a substring of the variable 'var' starting at OFFSET for the specified LENGTH. The first character in a variable is indexed at offset 0.

So, in line 5, ${date:8:10} returns 10 characters of the variable 'date' starting at the 9th character (remember, indexing starts at 0). Thus, we have now fed the unix epoch date string incorporated in the Messenger date code to the unix date command to be converted to local time in a human readable format.

Line five is a complex command, that indexes the 'date' variable, converts it in a sub-process with the date command, and then echo the result with the contents of the 'remainder' variable appended.

Where to go from here:

If you have some Bash skills, but want to advance them, I recommend the book "Pro Bash Programming: Scripting the GNU/Linux Shell" by Chris F.A. Johnson from Apress.

Mounting Split Raw Images

2011-07-19T18:37:00.000-07:00

A raw image, made with dd or a variant, is still a common image format, and will not go away soon even as many argue the benefits of forensic images such as the Expert Witness Format (supplied through libewf) and the Advanced Forensic Format (supplied through afflib). But raw images can be difficult to tote around because they are bit for bit copies which makes the copy as large as the original. As such, the images are often split to fit on external media such as DVD.

But splitting, while solving storage problems, creates a new problem. What if you want to mount the image for examination? True, Sleuthkit can handle the examination of split raw images, but sometimes there is no equal to simply mounting an image during an examination.

Let me illustrate using a situation I encountered yesterday. A colleague had a split raw image of over 200 segments that he wished to mount and then boot in a virtual machine. He tried to follow my tutorial but was unsuccessful, uncertain as to why. When I looked into the situation with him, the issue became clear: xmount, the tool used to create a virtual disk from a disk image, was only mounting the first segment of the split raw image, despite being given all the segments as arguments as is required with Expert Witness Format images. More simply put, xmount does not handle split raw images. It will handle a single raw image file just fine, however.

What to do? One could simply cat the files together, but that means doubling storage requirements, at least until the concatenation operation is concluded. That might not be feasible or desirable, and it can be very time consuming. In this case, we were talking 300 GB of data. It would be great to be able to treat the segments as one file, and pass that file to xmount to accomplish the purpose.

Affuse to the rescue! Affuse is part of the afflib tool suite. It creates a virtual file system using fuse and mounts it to a location you specify. You only pass the first segment of the split image as an argument. The command takes the form:

# affuse image mount_point

Affuse creates an image.raw file (that is, the name of the segment with '.raw' appended) in the mount point along with a log file. Yes, its that easy.

To finish the scenario, xmount can then take the image.raw file as an argument to create the virtual disk, thusly:

# xmount --in dd --out vdi --cache image.cache mount_point/image.raw new_mount_point/

This command tells xmount that the input file, image.raw, is raw data, the output desired is a VirtualBox vdi format, that a cache file called "image.cache" is desired to store system changes when the virtual machine is running. The .vdi file will be mounted in the "new_mount_point" directory. If xmount is unfamiliar to you, I recommend you read my previous post.

Like affuse, xmount utilizes the fuse file system. Both utilities accept fuse file system arguments as well as tool specific arguments, so read only mounting and permissions options exist (type "man fuse" at the command line for more details). As always, practice on non-case data to become familiar with the tools.

Google Chrome Download History

2011-06-29T13:35:00.000-07:00

Google Chrome keeps a wealth of data that is of interest to the forensic examiner. There are tools that look at the browser history and cache, but tools that examine the download history are less frequent. Part of the reason it is overlooked by examiner's, I suspect, is that the download history is a table in the 'History' SQLite database, and not a separate file itself.

The History database, found in '/home//.config/chromium/Default' for the open version of chrome, i.e., Chromium, contains the following tables:
$ sqlite3 History .tables
downloads presentation urls
keyword_search_terms segment_usage visit_source
meta segments visits

The downloads table is the focus of this post. It is structured as follows:

$ sqlite3 History .schema | grep downloads
CREATE TABLE downloads (id INTEGER PRIMARY KEY,full_path LONGVARCHAR NOT NULL,url LONGVARCHAR NOT NULL,start_time INTEGER NOT NULL,received_bytes INTEGER NOT NULL,total_bytes INTEGER NOT NULL,state INTEGER NOT NULL);

For the uninitiated, the table schema tells us that each record contains 'id', 'full_path', 'url', 'start_time', 'received_bytes', 'total_bytes', and 'state' fields. We can extract the data in csv format, as follows:

sqlite -header -csv History "SELECT * FROM downloads"

That's nice, but the dates are unixepoch and we don't know what the 'state' integer means, though it can be deduced from the 'received_bytes' and 'total_bytes' fields and studying a known source of data. I determined that state 1 and 2 mean "complete" and "incomplete" download respectively.

To convert unixepoch to local time, we use the SQLite datetime function. We tell datetime to take the start_time integer, interpret it as unixepoch time, and translate to localtime. Exerpted from the rest of the query, it looks like this:

datetime(start_time,'unixepoch','localtime')

For clarity in the output, it would be nice to convert the state integer to its text equivalent. We can do this with the case command. The case command is the if/then statement of SQLite. In its simplest application it takes the form 'CASE WHEN x=w1 THEN r1 WHEN x=w2 THEN r2 ELSE r3 END' where x is the field data, w is a comparison value, and r is the result that is returned. It is easiest to understand when we apply it to our current circumstance:

CASE WHEN state=1 THEN 'complete' WHEN state=2 THEN 'incomplete' ELSE 'unknown' END

Putting it all together, we can obtain nicely formatted output from the downloads table of the Chrome History database with the following command:

sqlite3 -header -csv History "SELECT id,full_path, url, datetime(start_time,'unixepoch','localtime') AS date, received_bytes, total_bytes, CASE WHEN state=1 THEN 'complete' WHEN state=2 THEN 'incomplete' ELSE 'unknown' END AS state FROM downloads"

The caps in the commands are not required, but are designed to make reading the SQLite operators easier. You may have noted the "AS" operators after the datetime function and case expressions. It serves to replace the expressions as the header for that column in your csv output.

Your output should look like:

id,full_path,url,date,received_bytes,total_bytes,state
1,/home/slosleuth/Downloads/keepnote_0.7.3-1_all.deb,http://keepnote.org/keepnote/download/keepnote_0.7.3-1_all.deb,"2011-06-23 12:04:46",500022,500022,complete
2,/home/slosleuth/Downloads/iso/nbcaine2.0.dd.gz,http://www.caine-live.net/Downloads/nbcaine2.0.dd.gz,"2011-06-24 13:25:42",85962197,0,incomplete
3,/home/slosleuth/Downloads/iso/nbcaine2.0gz.md5.txt,http://www.caine-live.net/Downloads/nbcaine2.0gz.md5.txt,"2011-06-24 13:27:23",51,51,complete

I hope this helps you with SQLite queries in general and specifically to parse the data in the Google Chrome downloads table of the History SQLite database.

Extending gThumb for forensics

2011-06-13T13:16:00.000-07:00

I've searched high and low, and many times, for a good image viewer to be used in Linux-based forensics. In the end, and despite some shortcomings, I always end up using gThumb, a GUI image viewer and browser. The advantages of gThumb are that it integrates well with the Gnome desktop environment, has a robust search facility, finds files recursively, and can be extended by the user.

The focus of this post is extending gThumb. Today, many images are rich in EXIF data, much of which gThumb displays in a property window. However, it misses data like GPS coordinates that can be found in some images like those taken by the iPhone. My favorite tool for reading EXIF data is exiftool by Phil Harvey, but it's a command line tool. So, how does one integrate command line data with a GUI tool like gThumb? Yad!

No, I didn't just utter an explicative. Yad stands for "yet another dialog" and is a fork of the zenity project. Zenity is a simple to use GUI front end for command line scripts. However, yad has many more features and is under active development. Yad had many stock dialog boxes, and I will illustrate its use here as I extend gThumb with exiftool by displaying the exiftool output in a yad dialog box.

gThumb has the option to run user scripts, accessed through the toolbar: Tools | Personalize.

This opens the "Command" dialog box. Clicking the "New" button opens the "New Command" dialog where the user command is inputted.

Assuming both yad and exiftool are installed in you operating system, you can create an exiftool extention as follows:

Name: exiftool

Command: exiftool %F | yad --text-info --title="exiftool: %B" --fontname="Monospace 10" --width=600 --height=800

Shortcut: <select from drop down if a shortcut key is desired>

Terminal command (shell script): leave checked

Save the command and close the main Command box. You can select your new command from the tools menu or by the number pad shortcut you created.

The results appear in a yad "text-info" dialog box:

If you made a mistake, or need to edit your command use the Command window through "Tool | Personalize."

With this basic format, gThumb extensions are only limited by your creativity!

SD Card Construction, (or Burning Ants with a Magnifying Glass)

2011-06-13T08:28:00.000-07:00

A coworker brought me an SD card today because she could not delete any files from it. I noticed the lock switch was missing from the card, and after inserting the card into a portable reader, I confirmed the card was write protected. This got me curious: just how does the switch on an SD card work? I've relied on it for write protecting evidence, but I didn't really know how reliable the switch was.

With great gusto, I snapped the card in half to examine its contents. I did it with as much joy of discovery as a young boy has when first focusing the suns rays on ants with a magnifying glass. What I found inside surprised me: a very simple chip occupying about 1/3 of the card housing. I determined--by examining similar cards--that the switch did not bridge any electrical contacts on the chip, so write protection was not a function on the card itself.

I turned my attention to several card readers I have lying about. One has a very shallow card well, which makes visualizing the electrical contacts quite easy. I noted that on the side of the reader, in a location corresponding to the switch on an SD card, there was a spring-loaded pin. Through experimentation, I determined that the pin, when depressed, allows data to be written to the card. When the pin is extended, write-blocking occurs. The position of the "lock" switch on the card determines the position of the pin.

Thus, (and to my surprise) write blocking is a really function of the reader, not the card. It is possible for a reader to be constructed or damaged such that the lock switch has no effect! Frequent inspection and testing of a card reader used for forensic analysis is warranted.

Lesson: know your equipment!

Defeating the Droid: Let the Pillaging Begin

2011-06-03T17:04:00.000-07:00

In my last post, Defeating the Droid, I explained how to root an Android phone, at least through version 2.2. Like other mobile devices, Android is a moving target and what works on one phone or one OS version may not work on the next. We left off with a running adb shell with root privileges. But how do we get the data we seek.

Bringing Up the Past

I mentioned previously that the adb shell is an ASH shell without auto-completion, inline editing, or command history. In other words, this car is a base model without all the bells and whistles with which you might be accustomed. Add to that a limited set of binary tools (no copy, tar, or find commands, for example) and its a bit like standing at the base of a cliff without any climbing tools.

Plan of Attack

We need tools, and we need them bad. But we don't want to change any more data on the phone than necessary. So, what to do?

Android phones use SD cards to store user media, like photos and music. This card should be removed and imaged, following standard forensic imaging protocols, as part of your examination. While the card can be imaged from within the phone, the interface is slow (took 45 minutes to image 16gb) and requires that the phone be placed in USB mass storage mode for the workstation to see it. A locked phone may make this impossible.

Significantly, removing the card allows us to replace it with one of our own containing tools for accessing data in the phone's internal memory. You can get by quite handily with one small tool: busybox.

You can copy busybox to the card before inserting it in the phone, but its not a necessity. You can also use the Android Debug Bridge (adb) utility to copy busybox to an installed card using the command:

# adb push busybox /sdcard

Finally, the card you use should have enough space to capture the full internal memory of the phone. The adb shell allows us to copy files within the device, but not to our workstations. So, we must copy files to the SD card.

The size of our SD card can very, depending on the amount of physical memory found in the phone. This can be quite small, as in the case of the original Motorola Droid which has only 256mb of internal storage. However newer devices, like the Droid 2, have significantly more memory with 8gb of internal storage capacity.

Sequential Processing: Starting with "dd"

I'm experienced in fingerprint development as well as computer forensics. In fingerprint processing, there is a phenomenon known as "Sequential Processing." Sequential processing simply means to do processing in such an order so that the first step does not inhibit the second step, and so on. We actually do this every day in computer forensics: we image a device before before we conduct an examination, and we use the image for the examination so as to not harm the original evidence.

It is my opinion that we should do physical imaging of the device before logical. The device we are working on has a running operating system and files are changing and being created and deleted as part of the normal process of the phone. Further, a physical image means we can recover deleted files, so we want the physical images before we do anything that could overwrite data. "dd" is the tool of choice here, and it can be deployed quickly (it is part of the resident operating system) and there is no set up involved, i.e., remounting of partitions.

Root the device. See my previous post if you don't know what I'm talking about.
Start the adb shell if its not already running. Look for the telltale root prompt to be sure you have "rooted" the device.

# adb shell
# <-- adb shell prompt
Determine the devices you want to image. In Android, the devices are called mtdblock devices, and are listed in /proc/mtd.

# cat /proc/mtd
dev: size erasesize name
mtd0: 00180000 00020000 "pds"
mtd1: 00060000 00020000 "misc"
mtd2: 00380000 00020000 "boot"
mtd3: 00480000 00020000 "recovery"
mtd4: 08c60000 00020000 "system"
mtd5: 05ca0000 00020000 "cache"
mtd6: 105c0000 00020000 "userdata"
mtd7: 00200000 00020000 "kpanic"
Ideally, we want to image everything. But of particular interest are the "cache" and "userdata" block devices. The size of each device is listed in hexadecimal code. In the above case, the "cache" device is 97124352 bytes [echo $((0x5ca0000))], and the "userdata" device is 274464768 bytes [echo $((0x105c0000))]. We need to know how the devices are addressed:

# mount
rootfs / rootfs ro,relatime 0 0
...
/dev/block/mtdblock4 /system yaffs2 ro,relatime 0 0
/dev/block/mtdblock6 /data yaffs2 rw,nosuid,nodev,relatime 0 0
/dev/block/mtdblock5 /cache yaffs2 rw,nosuid,nodev,relatime 0 0
/dev/block/mtdblock0 /config yaffs2 ro,relatime 0 0
...
We see that the "cache" mtdblock device is addressed as /dev/block/mtdblock5 and "userdata" as /dev/block/mtdblock6. We can make a dd image of each as follows:

# dd if=/dev/block/mtdblock5 of=/mnt/sdcard/cache.dd
189696+0 records in
189696+0 records out
97124352 bytes transferred in 24.001 secs (4046679 bytes/sec)
# dd if=/dev/block/mtdblock6 of=/sdcard/data.dd
/dev/block/mtdblock6: read error: I/O error
83808+0 records in
83808+0 records out
42909696 bytes transferred in 8.653 secs (4958938 bytes/sec)

Uh, oh, an error. Normally, we overcome dd errors by adding the "conv=noerror" argument, which tells dd to continue on read error. But the Android dd does not support that option. So, what do we do in such a circumstance? Give up? Hardly!

Phase Two: Busybox

Busybox can be described as "the swiss army knife of embedded Linux." It is a single executable file, about 1mb in size, containing many common Unix utilities, including cp, find, tar, and yes, dd! In order to use it, we need a version compiled for Android, but I will not discuss how to compile here. It is readily available on the Web and in Android Root kits, but I'd recommend compiling a trusted version if you can. One things certain, you cannot use the busybox binary from your workstation.

There's a catch to using busybox. We want to place it on our SD card so we don't change anything on the phone's internal memory, but busybox cannot be run from the SD card. "Why?" you ask. I'll show you:

# mount
...
/dev/block/vold/179:1 /mnt/sdcard vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0

So, to make the busybox on the SD card executable, we need to remount the card with the ability to execute binary files.

If busybox was not already on your card, push it to the card from a terminal on your workstation.

# adb push busybox /mnt/sdcard
1255 KB/s (1745016 bytes in 1.356s)
Remount your card with executable permission.

# adb shell
# mount -o remount,rw /mnt/sdcard /mnt/sdcard
# mount
/dev/block/vold/179:1 /mnt/sdcard vfat rw,dirsync,relatime,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0
Change directories to the SD card and optionally start a busybox shell. By starting a busybox shell, we attain inline editting and a shell history. In other words, entering commands becomes easier.

# cd /mnt/sdcard
# ./busybox sh
Now we can dd that damaged "userdata" mdtblock again, this time passing conv=noerror because we will use the busybox dd. Note: because busybox is not installed on the system or in the path, we have to prepend busybox utilities with ./busybox . To see a list of available busybox tools, issue "./busybox" alone.

# ./busybox dd if=/dev/block/mtdblock6 of=/mnt/sdcard/data.dd conv=noerror
With busybox, we can also create archives of the logical files on the device. Again, to simplify the discussion, we'll focus on the /cache and /data directories where we saw the respective mtdblock devices were mounted. Why should we archive the logical files when we have physical images made with dd? Because the file systems in the images are "yaffs" and present special problems I won't discuss here. It is enough to say, for now, that your workstation likely doesn't support yaffs by default and you cannot mount and examine the images you created. They are only useful file carving.

Therefore, we'll use tar to make archives of the /cache and /data directories.

# ./busybox tar czvf cache.tar.gz /cache
tar: removing leading '/' from member names
cache/
cache/downloadfile-30.jpeg
cache/downloadfile-15.jpeg
...
cache/app/I-Status_Monitor.apk
# ./busybox tar czvf data.tar.gz /data
tar: removing leading '/' from member names
data/
data/tombstones/
data/tombstones/tombstone_05
data/tombstones/tombstone_00
...
data/lost+found/

Wrapping Things Up

When you've acquired the images/files that you seek, you need to get the data from the SD card to your workstation. The easiest way is to turn off the phone and remove the card, the read it directly. But you can pull the files from the card with adb, too. You might want to do this if you need the data quickly, or to free up space on a smaller card for more images/archives. To pull the files from the card to your workstation:

Use "adb pull" to move files from the SD installed in the phone to your workstation. The command is run in a terminal on your workstation, NOT in the adb shell. You can exit adb shell or open a new terminal window if you want to remain in the adb shell environment.

# exit <-- exits the busybox shell
# exit <-- exits the adb shell to your terminal
# adb pull /mnt/sdcard/cache.tar.gz /some/evidence/directory/
# adb pull /mnt/sdcard/cache.dd /some/evidence/directory/
...
When you are done pulling all the files you plan to transfer, remove the exploit from the install directory.

# adb shell rm /data/local/tmp/ratc.bin

Note: I chose this method of removing the exploit to demonstrate that the adb shell command can pass arguments to the adbd daemon on the phone without dropping into the shell. I hope to use this ability to script the process we've been discussing.
Reboot the phone to kill the exploit and the phone is back in its pre-exam condition.

I'll leave the examination of the data you've recovered to you for now. Maybe in a future post, I'll highlight some key files and how to process them (hint: mmssms.db contains text messaging, and cache.cell contains GPS data).

Defeating the Droid

2011-06-03T13:18:00.000-07:00

I received a Motorola Droid 2 for analysis. Processing the 16gb SD card was trivial as it is removable and subject to traditional imaging techniques. But what of the internal phone memory? After a lot of researching, I found there just wasn't a lot of information out there for forensic examiners. I settled on a procedure called "rooting" which I'll document here.

Caveat 1: I wouldn't call this a "forensically sound" approach, per se, in that it makes changes to the phone. But I wouldn't say this leads to automatic disqualification of the evidence, either. The technique was a necessary approach under the circumstance of current technology to meet the needs of my investigation. The techniques and their impacts on the evidence can be explained and therefore may be admissible. Your mileage may vary.

Caveat 2: The "rooting" process I describe here borrows from common techniques found on the Web that were developed by users of Android phones to gain full access to their devices. But, the process described here differs significantly. Do not follow the user-oriented tutorials for rooting devices as they require you to install files unnecessary to forensics and make significant changes to the device, sometimes erasing user data!

Preparing the Phone

Luckily, the phone I received was unlocked. I accessed the settings, disabled wireless networking, and placed the device in airplane mode to minimize changes and prevent remote access. I also disabled the password and automatic screen locking, and I determined the OS version (v2.2.2) through "Settings | About Phone", in this case 2.2.2. Two other settings are required: set the device to enter USB debugging mode when connected by USB and set it to stay awake while charging through "Settings | Applications | Development."

I have since learned that the technique I describe below will work on locked Andriod phones that have USB debugging enabled. Working with another investigator, we rooted a locked Virgin Mobile LG Optimus and a locked HTC Incredible.

Preparing the Workstation

I downloaded and installed the Android SDK which includes the very useful Android Debug Bridge (adb) utility. Installation of the Linux version involves extracting the compressed archive (tar) and copying to a directory of your choice. In my case, I chose '/usr/local'.

# tar xzvf android-sdk_r11-linux_x86.tgz.tar -C /usr/local

Update the SDK to ensure you have the latest tools

# ./usr/local/android-sdk-linux_x86/tools/android update sdk

Creating a link to adb somewhere in your path, such as /usr/local/bin makes running the utility easy and is recommended. I created a link with the following command:

# ln -s /usr/local/android-sdk-linux_x86/platform-tools/adb /usr/local/bin/adb

Working with the Android Debug Bridge

With the SDK installed and the device connected by USB cable, the command 'adb devices', run as root, will start the adb server and show the serial number of a USB-attached Android device. If the serial number is represented by a series of question marks, kill the sever with 'adb kill-server' and restart with 'adb root'. The Android version and phone model influence whether or not you can start the adbd daemon as root. I recommend attaching only one device at a time so that subsequent commands need not specify the device on which to operate. Use 'adb help' to list all possible commands and a brief description of their use.

Default ADB Shell Access

There is limited access to the device file system by dropping into the adb shell with the command 'adb shell'. This is true even if you are root when you start the adb server and if you start the adbd daemon with 'adb root'. You'll note that you have the standard '$' user prompt. In other words, when you launch the adb shell, you are dropped into in a shell console on the running phone with standard user privileges.

The adb shell is an ASH shell with limited binary tool availability. For example, there are no copy or find commands. There is no inline command editing or shell history. Your limited privileges prevent you from accessing the /data and /cache directories which contain the bulk of the user data you are likely seeking. Other available adb commands (not part of adb shell), can be used to recursively copy files from the device to your local system (adb pull) or place local files into the Android file system (adb push), but your limited privileges still impede you.

Escalating Permissions with Rageagainstthecage

The solution is to "root" the device. Rooting, in this case, means to escalate your privileges in the adb shell to super-user status. I could not find a rooting method that prevented any changes to the phone's file system (the exploit has to be loaded into a partition on the phone), but I settled on the small 5k 'rageagainstthecage' exploit, which is an adb setuid exhaustion attack. In a nutshell, the exploit crashes the adbd daemon on the phone and prevents it from deescalating root privileges when it restarts. I know of no 'trusted' exploits for forensics, and this seemed the soundest approach to minimize changes to the running operating system on the device.

A variety of exploits can be downloaded from the Android Devices Website, but it is not the only location for rooting information. The xda-developers website is another good resource. In the case of the Motorola Droid 2, I'll use the rageagainstthecage binary file from the developer, which includes the source code for review.

Installing and Running the Exploit

The process of rooting is straight forward: Push the file to the device, make it executable, and run. The result is a crashed adbd daemon, that when restarted, allows the adb shell to start with root permissions.

Extract the the rageagainstthecage-arm5.bin bin from the tar file.

# tar xzvf RageAgainstTheCage.tgz.tar rageagainstthecage-arm5.bin
(optional) Rename rageaginstthecage-arm5.bin to something easy to type, like ratc.bin.

# mv rageagainstthecage-arm5.bin ratc.bin
Push the file to a user-accessible location on the phone where execution of files is permitted. I'd prefer a different location than the /data partition, but so far this has been my best option.

# adb push ratc.bin /data/local/tmp/ratc.bin
Drop into the shell, make the file executable, and run. Note the prompt change after executing adb shell as root.

# adb shell <-- your root terminal
$ chmod 0755 /data/local/tmp/ratc.bin <-- adb shell
$ ./data/local/tmp/ratc.bin
[*] CVE-2010-EASY Android local root exploit (C) 2010 by 743C

[*] checking NPROC limit ...
[+] RLIMIT_NPROC={1803, 1803}
[*] Searching for adb ...
[+] Found adb as PID 3323
[*] Spawning children. Dont type anything and wait for reset!
[*]
...
[*] adb connection will be reset. restart adb server on desktop and re-login.
$ # <-- your root terminal again

You will be dumped out of the adb shell back into your terminal. Be patient, as this can take a few moments. The exploit is launching processes and creating a race condition to prevent the adb daemon from deescalating privileges on restart. If you are not dumped from the shell, run the exploit again.
Finally, restart the adb shell (note the root prompt indicating you are now the root user on the device). DO NOT restart the adb server, as the last system message indicates. It will restart on its own , if needed, when you issue the 'adb shell' command.

# adb shell
# <-- adb shell prompt

If you receive the "device not found" error when restarting the adb shell, wait a few moments and try again: then adbd daemon on the phone has not finished restarting.

There, you've done it! You have rooted the device. The root adb shell will remain until you restart the adbd daemon with 'adb kill-server' or by rebooting the phone. The exploit will remain on the phone until you delete it in the adb shell with:

# rm /data/local/tmp/ratc.bin

You are now ready to start copying partitions and/or logical files for your examination... almost. You still lack the tools to copy files, (though not partitions), and copying whole partitions with dd, while possible, isn't as straight forward as it is with physical disks, the primary issue being the yaffs filesystem they contain. Since this post has gotten quite long, I'll discuss those issues if future posts.

Decoding Google Chrome timestamps in SQLite databases

2011-06-02T22:58:00.000-07:00

I had occasion to work with Google Chrome histories today. The timestamp in the History SQLite databases look like unixepoch time at first glance, but they are not. However, the Chrome timestamps are 7 digits longer than unixepoch time.

Unixepoch

Unixepoch time is the number of seconds elapsed since 01/01/1970 00:00:00. It can be converted in a SQLite statement with the datetime() function thusly:

sqlite> SELECT datetime(time, 'unixepoch', 'localtime') AS time FROM table;

"time" is the name of the field containing the (currently) 10-digit date string, e.g., "1307078210", and "table" is the name of the table containing the time field. The "AS time" displays the column label as "time". Without it, the column takes the full datetime() function as the column name which is a bit ugly and confusing to recipients of the data.

PRTime

The extra seven digits in the Chrome timestamp brought to mind Firefox PRTime. PRTime is the number of microseconds since 01/01/1970 00:00:00. The timestamp needs to be converted to unixepoch format before it can be converted to local time through the SQLite datetime() function. This can be done by dividing the value by one-million. In SQLite, this looks like:

sqlite> SELECT datetime(time/1000000, 'unixepoch', 'localtime') AS time FROM table;

Chrome Time (Webkit Format)

If there is an official name for Chrome time, I don't know it. EDIT: I have learned the name for the Chrome's format is Webkit format). What I do know, is that Chrome time IS NOT the same as PRTime. Though it is also a microsecond calculation, its base time is 01/01/1601 00:00:00. To calculate local time, Chrome time has to be converted to seconds by dividing by one-million, and then the seconds differential between 01/01/1601 00:00:00 and 01/01/1970 00:00:00 must be subtracted. But how do we figure that out?

Two ways, as it turns out. With SQLite itself, or with the Unix date command. This is how:

SQLITE:

sqlite> SELECT strftime('%s', '1601-01-01 00:00:00');
-11644473600

DATE:

$ date +%s -d 'Jan 1 00:00:00 UTC 1601'
-11644473600

In both commands above, the "%s" represents unixepoch time. The commands calculate the number of seconds between unixepoch time (1970) and the subsequent date (Chrome time base, 1601). Note that the seconds are negative. Of course, this is because you have to count backwards from 1970 to 1601! With this information, we can convert Chrome time in SQLite like this:

sqlite> SELECT datetime((time/1000000)-11644473600, 'unixepoch', 'localtime') AS time FROM table;

Unfortunately, there are more time formats out there than these. You may recall I blogged about Apple's Mac Absolute Time in February. But, armed with this information, You should be able to convert the times formats I've discussed and possibly discover the solution to any others you might encounter.

Open Source iPhone Exploits

2011-05-19T23:28:00.000-07:00

There are numerous and costly--and closed source--methods for recovery data from the Apple iPhone, iPod Touch, and iPad. If you are lucky enough to be part of Law Enforcement, then you have free access to Johnathan Zdziarski's tools and methodology (Thank you, Johnathan). But what to do if you don't have the proprietary tools, or they just don't work on the version of iOS you encounter?

Allow me to introduce the libimobiledevice project. It is a library that communicates to the aforementioned Apple devices, with the addition of Apple TV devices, but does not use any proprietary libraries, nor does it require jailbreaking to accomplish its goals. I'll let the description on the project home page take over from here: libimobiledevie "allows other software to easily access the device's filesystem, retrieve information about the device and it's internals, backup/restore the device, manage SpringBoard® icons, manage installed applications, retrieve addressbook/calendars/notes and bookmarks and (using libgpod) synchronize music and video to the device."

That said, it is not designed as a forensics tool. I may be characterizing it wrongly, but in short, it brings iTunes-like capabilities to Linux. It does not give full filesystem access to the device, just the media folder and sub folders. This means you can access the Photos and videos taken with the device, as well as access the music catalog through your file browser. But be careful--you have read/write access.

Best practice requires that you disable automounting in your system to avoid changing data on the device when you connect it to your system. This can be done in a Gnome desktop by using gconf-editor to disable automounting in nautilus ( apps | nautilus | preferences | media_automount [uncheck] ). It's a good idea to disable media_automount_open and enable media_autorun_never while you're at it. You won't find your iPhone with fdisk when you connect it, which might leave you wondering just how in the world you're going to mount it?

Libimobiledevice is incorporated into Ubuntu and Linux Mint Debian Edition (my preference) and uses the Gnome Virtual File System (GVFS) for mounting the Media folder of devices in the .gvfs/ folder of your home directory. But you've disabled automounting, so you need another method that allows read-only mounting. That's handled with ifuse, but you'll likely have to install it. With ifuse, the device can be mount read-only in a location you specifiy. The device is automatically identified unless you choose to identify the device by its UUID (only necessary if you have more than one device connected at a time). You mount with "ifuse [options] location", such as "iphone -o ro /mnt/analysis". If you happen to have a jailbroken device, then adding the '--root' parameter will give you full logical access to the user partition on the device.

Mounting with ifuse, on a device that has not been jailbroken, gives access to the same to the device as libimobiledevice through GVFS, only with write protection. Media files can be useful, as I indicated in an earlier post, but how does an investigator get to the meaty stuff, like SMS messages, call history, and the like?

Well, there are tools that have been built to use the imobiledevice library, aptly called imobiledevice-utils in the Debian package management system. The utilities consist of the following tools: idevicebackup, ideviceimagemounter, idevicescreenshot, ideviceenterrecovery, ideviceinfo, idevicesyslog, idevice_id, and idevicepair. Of particular interest are ideviceinfo, and idevicebackup.

The ideviceinfo tool displays data about the phone itself, including the device name, serial number, OS version, and phone number. Even locked phones provide useful data, though not as robust as an unlocked phone. The idevicebackup tool works like iTunes backup. It creates a device backup in a location specified by the user. idevicebackup2 is required for iOS 4.3. and above, but can only be obtained by building it from source, which is available through a link on the imobiledevice website. The backup includes photographs, videos, and databases from the device, including SMS messages, call history, address book, etc!

The files in an iPhone backup are renamed in hex and are referenced in two mbdb databases that index the original name, location and MAC times for the backup files. The mbdb_parse.py utility can be used to identify the files. I am working to modify mbdb_parse.py to automate renaming of the files, but the standard output can be used like an ls -lh output. The limitation of mbdb_parse.py is that it displays user/group values in hex and MAC times in unixepoch time. I have been able to modify the program to allow me to rename the files into their original names, but the program is not ready for release as yet.

I will cover forensic processing of the backup in a later post.

iPhone Forensics Tools

2011-05-04T23:06:00.000-07:00

I've written a couple of articles about my experience with iPhone data ("iPhone Sings like a Jailbird", "Recovering Data from Deleted SQL records", and "Parsing the iPhone SMS Database"), and recently I have been helping others with some iPhone data recovery. Those experiences have led me to create a few simple iPhone forensics tools. I'll list them in name order and briefly describe them:

iphone_ab
iphone_ab is a tool to parse the iPhone address book, which is stored in a SQLite file called AddressBook.db. The tool links two tables to produce a simple output containing first and last name, phone number/email address, record creation date and record modification date. There is much more data that can be mined from the database, but this is the basic data that interest most investigators.
iphone_ch
iphone_ch is a tool to parse the iPhone call history, which is stored in a SQLite file called call_history.db. The tool reads the 'call' table and reports the date, call type, phone number, and call duration of each record. Unixepoch time is converted to local time and call flags are interpreted (Incoming, Outgoing, etc.).
iphone_cs
iphone_cs is a tool to parse the iPhone consolidated.db, which is a SQLite file that stores gps data used by apps. Experimentation shows that one table in particular, the CellLocationLocal table, records the location of the iPhone handset when it runs apps that use location data. Don't think location data is restricted to mapping apps--I have seen dictionary apps that ask to use your location. Now, why is that? In a word: advertising. iphone_cs will parse the CellLocation, CellLocationLocal, and WifiLocation tables for GPS data and allow the data to be formatted for mapping tools like gpsbabel or websites like GPSVisualizer.com.
iphone_images
iphone_images is a tool that will search a path for images and videos (identified by mime type) and provide the EXIF data by use of the excellent exiftool. Alternatively, files containing GPS data can be parsed to export data suitable for mapping. Finally, videos purchased through the Apple Store can be sifted for the Apple Store user name and real name of the purchaser.
iphone_music
iphone_music is a tool that will search a path for audio files, particularly those in the 'iTunes Control/' folder (though not restricted to these files). iTunes uses random filenames for music it transfers to an iPhone or iPod. The tool, at its most basic level, reveals the song name, album, and artist to help owners identify their device by its content (think recovered stolen device). The tool can also produce rich metadata, again, thanks to exiftool, as well as single out those songs purchased through the Apple Store and report Apple Store credentials like iphone_images.
iphone_sms
iphone_sms is a tool to parse the iPhone SMS database, located in a file called sms.db. The tool reads the message table and reports the date, message type (sent, received, etc.), phone number, and text message in each record. Unixepoch time is converted to local time and message flags are interpreted (Sent SMS/MMS, Received MMS , etc.).

All of the tools, written for BASH, attempt to follow the Unix principal of make one tool to do one thing and do it well. They each have a variety of options that can be read by invoking help, e.g., 'iphone_sms -h'. The database tools do not find the target database for you, since your iPhone data may come from a variety of sources (iPhone images, backups, etc.). Output for all files is to stdout (the screen) but may be redirected to a file. For example, to redirect the mapping data from iphone images and videos, the command 'iphone_images -m /private/var/mobile/Media > images_gps.txt' could be used.

I hope you have success with the tools. Please contact me with any errors you detect and/or feature suggestions you might have. I am already contemplating an "export" function to, for example, export the images to a directory that were the source of the GPS data the mapping output from iphone_images.

You can download iphone_tools here.

iPhone songs sing like a jail bird!

2011-04-28T21:51:00.000-07:00

I received an iPhone yesterday that was suspected to be stolen. Proving the suspicion was another matter. The was found to have data (photos, mostly) only related to the suspected thief, but the investigator noted that the phones statistics had been reset and suspected the original owners data was deleted. He brought me the phone for examination to prove his hunch. The phone was not reported stolen anywhere that we could find.

My first thought was the SIM card. Problem was, the phone had no service, and the SIM card only revealed the assigned phone number and carrier. While that might lead to the owner, it would be a while after the drafting and execution of a search warrant. In fact, it could have been weeks before that got us anywhere.

I obtained some basic phone data with the libimobiledevice-utils package. I quickly learned that the iPhone 3G was running the 4.3.1 iOS. I also quickly discovered that the libimobiledevice package (v1.04) in Debian Testing was not going to help me too much because of a bug. Luckily, it was a known bug, and compiling the latestest stable package (v1.06 at this writing) from libimobiledevice.org fixed me right up.

The iPhone tools I use are not yet equipped to obtain the user partition of iOS 4.3 or higher. They cannot image unallocated space in iOS 4 in any event. I have successfully used idevicebackup (one of the libimobiledevice-utils) to backup and iphone and exam the sms, contacts, and other databases, but I could not successfully complete this task because the iPhone refused the backup request (I later did this in Windows with iTunes, however).

So, there I was without full user partition access, no means to recover deleted files, and that was suspected, and no backup of application databases that might lead to owner identification. I could see photos, and music, and nothing more when I mounted the phone with ifuse. The photos had already been ruled out as a possible source of original owner information. That left the music.

I knew from previous research that songs and videos purchased through iTunes contain the real name and iTunes username of the purchaser. The username is most often the user's email address, in my experience. The data is stored in the 'name' and 'apID' atoms of the mpeg-4 format. You can learn more about atoms and the format specifications at the AtomicParsley website.

I used the find command to locate the .m4p (music) and .m4v (video) files that are indicative of media purchased through iTunes. Of the more that 3k of music files, I found about 50 that were .m4p format and contained three different but related usernames (same last name). Not surprisingly, the names did not match the suspected thief. Each username had an associated email address, but a I was able to cross reference the names with other records and find a burglary victim.

So, while we are used to the performing artists singing their songs, forensic computer examiners need to keep in mind that iTunes songs sing themselves, providing purchaser real name and username/email address!

More NSRL Talk

2011-03-11T15:48:00.000-08:00

I've spent a bit more time study the NSRL problem. It is a fantastic but unwieldy database of hash values that is under utilized (at least in my experience). I spoke in my last post about importing the NSRL database into SQLite so the data can be better managed. I added at the end of the post that I had created a pair of shell scripts, 'getnsrl' and 'mknsrl' to automate SQLite creation.

I am happy to report that I have extended mknsrl to fully handle the NSRL database. My first version only imported the file table which contains the hash values. I thought, incorrectly, that this would be sufficient for generating has sets. I had not fully understood the NSRL Reference Data Set (RDS) construction, which actually consists of five CSV files:

NSRLFile.txt
NSRLMfg.txt
NSRLOS.txt
NSRLProd.txt
version.txt

The NSRLFile.txt is broken into four parts that need to be concatenated together. Currently, this database contains over 62 million lines, far exceeding the capabilities of the spreadsheet programs for which it is formatted. The database consists of 'known' files, not be confused with 'known good' files. The database contains hash values for well know hacker programs, for example, and eliminating them through hashing may be problematic to the investigation. There needed to be a way to export specific hash sets from the RDS for the data to be truly useful.

getnsrl

I wrote getnsrl to make processing the four iso files containing the NSRL RDS easy. The files must be mounted, and the database files extracted from a zip file. getnsrl does this automatically, appending the names of the NSRLFile.txt CSV files to distinguish them and prevent overwriting. On my system, this takes about four minutes. The user need only put the iso files in a common directory (not required, but easier on the fingers) and issue 'getnsrl RDS*'. The tool provides feedback and start/stop times.

mknsrl

I wrote mknsrl to allow for easy importation of the CSV files to a standalone SQLite database. It creates virtual tables from the CSV files and then imports them into regular tables. The NSRLFile.txt files are not (and should not be) concatenated before processing as this is done by mknsrl while inserting the data into the database. Though importing the files to a standard table is not required to use the CSV files with SQLite, I favored this approach because the database can be used independently of the CSV files. I just found it simpler to manage, and it has the added benefit of indexing if the user so chooses, since indexing virtual tables is not possible with SQLite. mknsrl reports its progress, start/stop times, and some database statistics. It no longer does any indexing (as in the first version). I leave that to the user to decide what to index if indexing is desired. On my system, it took about 12 minutes to import the NSRL RDS data. The statistical analysis can be commented out to save a few minutes, if desired.

Why use mknsrl?

The chief advantage of mknsrl is database control. Hash sets can be produced of any product in the database, which is not easy to do with the traditional methods for handling the RDS data. NSRL has recommended that the four NSRLFile.txt be concatenated and then reduced to uniq hash values, leaving about 18 million records. The problem is, you can't reliably know the source of the file this way, and you may hash out a program file of interest.

mknsrl allows any hash set or group of hash sets to be extracted from the database with a sqlite query in the following form:

$ sqlite3 -csv -header nsrl.db "select * from file where [expression];"

I used the following expression to create a hash set of all "Windows" operating systems in about 5 minutes (note the nested SELECT statement that makes this possible):

$ sqlite3 -csv -header nsrl.db "select * from file where opsystemcode in (select opsystemcode from os where opsystemname like '%Windows%'"

Your ability to create hash sets is limited only by your ability to craft SQLite queries.

If you would like to look at or use 'getnsrl' or 'mknsrl', you can find them at https://sites.google.com/site/slosleuth/.

Taming the NSRL Hash DB Beast

2011-03-08T13:29:00.000-08:00

Background

I've been trying to extend Sleuthkit in a variety of ways, such as by adding a table to the Sleuthit databese to capture file/mime types and improve strings extraction over that of the traditional 'blkls | strings > file' variety. I came to the realization that elimination of files through hashing could speed these processes. This is not a new concept to me or anyone else in data forensics, but I do believe hash elimination is under used. I think this is due in large part to the unwieldy size of the National Software Reference Library (NSRL) hash database.

The Problem

As of January 2011, The NSRL data is distributed in 4 .iso files and contains a whopping 62,294,036 file entries. The NSRL tries to help you manage the data volume by directing you to concatenate the NRSLFile.txt files (each containing over 4.7 million rows) found in each .iso and then export only rows with unique hash values. Such an action reduces the total number of rows to a paltry 18,840,521. However, the unique operation (easily performed with 'uniq' on the command line) hampers the ability of knowing the application or operating system from which the file came.

Size is not the only difficulty, however. After all, in most cases, large size just means longer processing time. But the CSV format of the database is a major obstacle. The field separator used in the database is the comma (','). But, many of the filenames listed in the db contain commas, and while, text fields are quoted, numeric fields are not. These conditions are not an obstacle for spread sheets, but keep in mind that OpenOffice Calc can handle only 1,000,000 rows. And, these conditions make it difficult to work with traditional command line text tools like 'cut', 'sed', and 'awk'.

Ideally, it would be best to have the NSRL db in a database like SQLite, where hash sets could be created based on products and operating systems. In this way, one could easily export a hash set for Windows XP if that was the type of system being investigated. However, SQLite splits on all field separators it encounters regardless of what comes before them (meaning they cannot be escaped). Therefore, trying to import the NSRLFile.txt data fails because there are filenames with commas in them.

The Solution

I am happy to report that there is a solution to the problem with SQLite, however. A CSV module has been written to overcome this difficulty. It allows SQLite to query a CSV file directly Using the module is not immediately apparent, so I document the process here:

Decompress the csvfile.zip containing the module source code.
Change to the csvfile/ directory and issue the 'make' command. If you get an error, it is probably because you don't have the SQLite headers installed (libsqlite3-dev package in Debian/Ubuntu).
Copy the resulting virtcsv.so module to somewhere common, like /usr/local/lib because you will need to load the module each time you query the database you create.
Create a new database with 'sqlite3 nrsl.db'.
Load the CSV module at the SQLite prompt with '.load /usr/local/lib/virtcsv.so'
Create a virtual table to connect to the NSRLFile.txt at the SQLite prompt with 'CREATE VIRTUAL TABLE hashes USING CSVFILE(NSRLFile.txt, CP1252, ',', '"');' where:

hashes is the table name you are creating
NSRLFile.txt is the CSV file you are attaching
CP1252 is the code page Windows Latin 1 (default)
, (comma) is the field separator in the CSV file (default is ';' [semicolon])
" (double quote) is the string delimiter (default)

View a sample of the data with 'select * from hashes limit 10;'

The process above shows the process for importing one file. The fact of the matter is, that if you concatenate the four NSRLFile.txt files provided by the NSRL, the file is too large for SQLite to handle. The files need to be imported to individual tables and queried together to produce hash sets. Further, Indexes should be created to speed queries (Edit: SQLite will not allow virtual tables to be indexed, so the data must be imported to regular table. I have written a script to automate this, see below.). I won't go into those process here, but a good online tutorial for using SQL can be found at W3Schools. Keep in mind that SQL (shown at W3Schools) and SQLite have some differences, but there are few inconsistencies for basic database query.

Edit: I've written two scripts to automate the process. The first, 'getnsrl', will mount the four RDS .iso files and extract/rename the NSRLFile.txt csv files to ready them for processing. The second, 'mknsrl', will create virtual tables from each csv file and import them into a 'hashes' table. It will also index the table based on the operating system the files come from to speed searches and hash table exports.

"Tracking" Down Credit Card Dumps

2011-02-21T16:48:00.000-08:00

I have been examining numerous computers and media cards to support fraud investigations in the past year. One group of thieves has been installing skimmers (devices that record magnetic stripe data) in the magnetic stripe readers on doors of certain banking institutions that limit access to their ATMs after hours. Then the thieves install micro video cameras in the light fixtures of the ATMs to capture the users' pin codes. The thieves later match the recorded pin code to the skimmed account number at the door, encode new cards, and pillage bank accounts by getting cash at another ATM location.

It is not difficult to find the ATM videos on the media cards from the cameras or on the computer hard drives. The credit card/ATM card numbers can be bit tougher, however. Searching for 16 digit numbers in a Windows computer will yield a huge volume of false hits. And, if the operation has been running for a while, many of the account numbers will be deleted. Searching unallocated space is necessary to be thorough and to ensure your suspects' get the full reward of their illicit activities.

All of the skimmers I have encountered are serial devices that communicate with software made for Windows-based computers by means of a USB-to-serial cable. The cables do not look dramatically different than a standard USB cable, but they contain a chip that bridges the two protocols. In a Windows computer, a USB driver for the cable must be installed before the program can communicate with the device.

The applications that communicate with the skimmer devices write the data to disk differently--some create a database while others use text files. In my experience, both formats contain the complete track 1 and/or track 2 data. Magnetic stripes contain three tracks, the first two in common use and encoded with particular standards.

Track 1 (IATA)

Track 1, known as "International Air Transport Association" format, has alphanumeric data. It takes on the following format (from wikipedia):

Track 1, Format B:

Start sentinel — one character (generally '%')
Format code="B" — one character (alpha only)
Primary account number (PAN) — up to 19 characters. Usually, but not always, matches the credit card number printed on the front of the card.
Field Separator — one character (generally '^')
Name — two to 26 characters
Field Separator — one character (generally '^')
Expiration date — four characters in the form YYMM.
Service code — three characters
Discretionary data — may include Pin Verification Key Indicator (PVKI, 1 character), PIN Verification Value (PVV, 4 characters), Card Verification Value or Card Verification Code (CVV or CVK, 3 characters)
End sentinel — one character (generally '?')
Longitudinal redundancy check (LRC) — it is one character and a validity character calculated from other data on the track. Most reader devices do not return this value when the card is swiped to the presentation layer, and use it only to verify the input internally to the reader.

Targeting track 1 data in a grep search is very simple:

$ grep -E ^'%.[0-9]{16,19}\^' file

Track 2 (ABA)

Track 2, known as the "American Banking Association" format, contains only numeric data. It takes on the following format (from wikipedia):

Track 2: This format was developed by the banking industry (ABA). This track is written with a 5-bit scheme (4 data bits + 1 parity), which allows for sixteen possible characters, which are the numbers 0-9, plus the six characters : ; < = > ? . The selection of six punctuation symbols may seem odd, but in fact the sixteen codes simply map to the ASCII range 0x30 through 0x3f, which defines ten digit characters plus those six symbols. The data format is as follows:

Start sentinel — one character (generally ';')
Primary account number (PAN) — up to 19 characters. Usually, but not always, matches the credit card number printed on the front of the card.
Separator — one char (generally '=')
Expiration date — four characters in the form YYMM.
Service code — three digits. The first digit specifies the interchange rules, the second specifies authorisation processing and the third specifies the range of services
Discretionary data — as in track one
End sentinel — one character (generally '?')
Longitudinal redundancy check (LRC) — it is one character and a validity character calculated from other data on the track. Most reader devices do not return this value when the card is swiped to the presentation layer, and use it only to verify the input internally to the reader.

Targeting track 2 data in a grep search is also straight forward, though a reading of the above standard suggests the following search can miss. I haven't seen any different start sentinels or separators than those in the expression:

$ grep -E ^';[0-9]{16,19}=' file

Though the preceding search has worked for me, the specifications for track 2 allow for other start sentinels and field separators. A better but slower search would be:

$ grep -e ^'[:;<=>?][0-9]{16,19}[:;<=>?]'

Tracking Down Both Tracks at Once

I composed the following search to track down both tracks, and display the date and time of the swipe typically captured by the skimmer device. The "-B1" argument shows one line of data before the hit, which is where the date and time are recorded.

$ grep -E -B1 ^'%.[0-9]{16,19}\^|;[0-9]{16,19}=' file

You may have noticed the caret ("^") preceding the single quoted search in each of the grep expressions. It requires the expression to be matched at the beginning of a line of data. This is a search optimization. I will now demonstrate two additional optimizations, one for searching allocated files, and the other for unallocated space (using the the sleuthkit tool "blkls"):

Searching mounted file systems (find files, export 7-bit strings, grep for track data):
$ find . -type f | while read i; do strings "$i" | grep -E -B1 -H --label="$i" ^'%.[0-9]{16,19}\^|;[0-9]{16,19}='; done

Searching unallocated space of a forensic image (export unallocated, filter out 7-bit strings showing offset, grep for track data)
$ blkls -o63 image.dd | strings -td | grep -E -B1 ^'%.[0-9]{16,19}\^|;[0-9]{16,19}='

Difficult Disk Imaging

2011-02-21T12:08:00.000-08:00

In the past few weeks, I've had the opportunity to make forensic disk images of what one might call "non-standard" devices. The devices were a Lenovo Thinkstation D20, an Acer Netbook, and an MacBook Air.

Lenovo Thinkstation

The Lenovo presented a few problems. First, it was seized and disassembled by non-computer-forensics professionals. Translated: the drives were removed and not marked as to their bays or cabling. Two drives were identical in size and the third was over three times larger. All were were SAS (Serial Attached SCSI) drives which have non-standard connectors. I had no connectors to remove the drives and image them individually (though cannibalization from the Lenovo was possible), and the computer specs suggested that there was a raid array on the two drives of matching size.

What did I do? I decided to use CAINE, a forensic boot disc, and an external hard drive. CAINE would allow me to use the Lenovo for the specialized connectors needed for the SAS drives, and allow the hardware controller on the motherboard to reassemble RAID array.

The first step was to ensure I could boot the system with CAINE. I was unable to boot from CD-ROM using the Lenovo's optical drive (which was unusual, to be sure) but I was able to get a USB version of CAINE booted. I ensured, by adjusting the BIOS, that the USB would be the first device to boot.

I reinstalled and connect the drives, uncertain as to proper order, and booted CAINE. Lucky for me, the on-board RAID controller detected the disks, reported there have been a change in the devices (the drive order), and then correctly reassembled the array. CAINE reported two drives (the large disk, with the OS as it turns out), and the array. I imaged both to the external drive with Guymager, a graphical front end for libewf, an open sourced disk imaging library and toolset that produces images in expert witness format.

Acer Netbook

The Acer Netbook was probably the least troublesome device, but did not lend itself well to disassembly. Drive removal and hardware write-blocking are the ideals in forensic disk imaging. However, this isn't always possible or convenient. In the case of the Acer Netbook D255, there was no simple hard disk cover to remove. Hard disk access appears to involve keyboard removal and an underlying cover, or seven case screws and an almost surgical separation of plastic catches. Simply put, I didn't want to break the netbook, and I know that some storage devices have ROM chips that prevent them from being read when disconnected from the particular motherboard anyway.

Again, CAINE to the rescue. In the case of the ACER, there was no boot menu. Changes to the BIOS were needed to ensure the USB device booted before the internal hard disk. I tried to boot CAINE with an attached USB optical drive and with a USB version of CAINE. The ACER did not register the USB optical drive in the BIOS, but the USB flash drive with CAINE was detected. I booted from the USB, mounted a external hard drive, and imaged the drive with libewf.

MacBook Air

This was my first encounter with the MacBook Air. Like the Acer, the construction of the device discouraged disassembly. I know that the Macs won't boot from a FAT formatted USB because of the EFI boot schema. However, booting from CD-ROM is possible by pressing and holding the "C" key immediately after powering the computer.

I attached a USB CD-ROM drive because the AIR does not have an optical drive like other MacBooks. I initially booted with CAINE, but the graphics drivers were incompatible with the Mac. I attempted a graphics safe-mode boot and a text-only boot, but the same result: a garbled display that made proceeding impossible.

I obtained a second forensic boot disk called DEFT. It is a newer release than CAINE and I hoped it had updated graphics drivers that might overcome the problem. The initial boot froze the system. DEFT boots into text mode, and there are no other menu choices. However, a series boot options at the bottom of the boot screen reminded me of some boot issues I have experienced in that past several versions of Ubuntu, on which both these forensics distributions are based. I passed the "nomodeset" option in the F6 menu (curiously named "Password"), and DEFT booted to a text screen. I was also able to but to a GUI with the deft-gui command.

With this in mind, I revisited CAINE. I have a preference for CAINE because I understand how it works and it's implementation of write-blocking and have tested it. The CAINE developer, Nanni Bassetti, is ever-ready to help new users and explain his techniques. I do not know how DEFT works and the information is not readily available, at least not in English. This is not to disparage DEFT in anyway. I'm just trying to highlight the fact that we must use tools that we understand and have tested.

I again booted the MacBook Air with Caine. At the boot screen, there is no obvious way to pass boot options. However, pressing escape brings up a boot command line. Pressing tab displays the boot options on the original boot screen. I passed the arguments "textonly nomodeset" and CAINE successfully booted to a console. At the console, I was able to start the GUI with "startx". I accomplished imaging as before, with libewf and an external USB hard disk drive.

Recovering Data from Deleted SQL records

2011-02-21T10:29:00.000-08:00

I previously posted about parsing iPhone SMS database. The particular focus was the recovery of deleted messages. I explained there are really two types of deleted messages in play here: records flagged as deleted within the database (thus not really deleted at all) and records deleted from the database itself. I discuss the second type of deleted data recovery here.

A SMS message deleted by the iPhone user is flagged in the database as "deleted." What happens next is not clear to me because I don't currently have an iPhone with which to experiment and I am not a sqlite expert. I am uncertain if the database immediately deletes the record or if that occurs on sync (there are a couple of database "triggers" I don't yet fully understand). If the data is only flagged deleted, then the record can be read with sqlite tools, which is what I discussing in my previous post.

But at some point a record can be deleted from the database, and as a result, it is not viewable with sqlite tools. So how do we find that data, and more importantly, how do we distinguish it from the non-deleted data? It helps, at this point, to understand what happens to deleted records in sqlite. When you delete a record, the space allocated to the record gets added to a free-list. In other words, the size of the database doesn't get any smaller with record removal, but the space is marked as available for future records. This remains true until the database is "vacuumed."

A database can have it's free space removed with the conveniently named "vacuum" command. This rebuilds the entire database, removing the space in the free-list and shrinking the database. Sqlite can be compiled to do this automatically, but fortunately for us, this is not currently the case for the sqlite compilation in iOS. We can use the vacuum command to help differentiate the data from the deleted records and non-deleted records, however.

The method I used was simple and would apply to any sqlite database, not just the iPhone sms.db.

Make a copy of the sms.db: 'cp sms.db sms.vac.db'
Vacuum the database with: 'sqlite3 sms.vac.db vacuum'
Examine the difference between the vacuumed file and the original file: 'diff sms.db sms.vac.db'

There are obvious shortcomings with such a method. The foremost problem is that the data is unstructured, and this causes interpretation difficulties. However, there is no other method of which I know that will produce structured data. And unstructured data can still be useful in an investigation, if only to verify a statement or corroborate another piece of data.

I am aware of one attempt at forensic recovery of deleted sqlite records. It is specific to the Firefox browser history. For more information, take a look here.

Calculating Embedded OS X Times

2011-02-21T09:27:00.000-08:00

I recently examined a Macintosh computer where I needed to look at Internet History. The only installed browser was Safari, and the history was stored in /Users//Library/Safari/History.plist, an XML file with visit dates in recorded in epoch format. An example of that time is "314335349.7".

The tricky thing is realizing that not all so called "epoch" time is the same. In a 'nix system, epoch time is defined as the number of seconds since 01/01/1970 00:00:00. However, the Mac epoch time is defined as the number of seconds since 01/01/2001 00:00:00. EDIT: Mac time is also known as "Mac Absolute Time."

Unix epoch time is a simple conversion in Linux. A current time is a ten digit number that resembles "1298307237". To convert that to human-readable date, simply:

$ date -d @1298307237
Mon Feb 21 08:53:57 PST 2011

The date command defaults to calculating from 1970 and not 2001 as we need for our Mac time conversion. To obtain a proper conversion, we need to tell the date command the starting point of the data calculation thusly:

$ date -d "2001-01-01 314335349.7 sec PST"
Sat Dec 18 03:22:29 PST 2010

Knowing the format, scripting the conversion should be relatively trivial. I hope this helps someone. I'll know I'll be back to this page often to remind myself of the conversion syntax!

EDIT: When processing Mac databases, like those found on the iPhone, it is possible to convert the times using SQLite commands. I determined the number of seconds since unixepoch time to Mac Absolute Time with "SELECT strftime('%s', '2001-01-01 00:00:00');" as 978307200 seconds. This value can be added to Mac Absolute Time and then converted to local time with the SQLite datetime() function thusly: datetime(time_field + 978307200, 'unixepoch', 'localtime').

Parsing the iPhone SMS Database

2011-02-02T17:27:00.000-08:00

I was asked recently to help recover deleted messages from an iPhone SMS database. Conveniently, this is called "sms.db" on the iPhone and it is located in the /mobile/Library/SMS/ directory. It is a sqlite3 file type, and there are several GUI tools to read sqlite3 databases, as well as a sqlite3 command line shell.

It seems like a pretty straight forward exercise at first blush. However, "How do I recover deleted messages from an iPhone database" is not as simple as question as it first seems. When a user chooses to delete a message on the iPhone, the record is flagged as deleted in the database, but the record is not deleted from the database. This means that records flagged as deleted are still recoverable with sqlite3 tools.

But when the phone is synced with iTunes, the records flagged "deleted" are actually removed (deleted) from the database itself and no longer recoverable with sqlite3 tools. However, the data is still within the database, but the space it occupies is added to a "free list" for use by new data. In other words, the data can still be recovered before it is overwritten by new data or the database is "vacuumed", a sqlite3 process that rebuilds the database removing all the free space and reducing the size of the database. As a caveat, sqlite3 can be configured to overwrite records immediately upon deletion, but this is not the case for the iPhone at present.

So really, there are two types of deleted data to be sought: records flagged as deleted (not really deleted records at all), and records deleted from the database. I'll discuss the first type in the remainder of this discussion. I'll consider ways to recover deleted records in another post.

The sms.db has the following tables:
_SqliteDatabaseProperties, msg_group, group_member, msg_pieces, message

The message table contains the text messages of interest. The contents of the table can be displayed from the shell with:

$ sqlite3 -header sms.db "select * from message"
ROWID|address|date|text|flags|replace|svc_center|group_id|association_id|height|UIFlags|version|subject|country|headers|recipients|read
1|+17132619725|1281033415|Hey, what's up?|2|0||1|0|0|4|0||us|||1
...

As you can see the interesting fields (revealed because the -header argument was used) are "ROWID," "address" (which is the source phone number), "date" (in unix epoch format), "text," and of less obvious value--"flags." Flags indicates the type of message, i.e.,:

2 = received
3 = sent
33 = Message send failure
129 = deleted

Adam Crosby

With a little sqlite razzle dazzle, we can get a well formatted output in a more human readable form:

$ sqlite3 -header sms.db "select ROWID as row, case flags when 2 then 'rcvd' when 3 then 'sent' when 33 then 'fail' when 129 then '*del' else 'unkn' end as type, address as phone_no, datetime(date,'unixepoch','localtime') as date, text as message from message"
row|type|phone_no|date|text
1|rcvd|+17132619725|2010-08-05 11:36:55|Hey, what's up?
...

SQL syntax can be a bit tricky, and look a bit intimidating. But by using internal commands, you can get a tremendous speed boost over using external text tools. By way of explanation, I'll break down the command:

sqlite3 -header sms.db #open the database with sqlite and display column names
"select #display the following columns from the table
ROWID as row, #display the ROWID column first, but rename it "row" in the output
case flags when 2 then 'rcvd' when 3 then 'sent' when 33 then 'fail' when 129 then '*del' else 'unkn' end as type, #display the flags column next, but change the "2" flag to "rcvd", "3" to "sent", etc., and rename the row "type"
address as phone_no, #display the address column renamed as "phone_no"
datetime(date,'unixepoch','localtime') as date, #convert the content of the date field to local time and display the column as "date"
text as message #display the text column renamed as "message"
from message" #all the columns of data are read from the message table

The command can be simplified if column renaming isn't required. I included it here to make the output as clear as possible, and since the command can be incorporated into a script, it need only be typed once. The quoted part of the command could be inserted into a GUI sqlite browser if that is your tool of preference. The query can be adjusted to show just deleted messages, for example, by appending "where type like '*del'".

Notice I did not link the data in the message table to any other tables in the database. While this can be done, my task was to seek out deleted messages. And as I said earlier, I will explore methods for recovering deleted records from sqlite databases in a future post.

As always I welcome any comments or suggestions....

TinyCore: A Mighty Platform (Part 2)

2010-12-12T22:35:00.000-08:00

TinyCore Linux is an ideal platform for building a light weight forensics distribution with the purposes I have in mind (See Part 1 of this post). It is only a 10mb download for the base distribution and boots to a simple GUI desktop. It boots and loads entirely into a ram disk as small as 48mb, but allocates as much ram as possible. The ram disk makes TinyCore very fast because the entire operating system resides in ram and there are no drive seek time delays.

TinyCore uses a modern kernel with good hardware support and an xvesa video driver which all but insures a working GUI. Applications are installed as modules (called extensions) that can be run at boot time or on demand. The root file system and the applications are read-only and are renewed on every boot eliminating file corruption that can creep into installed software.

The Basic Structure

At its most rudimentary level, TinyCore consists of two files, the kernel (bzimage) and a compressed file system (tinycore.gz). Add to that a means to boot the operating system, such as isolinux, and your full file tree is a simple:

./boot
./boot/isolinux
./boot/isolinux/isolinux.cfg
./boot/isolinux/isolinux.bin
./boot/isolinux/f4
./boot/isolinux/f2
./boot/isolinux/boot.msg
./boot/isolinux/f3
./boot/isolinux/boot.cat
./boot/bzImage
./boot/tinycore.gz

Making the Read-Only Environment

Attached storage devices detected by the kernel are identified by the udev daemon. Udev applies rules to the devices based on their type. In the case of TinyCore, udev calls the /usr/sbin/rebuildfstab script to build the /etc/fstab file which contains the mounting options for the attached devices. When the device is subsequently mounted (devices are not automatically mounted in TinyCore when attached), the mount options in the fstab file are applied. One need only modify rebuildfstab mounting options to make the system mount devices read-only.

I have been able to modify the rebuildfstab file to mount devices read only and address other forensic mounting issues, like mounting ext3/4 devices as ext2 to avoid any possible journal changes and mounting physical devices as loopback devices to avoid attempted repairs of corrupted file systems on mount.

The process of modifying, adding, or removing files in the core file system is well documented here. It involves decompressing the tinycore.gz file extracted from a TinyCore iso, making the desired changes, and zipping it back up. The new tinycore.gz can then be remastered into a new iso.

Making Application Modules (Extensions)

Though new applications can be remastered into the core file system, I favor the modular approach implemented by the TinyCore developers. Applications are compiled and the stored in a read-only squashfs file system. The application, when installed, is mounted into the core file system. Applications can be triggered to mount at boot time, or on demand. On demand ensures quicker boot times and frees more space in the ram disk if the application is not needed in the session. Though there is not gui method for this, installed applications can be "uninstalled" in the middle of the session by simply unmounting them and thus freeing ram allocated to them.

Though TinyCore has some suitable modules for forensics, like foremost for example, it lacks libraries and application such as libewf (Expert Witness imaging format), afflib (Advanced Forensics imaging Format), and sleuthkit (disk investigation tool) that a forensics practitioner would desire. If you are familiar with building application from source, however, then building TinyCore application modules is a snap. I have already built libewf, afflib, aimage, and sleuthkit modules and will submit them to the repository once I complete testing. You can take a look at the building method here.

Persistence

Everything I've mentioned about TinyCore so far mentions "read-only." The rebuildfstab script can be modified to ensure devices are mounted read-only, a must for live forensic examinations. The core file system and application modules are mounted read-only ensuring a "clean" operating system and software environment with each boot. But how does a user save evidence from examinations?

TinyCore allows the home directory to be saved to a storage device. On shutdown, user data is written to the storage device designated by the user. A boot option allows the device to be specified on the next boot to restore the user data, or it can be loaded after boot.

Putting it All Together

If you read Part 1 of this post, you know that my goal is the creation of a bootable disc/USB that an investigator with average computer skills (not a computer forensics practitioner) could use to search for and seize evidence from digital storage devices. TinyCore, in my estimation, has it all:

Small size, loads entirely into ram, and fast with a simple GUI
Easily modified and remastered as a read-only environment
Easily add and created application modules with minimal ram impact
Means for easy creation and restoration of persistent storage

If one adds to the base a decent file browser, like ROX (I'll explain why I think this file browser is great option for forensic examination another time), a word processor (Abiword) with decent file format support, an audio/video player (VLC), and maybe a few other applications, and you have a light-weight, fast, and safe operating system that an investigator with basic computer skills can use to advance his case.

I have a basic version of this concept in place and am currently testing and refining. I plan to host it on Google Code to get community feedback and to publish the changes I make to the core operating system. I'll also host the modules I build there (at least until they are accepted into the TinyCore repository). As always, I welcome any feedback.

TinyCore: A Mighty Platform (Part 1)

2010-12-11T15:54:00.000-08:00

Last week I rediscovered TinyCore Linux. I had taken a look at it about 6 months ago and was intrigued, but didn't have the time to explore it further. However, I have been seeking a small Linux distribution on which to build a specialized forensics distro, and last week I gave TinyCore another look.

Background

I believe there is a need in computer forensics for an investigator with limited training to be able to search for and seized digital evidence from storage devices. Some of the reason's I believe this are:

There are not enough trained forensic computer examiners to keep pace with the number of cases involving digital evidence.
The backlog created by a lack of examiners means cases don't get filed for month or even years after the discovery of the crime. Meanwhile, the perpetrator is free to commit more crimes.
Prosecutors are less likely to pursue older cases, in part because witness recall becomes unreliable.
The majority of charges filed against perpetrators are settled out of court through plea bargaining.

Therefore, in most circumstances digital storage devices are taken to computer forensics laboratories to search for evidence to support a filing of criminal charges. But the labs are too busy to get to the examinations very quickly, and by the time they do, Prosecutors are reluctant to file charges because of the delayed filing and/or the perpetrators have been committing additional crimes. I know this doesn't describe all situations, but it should ring true with most people in some manner.

Solution

The obvious solution is to increase the number of forensic computer examiners and computer forensics laboratories. However, that isn't going to happen, at least not in the near and not-so-near futures. And, since I'm a "work with what I've got" kind of guy, I've been working on another solution:

Criminal investigators need simple but effective tools to search for and seize evidence from digital storage devices. The tools need to be forensically sound, i.e., they do not alter the original media in any way, but easy enough to use that a basic computer user can feel comfortable and conduct effective examinations.

Think about it this way: If a criminal investigator could retrieve his own digital evidence, he could file charges immediately, and most of the cases filed would be settled without the need of further forensic computer examination. In cases that do not settle because the digital evidence is disputed, the storage devices could be sent to the computer forensics labs for more traditional analysis.

More cases filed, more perpetrators convicted, less workload at the lab!

But how do we create such tools? Forensic boot discs like CAInE are great for experience investigators, and the latest version contains nautilus scripts to make live examinations like I'm contemplating here possible. But the operating system is resource heavy, slow to boot from CD, and still to complicated for basic criminal investigators (for example, it is confusing and difficult for most basic users to mount a storage device read-write to collect evidence because the CAInE mounting policies rightly auto-mount devices read-only). In other words, CAInE and other existing boot discs are not the right tool for users with limited computer forensics training.

TinyCore

I believe the best tool for criminal investigators with basic computer skills will:

Boot quickly (Criminal investigators may be in the field without the luxury of time.
Work in nearly any machine (basic video drivers, e.g., xvesa)
Not alter the media being examined (i.e., mount devices read-only)
Create an writeable storage location automatically (no command line or confusing the evidence device for the storage device)
Contain programs or scripts that are easily accessible to find evidence files (e.g., nautilus-scripts)
Create reports about files saved as evidence containing file metadata (so evidence can be commented upon by trained investigators, if needed)
Allow for the creation of forensic images (in the event the device cannot be seized).

TinyCore linux appears to be an ideal platform from which to build this tool. And, I'll explain why in Part 2...

Sleepwaking

2010-11-16T17:39:00.000-08:00

No, the title of this post is not in error. I don't mean "sleepwalking." I meant what I said: sleepwaking.

Those who know me know I am a proponent of "previewing" computers for content before conducting a full forensic exam. There are many reasons for this including the most common purpose: triage. Previewing entails booting the computer with a forensic boot disc such as CAINE which prevents writes to the media being examined, and examining the file system(s) on the attached storage devices for evidence related to the investigation.

There are, of course, limitations to Previewing approach that I won't discuss here, But there is a danger that I feel must be discussed: Previewing the computer that is in a sleep or hibernated state which is commonly (but not exclusively) found in laptop computers. Dropping a forensic boot disc into the CD drive (or using a USB incarnation) will not prevent the computer's operating system from running and changing data on storage device. So, what is a person to do?

For sake of this discussion, envision a laptop computer with a single internal hard disk drive with an Windows operating system in a hibernated state. Booting the computer after inserting a forensic boot disc will NOT preserve the data on drive, because the BIOS detects the hibernated state and begins restoring the contents of the hyberfil.sys file to RAM and restores the operating system to a running state. If you interrupt this restoration, quickly enough, you will find that the data on the hard disk drive hasn't changed, but metadata (access times) will be changed. Depending on the version of Windows installed, you still may not get a boot selection screen the next time you boot (Windows 7 comes to mind).

The best practice in my experience is as follows:

remove the hard drive from the computer.
insert your forensic boot disc .
boot the computer and access the BIOS or boot selection menu (ensure you can select the boot disc as the boot device)
reinstall the hard disk drive.
reboot the computer and boot into your forensic environment. The BIOS will not now automatically attempt to restore the resident OS from hibernation.

And don't forget that the hibernated system has saved what is essentially a RAM image in the hyberfil.sys. I guess that's what you call a "twofer." It takes a different shape in Macintosh and Linux systems, but RAM contents are available in those hibernated operating systems, too.

Of course, true "sleeping" is different than hibernation, though the line has been blurred since Windows Vista. A true "sleep" state suspends the computer, preserving the running state by providing power to RAM in order to retain data. It is usually indicated by a flashing power light and the data disappears scant seconds after loss of power. On the other hand, Windows Vista introduced "Fast Sleep" that also saves the RAM content to hard disk in conjunction with providing power to the RAM. The investigator will have to make his/her own decisions based on the details of the investigation on how to handle a sleeping computer (e.g, wake it and save volatile memory before shutting down the system or abandoning the RAM contents by pulling the drive).

Garmin GPS: What you don't know can track you!

2010-11-03T15:17:00.000-07:00

Garmin GPS devices track their position by default (caveat: at least every device I have been given to examine!). They will do so, approximately every 30 seconds, when powered on. Notice I said nothing about navigating. Simply powering the devices causes them to start logging their location. While this feature can be disabled, it is buried in the settings and I suspect that most users are not even aware of it.

The data is stored in a GPX file, also know as the Global Positioning Satellite (GPS) Exchange Format. The most current track, appropriately named "Current.gpx," is stored in the "/Garmin/GPX" directory on the device. Older tracks are stored in "/Garmin/GPX/Archive" directory. The archives take on the name ".gpx," e.g,. "1.gpx," "2.gpx," etc. I have never seen more that 17 archived files, but I don't know if this is a system limitation or just a coincidence that I have seen it more than once. The history can cover quite a time span: my most recent examination revealed a history of 6 months!

GPX files are in xml. The Current.gpx file can have interesting entries, including the "Home" address of the device owner. I have used this setting to reunite stolen devices with their owners or thieves back to their homes. But the most interesting information is the device track, which consists of a series of GPS waypoints or "trackpoints" recorded by the device. Here is a sample from an archive file:

There are many ways to handle a GPX file, but I have found it is most useful to convert it to a KML, or Key Hole Markup Language, file for use with Google Earth. While I know that Google Earth is not an open sourced application, and other tools like "gpxviewer" can map the GPX file directly, most of the people I support are Windows users that have experience with Google Earth.

There are two methods I am aware of for creating KML files. The first is using an online resource, like GPSVisualizer. Just complete the online form and upload your file to make a map that meets your requirements. Other formats, besides Google Earth, are possible, including Google Maps, JPG, PNG, SVG, and text.

I don't like to rely on websites, however, because Internet connectivity is never assured. Enter GPSBabel. GPSBabel is a command line tool (gui available) to convert over 100 different types of GPS data formats. A basic conversion can be as simple as:

gpsbabel -i gpx -f input.gpx -o kml -F output.kml

There are numerous options, that I won't cover here, to customize your output file. They include labeling the way points with the date and time they were created, allowing you to easily visualize the track. I'd recommend the use of a GUI to familiarize yourself with the customization options, though be aware that the GUIs seldom provide all available options.

I have used Garmin GPX files to map a suspects' travels and place him them at crime scenes. I hope with this information you will be able to do so, too!

Sleuthkit 3.2.0 Released

2010-11-03T13:04:00.000-07:00

A new version of The Sleuth Kit (TSK), a command line forensics tool set for both Linux and Windows, was released a few days ago. This release brings new automation tools that can greatly speed processing. Brian Carrier, the developer, describes the release thusly:

New features include:
   • New tsk_recover tool that extracts files from an image to a local directory.
   • New tsk_loaddb tool that dumps file system metadata to SQLite database.
   • New tsk_getimes tool that collects MAC time data on all file systems (equivalent to fls -m on a series of volumes)
   • New tsk_comparedir tool that compares a directory to an image to detect rootkits.
   • New C++ TskAuto class that makes it easier to create automated tools that analyze all files.
   • Name cleanup out of libraries and into tools.
   • img_cat -e and -s flags.
   • Changed how default NTFS $Data attribute is named.
   • HFS+ Case sensitive flag in fsstat.

Bug fixes include:
   • FAT performance
   • Crash fix for corrupt NTFS file
   • Adding attribute runs on fragmented files with multiple attributes of the same type.

I can personally attest to the benefits of the tsk_loaddb tool. It very quickly creates a sqlite database containing file system metadata for each volume in the forensic image. The database can be queried for any data sought in a fraction of the time it takes to read a file system itself.

A knowledge of basic sqlite commands is essential to take advantage of the database, but with that knowledge, it is quite easy to script tools for your use. For example, I wrote a shell script that reads the database and automatically mounts all the partitions read-only for quick evaluation. This, of course, is only one small way that data can be used.

For a concise overview of all the tools that TSK has to offer, take a look at the wiki located here.

Booting Disk Images in Linux

2010-10-19T16:01:00.000-07:00

A Picture can be Worth a Thousand Words

Sometimes there is no substitute for booting a computer to gather data. But the forensic consequences of directly booting the system under investigation are prohibitive. And, while it can be possible to restore a computer image to another hard drive and install that drive in the computer hard drive for booting, this can be costly, very time consuming, and you only get one shot before you've changed the data from original (and repeated boots may be desirable depending on the investigation).

In response to this, tools like LiveView were created. The concept was to boot the forensic image, rather than restore it, and capture all system changes in a disk cache. The operating system could be booted quickly and repeatedly from an original state. However, I never found LiveView to be very effective, and it the past few years I have avoided Windows-based tools whenever possible .

Enter xmount by Dan Gillen. Xmount will mount raw (dd), Expert Witness Format (ewf, aka, EnCase), and possibly Advanced Forensics Format (aff) images and create a virtual disk that can be run in a virtual machine. Supported virtual disks are raw (qemu), vdi (VirtualBox), and vmdk (VMware). Xmount is easy to use: its as simple as invoking the command, listing the image type, the virtual disk type desired, assigning a disk cache file, and selecting the disk image to mount. For example:

$ sudo xmount --in ewf --out vdi --owcache disk.cache image.E?? /mnt

Of course, you've only come partway to actually booting that image. You could try to boot the virtual disk in your virtual machine, but if it's a Windows operating system, chances are it will fail.

Open the Flood Gates with "opengates"

Windows operating systems rely on the hardware of the system in which they are installed to boot. Thus, you cannot normally remove a hard disk drive with a Windows operating system, install it in a differently configured system, and expect it to boot. Thus, neither will your virtual disk boot in your virtual machine, unless the configuration of the virtual machine closely matches that of the original system. However, xmount creator Dan Gillen has a solution for us: "opengates."

Opengates is a utility to overcome the hardware limitations of Windows Operating systems. It does several other things to make the virtual disk bootable, but one most interesting and useful capability is the removal of user passwords. Opengates is deployed with BartPE, a bootable live windows environment. Gillen provides very clear instructions for creating an opengates enabled BartPE disc with PEbuilder. But in short, you need a Windows XP or 2003 machine (to run PEbuilder), a licensed Windows XP install disc, the opengates software, and about 10 minutes.

Configuring the VM

Qemu

If you are using qemu, the steps to a running VM are pretty short. You simply setup your VM with a CD-ROM and the virtual disk. The CD-ROM contains the BartPE disc (I use an ISO I call opengates.iso). I use the command line thusly:

$ sudo xmount --in ewf --out dd --cache disk.cache image.E?? /mnt

$ qemu -cdrom opengates.iso -drive file=/mnt/image.dd -boot menu=on

The "-boot menu=on" argument allows a boot menu to be selected with F12. I boot initially with BartPE, and opengates runs immediately. You can simply take the defaults for most options, the only catches being user password removal and AntiWPA (a Windows Product Activation) workaround. When opengates is done, it displays some necessary VM settings (but Qemu can't be configured--that I know of--with those settings), and reboots the system.

On the second boot, I boot from the virtual disk. In a recent case, the Windows Vista Business edition did not boot under qemu, however. So, I tried VirtualBox.

VirtualBox

VirtualBox can be deployed immediately, using the changes made with opengates under Qemu if desired. The changes are recorded in disk cache set when mounting, and the cache can be reused with subsequent mounts/boots. However, I'm going discuss VirtualBox as though the whole process was centered around its use.

First, xmount must be used to create the virtual disk:

$ sudo xmount --in ewf --out vdi --cache disk.cache image.E?? /mnt

Next, start and configure VirtualBox. I recommend the PUEL edition (downloaded from the website) if you plan to use USB, and it is easier to install. Setup your VM, using your opengates enabled BartPE ISO and the xmount virtual disk.

Boot the ISO, either by using the F12 key or by manipulating the boot order in the VM. Use opengates to configure the virtual disk as indicated in the Qemu section above. Be sure to note the VM settings recommended by opengates and configure your VM accordingly. You may not find exact matching settings based on the VM you are using and the version, but it shouldn't be hard to determine and you'll make them in the System tab. Common settings with be "Mother Board | Enable IO ACPI" and "Processor | Enable PAE/NX"

If the OS in your image was in a hibernated state, you'll likely not be able to restore it, but there is no harm in trying. If the OS fails to load the hibernation file, it will delete it and boot normally. If all goes well, you should boot into Windows. Remember, all changes are being cached in your disk cache file.

Snakes in the Grass

A couple of gotchas to look out for:

Make sure you are using xmount v0.4.4 if you want to use VirtualBox (v3.2.8 or later). A bug in previous versions causes VirtualBox to reject the image.
Make sure you configure your virtual machine with the settings provided by the opengates utility. For example, I needed to configure my VM to use ACPI before the Windows Vista Business OS in my virtual disk would boot.
Make sure you put the virtual disk on the IDE controller. Mine was automatically added to the SATA controller, but would not boot.

Android SMS Parsing

2010-10-07T12:45:00.000-07:00

There are two ways to skin this cat... but one is much better!

A colleague was looking for help parsing a Android mms/sms database (mmssms.db). It was an archived sqlite file pulled from a computer hard disk drive, apparently created through by a phone/computer sync. The main problem was that the sms table dates were in unix epoch time (with microseconds), ant there were about 2700 entries in the sms table alone.

Initially, I wrote a command that exported the contents of the "sms" table with sqlite using "select * from sms", read each line, assigned the date data to a variable, removed the microseconds (divide by 1000) from the date data, and used "date -d @$variable" to convert the date. Finally, I tacked on the converted date to the end of the line. The whole thing looked like this:

$ sqlite3 -header mmssms.db "select * from sms" | while read i; do a="$(echo $i)"; b="$(echo "$i" | cut -d '|' -f5 )"; c="$(date -d @$(echo $(($b/1000)))); echo $a\|$c; done

Effective, but not very elegant.

I suspected there was a way to convert the date in sqlite, and there was. It took me a short while to master it, but essentially, it was "select datetime(date, 'unixepoch', 'localtime') as date from sms". That will only get you the date from each line, however, so the full command looks like this:

$ sqlite3 -header mmssms.db "select _id, thread_id, address, person, datetime(date/1000,'unixepoch','localtime') as date, protocol, read, status, type, reply_path_present, subject, body, service_center, locked from sms"

Still a mouthful, but much faster and cleaner output.

Finally, I decided that it would be really nice to have all the tables in the database, a total of 13, exported to csv for documentation and review. Plus, I needed a way to remember this nifty conversion trick (the same date format is found in Firefox and Chrome histories). So I wrote a program to automate future processing. Two of the tables, sms and threads, had dates that needed to be converted, and the program makes provisions for these tables. You can download it here.

Previewer is Dead, long live CAINE!

2010-10-06T10:51:00.000-07:00

A brief history...

In the summer of 2009, I created a forensic boot disc primarily intended for preview examinations of digital media called, appropriately, "Previewer." It was based on Ubuntu 8.10 and included my home-grown tool, Ipod-ID, a bash script that searches iPod devices for evidence of ownership. Basic use of Previewer was taught at a training event for the Central California chapter of the HTCIA in September, 2009, and investigators from four counties were in attendance.

Previewer was modeled after CAINE 1.0, a forensic boot disc targeted at expert forensic examiners. CAINE was feature rich, but required a good foundation in computer forensics to fully utilized. My goal was bring a basic forensic examination ability to non-experts so that criminal cases could be filed before the computer got lost in the computer lab backlog. We debate the merits of that philosophy later.

With that goal in mind, I wrote a handful of nautilus-scripts to assist layman investigators with examination of digital media, primarily computer hard disk drives. The Nautilus file browser was the main examination tool, the the scripts, accessed with a right click in the file pane, providing examination functionality, including the ability to save data as evidence and produce simple reports about the saved data.

A new beginning...

Fast forward to today. Previewer remained stagnant after one revision to correct some minor issues with some of the nautilus-scripts. Since then, the Linux kernel has made advancements in hardware support, and the Ubuntu Linux distribution on which Previewer is based has undergone some major changes. More importantly, mounting schemes in most forensics boot discs, including Previewer and CAINE 1.0, were found to be flawed (in limited, but none-the-less important, ways).

Increased work load and decreased personal free time caused me to turn back the inspiration for Previewer: CAINE. CAINE has an active development team and a very capable project manager, Nanni Bassetti. CAINE was about to update to version 2.0, and the mount issues were corrected. I contacted Nanni and told him about my scripts and the philosophy behind them. He took a look at the scripts and felt they would be a good addition to CAINE.

Therefore, I am happy to announce that CAINE 2.0 was release 9/14/2010. It includes a much-improved set of nautilus-scripts from Previewer as well as all the expert tools expected from previous CAINE releases. Anyone accustomed to Previewer should have no trouble with using CAINE 2.0, and in fact, should find it more effective and useful than Previewer. The main "gotcha" is that the administrator account (root) now has a password (i.e, "caine").

Please contact Nanni (through the CAINE website) with comments or concerns about the main distribution. You can always contact me regarding the nautilus-scripts (here, by email, or through the CAINE webiste).

Processing Vista $RECYCLE.BIN

2009-10-20T09:45:00.000-07:00

The MS Windows recycle bin differs in Vista from XP. In XP, an deleted file is moved to the "\Recycler\$SID\ folder where "$SID" is the security identifier of the user deleting the file. The file is renamed, in the format "D" An "info2" file is created containing the file's original name, file deleted time, and original file size. In Linux, the command line tool rifuiti can be used to parse the info2 file. More information about the XP recycle bin can be found here.

Vista has a different schema, however. Deleted files are moved to the "\$RECYCLE.BIN\$SID\" folder, but this time no info2 file is created. Instead, a pair of files is created: the file itself is renamed randomly beginning with "$R" and an information file containing the 16-bit filename with the same name as the data file only beginning instead with "$I". Both files take on the extension of the original file. Deleted folders are subjected to the same deletion scheme, but the files and subdirectories within the deleted folders retain their original names and paths within the directory and are not otherwise represented in the recycle bin. The original file size and date of deletion are also contained in hex within the file. More information about the Vista recycle bin can be found here.

In Linux, one can quickly examine all the filenames of deleted files found in the Vista recycle bin. In the command line, simply change to the "/$RECYLE.BIN/$SID/" directory and issue this simple command:

# strings -el -f \$I*

The -el flag tells the command to look for 16-bit encoding and "-f" to print the filename of each processed file. The output resembles the following:

# strings -el -f \$I*
$I3XJZSP.docx: C:\Users\957630\Documents\1234567.docx
$I42NQXL.lnk: C:\Users\Public\Desktop\HP Total Care Advisor.lnk
$I4ULAZJ.url: C:\Users\957630\Desktop\Videos.url
$I96F8EZ.txt: C:\Users\957630\Documents\dm123.txt
$IA7VIFO.lnk: C:\Users\957630\Desktop\Mini400.lnk
...

File size and deletion date of files of interest can be determined from a hex view of the file.

0000000: 0100 0000 0000 0000 da26 0000 0000 0000 .........&......
0000010: d02a 0401 4c5c c901 4300 3a00 5c00 5500 .*..L\..C.:.\.U.
0000020: 7300 6500 7200 7300 5c00 3900 3500 3700 s.e.r.s.\.9.5.7.
0000030: 3600 3300 3000 5c00 4400 6f00 6300 7500 6.3.0.\.D.o.c.u.
0000040: 6d00 6500 6e00 7400 7300 5c00 3100 3200 m.e.n.t.s.\.1.2.
0000050: 3300 3400 3500 3600 3700 2e00 6400 6f00 3.4.5.6.7...d.o.
0000060: 6300 7800 0000 0000 0000 0000 0000 0000 c.x.............
0000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................

The first 8-bits of the file is the file header, the second is the file size, and the third is the deletion date. Thus, for this file, the original file size was 0xda26 or 55846 bytes:

# echo $((0xda26))
55846

And, the file deletion date was 0xd02a04014c5cc901 or "Fri Dec 12 03:23:06 PST 2008." This determination is a little more difficult and would benefit from a script, but for now, I'll show how its done manually.

Windows Time is recorded in Little endian format, thus we start with 0x01c95c4c01042ad0 which is 128735545861090000 nanoseconds since January 1, 1601. We devide by 10,000,000 (to convert from nanoseconds to seconds), and then subtract 11644473600 (the number of seconds between Windows Time and Unix Time, which is the number of seconds since January 1, 1970). Finally we convert to standard notation with the "date" command.

# echo $((0x01c95c4c01042ad0))
128735545861090000

# echo $((128735545861090000/10000000))
12873554586

# echo $((12873554586-11644473600))
1229080986

# date -d @1229080986
Fri Dec 12 03:23:06 PST 2008

In one command, this looks like:

# date -d @$((((0x01c95c4c01042ad0)/10000000)-11644473600))
Fri Dec 12 03:23:06 PST 2008

I'll contemplate writing a script to automate the whole process--filename, file size, deletion date--in the future. But for now, this will work.

linuxsleuthing code project

2009-08-04T10:18:00.000-07:00

Computers, iPods, Thumbdrives, oh My!

I've been busy with a major case during which many smaller cases have walked through my door. I began longing for a way for criminal investigators to be able to conduct preview examinations of digital storage devices without having to drop the devices off at the computer lab.

Many of the questions I receive are relatively simple to answer:

"Who owns this computer/iPod?"
"Does this computer have any illegal images/videos/files?"
"Are there any emails/chats between X and Y stored on this device?"

While I understand a preview examination of a digital storage device is not the equivalent of full forensic examination, the plethora of storage devices and the dwindling number of public sector forensic computer examiners begs for an intermediate solution. And, while forensically sound boot-discs exist, e.g., CAINE or the FBI's ImageScan, they do not lend themselves to criminal investigators with little or no computer forensics training (CAINE) and/or they only do one thing well (ImageScan).

My idea is to modify a disc like CAINE to include scripts accessible through a right-click menu that make basic digital storage device examination simpler. Virtually anyone who uses a computer understands the basics of navigating a file system with a file browser, and Nautilus is the file browser of choice in CAINE which utilizes the Gnome desktop. Nautilus has a built-in option for right-click scripts, and you'll find five already deployed in CAINE. The CAINE scripts have limitations and appear to have been obtained from g-scripts. When using them in a filtered file list (e.g., searching for documents in Nautilus), the scripts don't always work, nor are the scripts from a root Nautilus window.

My Solution

I created (and continue to create) a series of Nautilus scripts with which to remaster CAINE or add to an installed Linux distro. The problem has been that I do this across five different computer platforms, tweaking things as I go, to the point I'm not sure on which computer any particular script resides. Plus, no one else has access to them for use/testing/improvement, at least until I remaster CAINE and release. Therefore I have created a the LinuxSleuthing Google Code Project. I'll be populating the site with the scripts I create and encourage any feedback in the form of requests, bugs, suggestions, or improvement to the code.

The HTCIA Central California Chapter will be conducting training in September in the use of CAINE with many of these scripts, with primary focus on finding and previewing images, basic keyword searching, and iPod ownership identification, so the immediate focus of the scripts posted to the code project will be on these topics.

The scripts will follow the unix principle of "do one thing and do it well." They are not designed to run blind, finding all occurrences of index.dat, for example, and parsing them for histories. It will be up to the user to find the files and apply the parsing script. While this might sound labor intensive, it allows the user to surgically strike at desired data as time allows rather than wait for whole disk searches. That said, this concept may morph as real world trials expose flaws or beg enhancements.

Identifying Owners of Stolen iPods

2009-03-10T11:28:00.000-07:00

iTunes is your friend

I have gone from years of computer forensics without ever encountering Apple Products in any meaningful way to seeming only dealing with Apple laptops and iPods. I've been asked to attempt to identify the previous owners of suspected stolen devices. Today I'll discuss the iPod.
iPods are obviously small, portable devices that are easily stolen. I don't own one myself, so I don't know much about their internal functioning. I do know from experience that depending on the generation of the device, the file system can by fat32 or hfs+.

Determining the current username

One of the first things I discovered in trying to identify the current owner is that the iPod can be plugged into a Windows or Mac computer with iTunes installed, and iTunes will reveal the user name. Turning on the device and navigating the menu system will do the same. Neither effort is forensically sound, however, and changes to the stored data can/will occur.

In an Ubuntu Linux system (properly configured to not automount), the user name is displayed in the Disk Mounter applet or in Nautilus as "User's iPod" where User is the volume name of the data partition on the iPod. You can see this with the SleuthKit's mmls command using something like:

# mmls /dev/sdf

The problem with this approach is:

This is the current username. And it's just a username. If it's something like: Chuck E. Cheese, well you don't have much of a clue as to who the person is.
If the iPod is stolen and the owner data has been changed by theif, then you don't have the victim's username, but instead the suspect's.

Determining the victim's real name/email address

A little "googling" led me to understand that iTunes purchased music contains the purchaser's email address embedded in the song to help Apple track music pirates. A little hexeditor fun helped me determine that there is quite a bit of information embedded in the .m4p songs from iTunes, including email address of the purchaser AND the first and last name. The name's source looks to be the account information from registering with iTunes (which requires credit card payment), but I have not confirmed this.
My approach to determining the victim's real name is to search the unallocated portions of the partition for the tag "name" followed immediately by an alphanumeric character (alphanumeric in the event that I'm incorrect in the assumption that the music purchaser's name coming from the iTunes credit card-verified account).
Using the Sleuthkit's blkls command to extract unallocated blocks, we pipe them through grep to search for the name tag:
# blkls /dev/sdf2 | grep -aE 'name[0-9a-zA-Z]+'
This could be similarly accomplished with:
# blkls /dev/sdf2 | grep -aE 'name[[:alnum:]]+'
The breakdown of the command is as follows:

blkls /dev/sdf2 - blkls sends unallocated blocks to standard input (the terminal window). Rather than choose the raw device /dev/sdf2 and designate an offset, I have chosen to address the partition directly for simplicity.
grep -aE - grep searches the data piped to the terminal. The -a flag treats the data as binary, display matches in the terminal rather than just reporting match occurred. The -E treats the following quoted expression as a regular expression rather than matching it literally.
'name[0-9a-zA-Z]+' or 'name[[:alnum:]]+' - both expressions mean: search for the ascii string "name" followed immediately by at least one number or letter.

The grep search could be run against the allocated data as well, to compare the results to the unallocated data as you assess the likelihood the unallocated matches represent information about a previous owner.
This methodology represents a "down and dirty" approach to determining ownership. It does not provide a well documented location evidence can be found to prove theft for court purposes. Consider it a triage approach to determining ownership to propell a case forward. Further documentation would be needed to produce useful forensic information, such as documenting byte offset, carving the files from which the hits were made, etc., but this exceeds the scope of this article.