TinyCore: A Mighty Platform (Part 2)

TinyCore Linux is an ideal platform for building a light weight forensics distribution with the purposes I have in mind (See Part 1 of this post).  It is only a 10mb download for the base distribution and boots to a simple GUI desktop.  It boots and loads entirely into a ram disk as small as 48mb, but allocates as much ram as possible.  The ram disk makes TinyCore very fast because the entire operating system resides in ram and there are no drive seek time delays.

TinyCore uses a modern kernel with good hardware support and an xvesa video driver which all but insures a working GUI.  Applications are installed as modules (called extensions) that can be run at boot time or on demand.  The root file system and the applications are read-only and are renewed on every boot eliminating file corruption that can creep into installed software.

The Basic Structure

At its most rudimentary level, TinyCore consists of two files, the kernel (bzimage) and a compressed file system (tinycore.gz).  Add to that a means to boot the operating system, such as isolinux, and your full file tree is a simple:

./boot
./boot/isolinux
./boot/isolinux/isolinux.cfg
./boot/isolinux/isolinux.bin
./boot/isolinux/f4
./boot/isolinux/f2
./boot/isolinux/boot.msg
./boot/isolinux/f3
./boot/isolinux/boot.cat
./boot/bzImage
./boot/tinycore.gz

Making the Read-Only Environment

Attached storage devices detected by the kernel are identified by the udev daemon. Udev applies rules to the devices based on their type.  In the case of TinyCore, udev calls the /usr/sbin/rebuildfstab script to build the /etc/fstab file which contains the mounting options for the attached devices.  When the device is subsequently mounted (devices are not automatically mounted in TinyCore when attached), the mount options in the fstab file are applied.  One need only modify rebuildfstab mounting options to make the system mount devices read-only.

I have been able to modify the rebuildfstab file to mount devices read only and address other forensic mounting issues, like mounting ext3/4 devices as ext2 to avoid any possible journal changes and mounting physical devices as loopback devices to avoid attempted repairs of corrupted file systems on mount.

The process of modifying, adding, or removing files in the core file system is well documented here.  It involves decompressing the tinycore.gz file extracted from a TinyCore iso, making the desired changes, and zipping it back up.  The new tinycore.gz can then be remastered into a new iso.

Making Application Modules (Extensions)

Though new applications can be remastered into the core file system, I favor the modular approach implemented by the TinyCore developers.  Applications are compiled and the stored in a read-only squashfs file system.  The application, when installed, is mounted into the core file system.  Applications can be triggered to mount at boot time, or on demand.  On demand ensures quicker boot times and frees more space in the ram disk if the application is not needed in the session.  Though there is not gui method for this, installed applications can be "uninstalled" in the middle of the session by simply unmounting them and thus freeing ram allocated to them.

Though TinyCore has some suitable modules for forensics, like foremost for example, it lacks libraries and application such as libewf (Expert Witness imaging format), afflib (Advanced Forensics imaging Format), and sleuthkit (disk investigation tool) that a forensics practitioner would desire.  If you are familiar with building application from source, however, then building TinyCore application modules is a snap.  I have already built libewf, afflib, aimage, and sleuthkit modules and will submit them to the repository once I complete testing.  You can take a look at the building method here.

Persistence

Everything I've mentioned about TinyCore so far mentions "read-only."  The rebuildfstab script can be modified to ensure devices are mounted read-only, a must for live forensic examinations.  The core file system and application modules are mounted read-only ensuring a "clean" operating system and software environment with each boot.  But how does a user save evidence from examinations?

TinyCore allows the home directory to be saved to a storage device.  On shutdown, user data is written to the storage device designated by the user.  A boot option allows the device to be specified on the next boot to restore the user data, or it can be loaded after boot.

Putting it All Together

If you read Part 1 of this post, you know that my goal is the creation of a bootable disc/USB that an investigator with average computer skills (not a computer forensics practitioner) could use to search for and seize evidence from digital storage devices.  TinyCore, in my estimation, has it all:

1. Small size, loads entirely into ram, and fast with a simple GUI
2. Easily modified and remastered as a read-only environment
3. Easily add and created application modules with minimal ram impact
4. Means for easy creation and restoration of persistent storage

If one adds to the base a decent file browser, like ROX (I'll explain why I think this file browser is great option for forensic examination another time), a word processor (Abiword) with decent file format support, an audio/video player (VLC), and maybe a few other applications, and you have a light-weight, fast, and safe operating system that an investigator with basic computer skills can use to advance his case.

I have a basic version of this concept in place and am currently testing and refining.  I plan to host it on Google Code to get community feedback and to publish the changes I make to the core operating system.  I'll also host the modules I build there (at least until they are accepted into the TinyCore repository).  As always, I welcome any feedback.

TinyCore: A Mighty Platform (Part 1)

Last week I rediscovered TinyCore Linux.  I had taken a look at it about 6 months ago and was intrigued, but didn't have the time to explore it further.  However, I have been seeking a small Linux distribution on which to build a specialized forensics distro, and last week I gave TinyCore another look.

Background

I believe there is a need in computer forensics for an investigator with limited training to be able to search for and seized digital evidence from storage devices.  Some of the reason's I believe this are:

1. There are not enough trained forensic computer examiners to keep pace with the number of cases involving digital evidence.
2. The backlog created by a lack of examiners means cases don't get filed for month or even years after the discovery of the crime.  Meanwhile, the perpetrator is free to commit more crimes.
3. Prosecutors are less likely to pursue older cases, in part because witness recall becomes unreliable.
4. The majority of charges filed against perpetrators are settled out of court through plea bargaining.  

Therefore, in most circumstances digital storage devices are taken to computer forensics laboratories to search for evidence to support a filing of criminal charges.  But the labs are too busy to get to the examinations very quickly, and by the time they do, Prosecutors are reluctant to file charges because of the delayed filing and/or the perpetrators have been committing additional crimes.  I know this doesn't describe all situations, but it should ring true with most people in some manner.

Solution

The obvious solution is to increase the number of forensic computer examiners and computer forensics laboratories.  However, that isn't going to happen, at least not in the near and not-so-near futures.  And, since I'm a "work with what I've got" kind of guy, I've been working on another solution:

Criminal investigators need simple but effective tools to search for and seize evidence from digital storage devices.  The tools need to be forensically sound, i.e., they do not alter the original media in any way, but easy enough to use that a basic computer user can feel comfortable and conduct effective examinations.

Think about it this way: If a criminal investigator could retrieve his own digital evidence, he could file charges immediately, and most of the cases filed would be settled without the need of further forensic computer examination.  In cases that do not settle because the digital evidence is disputed, the storage devices could be sent to the computer forensics labs for more traditional analysis.

More cases filed, more perpetrators convicted, less workload at the lab!

But how do we create such tools?  Forensic boot discs like CAInE are great for experience investigators, and the latest version contains nautilus scripts to make live examinations like I'm contemplating here possible.  But the operating system is resource heavy, slow to boot from CD, and still to complicated for basic criminal investigators (for example, it is confusing and difficult for most basic users to mount a storage device read-write to collect evidence because the CAInE mounting policies rightly auto-mount devices read-only).  In other words, CAInE and other existing boot discs are not the right tool for users with limited computer forensics training.

TinyCore

I believe the best tool for criminal investigators with basic computer skills will:

1. Boot quickly (Criminal investigators may be in the field without the luxury of time.
2. Work in nearly any machine (basic video drivers, e.g., xvesa)
3. Not alter the media being examined (i.e., mount devices read-only)
4. Create an writeable storage location automatically (no command line or confusing the evidence device for the storage device)
5. Contain programs or scripts that are easily accessible to find evidence files (e.g., nautilus-scripts)
6. Create reports about files saved as evidence containing file metadata (so evidence can be commented upon by trained investigators, if needed)
7. Allow for the creation of forensic images (in the event the device cannot be seized).

TinyCore linux appears to be an ideal platform from which to build this tool.  And, I'll explain why in Part 2...

Sleepwaking

No, the title of this post is not in error.  I don't mean "sleepwalking."  I meant what I said: sleepwaking.

Those who know me know I am a proponent of "previewing" computers for content before conducting a full forensic exam.  There are many reasons for this including the most common purpose: triage.  Previewing entails booting the computer with a forensic boot disc such as CAINE which prevents writes to the media being examined, and examining the file system(s) on the attached storage devices for evidence related to the investigation.

There are, of course, limitations to Previewing approach that I won't discuss here, But there is a danger that I feel must be discussed: Previewing the computer that is in a sleep or hibernated state which is commonly (but not exclusively) found in laptop computers.  Dropping a forensic boot disc into the CD drive (or using a USB incarnation) will not prevent the computer's operating system from running and changing data on storage device.  So, what is a person to do?

For sake of this discussion, envision a laptop computer with a single internal hard disk drive with an Windows operating system in a hibernated state.  Booting the computer after inserting a forensic boot disc will NOT preserve the data on drive, because the BIOS detects the hibernated state and begins restoring the contents of the hyberfil.sys file to RAM and restores the operating system to a running state.  If you interrupt this restoration, quickly enough, you will find that the data on the hard disk drive hasn't changed, but metadata (access times) will be changed.  Depending on the version of Windows installed, you still may not get a boot selection screen the next time you boot (Windows 7 comes to mind).

The best practice in my experience is as follows:

1. remove the hard drive from the computer.
2. insert your forensic boot disc .
3. boot the computer and access the BIOS or boot selection menu (ensure you can select the boot disc as the boot device)
4. reinstall the hard disk drive.
5. reboot the computer and boot into your forensic environment.  The BIOS will not now automatically attempt to restore the resident OS from hibernation.

And don't forget that the hibernated system has saved what is essentially a RAM image in the hyberfil.sys.  I guess that's what you call a "twofer."  It takes a different shape in Macintosh and Linux systems, but RAM contents are available in those hibernated operating systems, too.

Of course, true "sleeping" is different than hibernation, though the line has been blurred since Windows Vista.  A true "sleep" state suspends the computer, preserving the running state by providing power to RAM in order to retain data.  It is usually indicated by a flashing power light and the data disappears scant seconds after loss of power.  On the other hand, Windows Vista introduced "Fast Sleep" that also saves the RAM content to hard disk in conjunction with providing power to the RAM.  The investigator will have to make his/her own decisions based on the details of the investigation on how to handle a sleeping computer (e.g, wake it and save volatile memory before shutting down the system or abandoning the RAM contents by pulling the drive).

Garmin GPS: What you don't know can track you!

Garmin GPS devices track their position by default (caveat: at least every device I have been given to examine!).  They will do so, approximately every 30 seconds, when powered on.  Notice I said nothing about navigating.  Simply powering the devices causes them to start logging their location.  While this feature can be disabled, it is buried in the settings and I suspect that most users are not even aware of it.

The data is stored in a GPX file, also know as the Global Positioning Satellite (GPS) Exchange Format.  The most current track, appropriately named "Current.gpx," is stored in the "/Garmin/GPX" directory on the device.  Older tracks are stored in "/Garmin/GPX/Archive" directory.  The archives take on the name ".gpx," e.g,. "1.gpx," "2.gpx," etc.  I have never seen more that 17 archived files, but I don't know if this is a system limitation or just a coincidence that I have seen it more than once.  The history can cover quite a time span: my most recent examination revealed a history of 6 months!

GPX files are in xml.  The Current.gpx file can have interesting entries, including the "Home" address of the device owner.  I have used this setting to reunite stolen devices with their owners or thieves back to their homes. But the most interesting information is the device track, which consists of a series of GPS waypoints or "trackpoints" recorded by the device.  Here is a sample from an archive file:

There are many ways to handle a GPX file, but I have found it is most useful to convert it to a KML, or Key Hole Markup Language, file for use with Google Earth.  While I know that Google Earth is not an open sourced application, and other tools like "gpxviewer" can map the GPX file directly, most of the people I support are Windows users that have experience with Google Earth.

There are two methods I am aware of for creating KML files.  The first is using an online resource, like GPSVisualizer.  Just complete the online form and upload your file to make a map that meets your requirements.  Other formats, besides Google Earth, are possible, including Google Maps, JPG, PNG, SVG, and text.

I don't like to rely on websites, however, because Internet connectivity is never assured.  Enter GPSBabel.  GPSBabel is a command line tool (gui available) to convert over 100 different types of GPS data formats.  A basic conversion can be as simple as:

gpsbabel -i gpx -f input.gpx -o kml -F output.kml

There are numerous options, that I won't cover here, to customize your output file.  They include labeling the way points with the date and time they were created, allowing you to easily visualize the track.  I'd recommend the use of a GUI to familiarize yourself with the customization options, though be aware that the GUIs seldom provide all available options.

I have used Garmin GPX files to map a suspects' travels and place him them at crime scenes.  I hope with this information you will be able to do so, too!

Sleuthkit 3.2.0 Released

A new version of The Sleuth Kit (TSK), a command line forensics tool set for both Linux and Windows, was released a few days ago.  This release brings new automation tools that can greatly speed processing.  Brian Carrier, the developer, describes the release thusly:

New features include:
   • New tsk_recover tool that extracts files from an image to a local directory.
   • New tsk_loaddb tool that dumps file system metadata to SQLite database.
   • New tsk_getimes tool that collects MAC time data on all file systems (equivalent to fls -m on a series of volumes)
   • New tsk_comparedir tool that compares a directory to an image to detect rootkits.
   • New C++ TskAuto class that makes it easier to create automated tools that analyze all files.
   • Name cleanup out of libraries and into tools.
   • img_cat -e and -s flags.
   • Changed how default NTFS $Data attribute is named.
   • HFS+ Case sensitive flag in fsstat.

Bug fixes include:
   • FAT performance
   • Crash fix for corrupt NTFS file
   • Adding attribute runs on fragmented files with multiple attributes of the same type.

I can personally attest to the benefits of the tsk_loaddb tool.  It very quickly creates a sqlite database containing file system metadata for each volume in the forensic image.  The database can be queried for any data sought in a fraction of the time it takes to read a file system itself.

A knowledge of  basic sqlite commands is essential to take advantage of the database, but with that knowledge, it is quite easy to script tools for your use.  For example, I wrote a shell script that reads the database and automatically mounts all the partitions read-only for quick evaluation.  This, of course, is only one small way that data can be used.

For a concise overview of all the tools that TSK has to offer, take a look at the wiki located here.

Booting Disk Images in Linux

A Picture can be Worth a Thousand Words

Sometimes there is no substitute for booting a computer to gather data.  But the forensic consequences of  directly booting the system under investigation are prohibitive.  And, while it can be possible to restore a computer image to another hard drive and install that drive in the computer hard drive for booting, this can be costly, very time consuming, and you only get one shot before you've changed the data from original (and repeated boots may be desirable depending on the investigation).

In response to this, tools like LiveView were created.  The concept was to boot the forensic image, rather than restore it, and capture all system changes in a disk cache.  The operating system could be booted quickly and repeatedly from an original state.  However, I never found LiveView to be very effective, and it the past few years I have avoided Windows-based tools whenever possible .

Enter xmount by Dan Gillen.  Xmount will mount raw (dd), Expert Witness Format (ewf, aka, EnCase), and possibly Advanced Forensics Format (aff) images and create a virtual disk that can be run in a virtual machine.  Supported virtual disks are raw (qemu), vdi (VirtualBox), and vmdk (VMware).  Xmount is easy to use: its as simple as invoking the command, listing the image type, the virtual disk type desired, assigning a disk cache file, and selecting the disk image to mount.  For example:

$ sudo xmount --in ewf --out vdi --owcache disk.cache image.E?? /mnt

Of course, you've only come partway to actually booting that image.  You could try to boot the virtual disk in your virtual machine, but if it's a Windows operating system, chances are it will fail.

Open the Flood Gates with "opengates"

Windows operating systems rely on the hardware of the system in which they are installed to boot.  Thus, you cannot normally remove a hard disk drive with a Windows operating system, install it in a differently configured system, and expect it to boot.  Thus, neither will your virtual disk boot in your virtual machine, unless the configuration of the virtual machine closely matches that of the original system.  However, xmount creator Dan Gillen has a solution for us: "opengates."

Opengates is a utility to overcome the hardware limitations of Windows Operating systems.  It does several other things to make the virtual disk bootable, but one most interesting and useful capability is the removal of user passwords.  Opengates is deployed with BartPE, a bootable live windows environment.  Gillen provides very clear instructions for creating an opengates enabled BartPE disc with PEbuilder.  But in short, you need a Windows XP or 2003 machine (to run PEbuilder), a licensed Windows XP install disc, the opengates software, and about 10 minutes.

Configuring the VM

Qemu

If you are using qemu, the steps to a running VM are pretty short.  You simply setup your VM with a CD-ROM and the virtual disk.  The CD-ROM contains the BartPE disc (I use an ISO I call opengates.iso).  I use the command line thusly:

$ sudo xmount --in ewf --out dd --cache disk.cache image.E?? /mnt

$ qemu -cdrom opengates.iso -drive file=/mnt/image.dd -boot menu=on

The "-boot menu=on" argument allows a boot menu to be selected with F12.  I boot initially with BartPE, and opengates runs immediately.  You can simply take the defaults for most options, the only catches being user password removal and AntiWPA (a Windows Product Activation) workaround.  When opengates is done, it displays some necessary VM settings (but Qemu can't be configured--that I know of--with those settings), and reboots the system.

On the second boot, I boot from the virtual disk.  In a recent case, the Windows Vista Business edition did not boot under qemu, however.  So, I tried VirtualBox.

VirtualBox

VirtualBox can be deployed immediately, using the changes made with opengates under Qemu if desired.  The changes are recorded in disk cache set when mounting, and the cache can be reused with subsequent mounts/boots.  However, I'm going discuss VirtualBox as though the whole process was centered around its use.

First, xmount must be used to create the virtual disk:

$ sudo xmount --in ewf --out vdi --cache disk.cache image.E?? /mnt

Next, start and configure VirtualBox.  I recommend the PUEL edition (downloaded from the website) if you plan to use USB, and it is easier to install.  Setup your VM, using your opengates enabled BartPE ISO and the xmount virtual disk.

Boot the ISO, either by using the F12 key or by manipulating the boot order in the VM.  Use opengates to configure the virtual disk as indicated in the Qemu section above.  Be sure to note the VM settings recommended by opengates and configure your VM accordingly.  You may not find exact matching settings based on the VM you are using and the version, but it shouldn't be hard to determine and you'll make them in the System tab.  Common settings with be "Mother Board | Enable IO ACPI" and "Processor | Enable PAE/NX"

If the OS in your image was in a hibernated state, you'll likely not be able to restore it, but there is no harm in trying.  If the OS fails to load the hibernation file, it will delete it and boot normally.  If all goes well, you should boot into Windows.  Remember, all changes are being cached in your disk cache file.

Snakes in the Grass

A couple of gotchas to look out for: 

1. Make sure you are using xmount v0.4.4 if you want to use VirtualBox (v3.2.8 or later).  A bug in previous versions causes VirtualBox to reject the image.  
2. Make sure you configure your virtual machine with the settings provided by the opengates utility.  For example, I needed to configure my VM to use ACPI before the Windows Vista Business OS in my virtual disk would boot.
3. Make sure you put the virtual disk on the IDE controller.  Mine was automatically added to the SATA controller, but would not boot.

Android SMS Parsing

There are two ways to skin this cat... but one is much better!

A colleague was looking for help parsing a Android mms/sms database (mmssms.db).  It was an archived sqlite file pulled from a computer hard disk drive, apparently created through by a phone/computer sync.  The main problem was that the sms table dates were in unix epoch time (with microseconds), ant there were about 2700 entries in the sms table alone.

Initially, I  wrote a command that exported the contents of the "sms" table with sqlite using "select * from sms", read each line, assigned the date data to a variable, removed the microseconds (divide by 1000) from the date data, and used "date -d @$variable" to convert the date.  Finally, I tacked on the converted date to the end of the line.  The whole thing looked like this:

$ sqlite3 -header mmssms.db "select * from sms" | while read i; do a="$(echo $i)"; b="$(echo "$i" | cut -d '|' -f5 )"; c="$(date -d @$(echo $(($b/1000)))); echo $a\|$c; done

Effective, but not very elegant.

I suspected there was a way to convert the date in sqlite, and there was.  It took me a short while to master it, but essentially, it was "select datetime(date, 'unixepoch', 'localtime') as date from sms".  That will only get you the date from each line, however, so the full command looks like this:

$ sqlite3 -header mmssms.db "select _id, thread_id, address, person, datetime(date/1000,'unixepoch','localtime') as date, protocol, read, status, type, reply_path_present, subject, body, service_center, locked from sms"

 Still a mouthful, but much faster and cleaner output.

Finally, I decided that it would be really nice to have all the tables in the database, a total of  13, exported to csv for documentation and review.  Plus, I needed a way to remember this nifty conversion trick (the same date format is found in Firefox and Chrome histories).  So I wrote a program to automate future processing. Two of the tables, sms and threads, had dates that needed to be converted, and the program makes provisions for these tables.  You can download it here.

Previewer is Dead, long live CAINE!

A brief history...

In the summer of 2009, I created a forensic boot disc primarily intended for preview examinations of digital media called, appropriately, "Previewer."  It was based on Ubuntu 8.10 and included my home-grown tool, Ipod-ID, a bash script that searches iPod devices for evidence of ownership.  Basic use of Previewer was taught at a training event for the Central California chapter of the HTCIA in September, 2009, and investigators from four counties were in attendance.

Previewer was modeled after CAINE 1.0, a forensic boot disc targeted at expert forensic examiners.  CAINE was feature rich, but required a good foundation in computer forensics to fully utilized.  My goal was bring a basic forensic examination ability to non-experts so that criminal cases could be filed before the computer got lost in the computer lab backlog.  We debate the merits of that philosophy later.

With that goal in mind, I wrote a handful of nautilus-scripts to assist layman investigators with examination of digital media, primarily computer hard disk drives.  The Nautilus file browser was the main examination tool, the the scripts, accessed with a right click in the file pane, providing examination functionality, including the ability to save data as evidence and produce simple reports about the saved data.

A new beginning...

Fast forward to today.  Previewer remained stagnant after one revision to correct some minor issues with  some of the nautilus-scripts.  Since then, the Linux kernel has made advancements in hardware support, and the Ubuntu Linux distribution on which Previewer is based has undergone some major changes.  More importantly, mounting schemes in most forensics boot discs, including Previewer and CAINE 1.0, were found to be flawed (in limited, but none-the-less important, ways).

Increased work load and decreased personal free time caused me to turn back the inspiration for Previewer: CAINE.  CAINE has an active development team and a very capable project manager, Nanni Bassetti.  CAINE was about to update to version 2.0, and the mount issues were corrected.  I contacted Nanni and told him about my scripts and the philosophy behind them.  He took a look at the scripts and felt they would be a good addition to CAINE.

Therefore, I am happy to announce that CAINE 2.0 was release 9/14/2010.  It includes a much-improved set of nautilus-scripts from Previewer as well as all the expert tools expected from previous CAINE releases.  Anyone accustomed to Previewer should have no trouble with using CAINE 2.0, and in fact, should find it more effective and useful than Previewer.  The main "gotcha" is that the administrator account (root) now has a password (i.e, "caine").

Please contact Nanni (through the CAINE website) with comments or concerns about the main distribution.  You can always contact me regarding the nautilus-scripts (here, by email, or through the CAINE webiste).