Tuesday, June 19, 2012

Data exchange: Archiving and Moving Data between OSes

The Back Story


It finally happened (sigh): I found myself in need of a Mac.  I don't much enjoy Mac computers, but I'll save my rants for my Mac-user friends who love to hear from me on the topic.  But I needed to extract data from an iPhone4 with iOS 5.0.1.  I've discussed previously that libimobiledevice provides an open-source suite of tools that can be used to extract data from an iPhone with Linux.  Frankly, I need to freshen that post as the tools have become more robust and, as of this writing, support extracting data (by means of an iTunes-like backup with the idevicebackup tool) through iOS 5.1.1.

But, in my case, though I only needed files that were available through idevicebackup, the phone was pin-locked which prevents the utility from functioning.  I am fortunate enough to have access to the tools provided by Johnathan Zdziarski at the iOS Forensic Research site, one of which provide the pin code from a locked iPhone.  One of the facts required for the pin code tool to work is knowledge of the device model, discoverable on the back of the phone, and the iOS version installed.  Discovering the iOS version is not always trivial, unless you have libimobiledevice-utils.

Useful Digression: Determining the iOS Version of a Device


The libimobiledevice-utils toolset provides the ideviceinfo tool which displays valuable information about an unlocked iPhone, Touch, or iPad in key/value pairs.  The info includes the device serial number, mac address, bluetooth address, phone number, device name ("Badguy's iPhone), cpu type, hardware model, firmware version, even the color of the phone.  Under the product version key, the iOS version is displayed. You may have noticed I said "unlocked" device.  Information, though in to a lesser degree, can be obtained from a locked device as well, but nothing you read in the help will make that obvious.  If you pass the -s / --simple option, you can still obtain limited but critical information to include the device name and the product (iOS) version.

With the iOS product version in hand, I proceeded to obtain the pin code with the aide of Zdziarski's tools.  I also downloaded the logical file system, consisting of over 12gb of data in one tar ball.  I needed to preserve the archive as evidence, and I also wanted to transfer it to a Linux box for examination, where I have more contemporary command line tools available to me than provided through OS X.

The Point: Moving Data Between Operating Systems


I like easy, and I like methodology that is cross-platform, whenever possible.  To be more specific, I prefer to use the same tools no matter the OS.  I have sort of adopted a unix philosophy of forensics: the unix tool philosophy is make one tool to do one thing and do it well.  My forensics philosophy is learn one tool to do one task and learn it well.  Now, I know its not always possible, but I want to avoid the problem of using Zip compression and tools because I'm in Windows, Gzip/Bzip and related tools because I'm in Linux where such compression is dominant, and Stuffit for Mac, etc.  Much better to learn one method that will work in most any situation without the need to download and install yet another utility, in my way of thinking.


Because I was working on the Mac command line, it made sense to stay there and chop up that 12gb tar file into DVD sized chunks.  And for fun, I decided to see if I could squeeze down the size some (it turned out that much of the data was video, and there wasn't much compression to be had, however).  So, how can it be done?  Newer incarnations of tar allow the creation of volumes, but not so in OS X Lion.  However, the bzip2 compressor and the split commands are part of the standard command line tools.

The concept is simple: compress the tar file and chop it into 4.1gb chunks for burning.  Bzip2 handles the compression, but it creates a file of the same name as the target file with the .bz2 extension by default.  Thus, "archive.tar" becomes "archive.tar.bz2."  This does not accomplish my goal, because I'm just left with a somewhat smaller file, but still too large to burn to DVD.  However, bzip2 takes the -c argument which pipes the data to stdout instead of creating a new file!  Now we're getting somewhere.

We can take the data piped from bzip2 and use our Ginsu (that's a knife for those of you who didn't watch a lot of TV in the 1980's) command called split.  Split will divide if file into user-defined sized pieces.  The sizes can be based on the number of lines, amount of data, and if I read the man page correctly on the Mac, even a data pattern.  In my case, I want to define it by size, and the Mac version of the tool tells me to specify the -b flag for the byte size of the resulting files, appending 'k' for kilobytes and 'm' for megabytes.  The single hyphen, used as an argument, replaces a file name when data is being piped from another tool like bzip2.

So, putting it all together, I split my archive.tar file thusly:
$ bzip2 -c archive.tar | split -b 4100m - archive.tar.
Bzip compresses archive.tar and sends the data to stdout.  That data is piped through the split command, which creates 4.1gb chunks.  The resulting files were archive.tar.aa, archive.tar.ab, and archive.tar.ac.  Note the hyphen in the command after the -b file size option: that is 'shortcut' for the data from the bzip2 command and replaces what would otherwise be the name of the file for split to operate upon.  Also note the period at the end of archive.tar.  Without it, the output files would have the name 'archive.taraa,' etc., without the period separator between the original file extension and the new one supplied by split.

What Now?


I burned those files to three DVDs, which provided the necessary evidence retention medium.  I also copied those files to my Linux box where I reassembled and decompressed them in one operation:
$ cat archive.tar.* | bunzip2 > archive.tar
I did not untar the archive, because I can mount that read-only and examine it as is.  This provides a better forensic environment for examination.  I'll cover this method in a future post.  Finally, because I install and maintain a Cygwin environment on my Windows workstation, I could have used the same method to transfer the files to my Windows operating system.  One method, three operation systems interchange data.  Now, that's the way I like it!

Yet Another YAFFS Discussion

In previous posts, I've discussed rooting and imaging Android devices.  While the exploits change from one Android version to another, the principals are the same as I detailed in the past.  Most Android devices, small portable devices like smart phones in particular, use NAND flash memory with the yaffs file system for storage.

If you are new to building binaries from source code, then this tutorial is probably not for you.  However, I hope to explain it well enough that you can still follow along even if you have very little build experience.  For starters, make sure you have the appropriate build tools.  In Debian and Ubuntu, it's easiest to to install the "build-essential" package:

$ sudo apt-get install build-essential

Though the next step is not required, you'll likely want to install the "git" software versioning system so you can easily obtain and install the latest yaffs source code.  Otherwise, it is possible to download the source code as a tar archive from the source code repository.  I'll be demonstrating the git method here:

$ sudo apt-get install git

Finally, you'll likely want to install the module for easy access.

$ sudo apt-get install module-init-tools

Building the Module

In order to mount Android images, download the latest source code from the online repository.  You'll probably have to install git if you haven't done so in the past.  It is not standard in most Linux Distros.
$ git clone git://www.aleph1.co.uk/yaffs2 Cloning into yaffs2...
remote: Counting objects: 7027, done.
remote: Compressing objects: 100% (4247/4247), done.
remote: Total 7027 (delta 5566), reused 3473 (delta 2700)
Receiving objects: 100% (7027/7027), 3.43 MiB | 304 KiB/s, done.
Resolving deltas: 100% (5566/5566), done.

The source code is downloaded into a subdirectory called 'yaffs2' is used the example command above.  If you want to clone into a different directory, add the directory name as an argument following the web address.  If the directory doesn't already exist, it will be created.

Next, change into the source code directory and issue the "make" command to build the source according to the parameters already laid out in the Makefile.

$ cd yaffs2
$ make 
make -C /lib/modules/2.6.38-13-generic/build M=/home/jlehr/projects/yaffs2 modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.38-13-generic'
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_mtdif.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_mtdif2_multi.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_mtdif1_multi.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_packedtags1.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_ecc.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_vfs_multi.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_guts.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_packedtags2.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_tagscompat.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_checkptrw.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_nand.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_nameval.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_allocator.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_bitmap.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_attribs.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_yaffs1.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_yaffs2.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_verify.o
  CC [M]  /home/jlehr/projects/yaffs2/yaffs_summary.o
  LD [M]  /home/jlehr/projects/yaffs2/yaffs2multi.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/jlehr/projects/yaffs2/yaffs2multi.mod.o
  LD [M]  /home/jlehr/projects/yaffs2/yaffs2multi.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.38-13-generic'

Finally, install the module.
$ sudo make mi #or sudo make modules_install
make -C /lib/modules/2.6.38-13-generic/build M=/home/jlehr/projects/yaffs2 modules_install
make[1]: Entering directory `/usr/src/linux-headers-2.6.38-13-generic'
  INSTALL /home/jlehr/projects/yaffs2/yaffs2multi.ko
  DEPMOD  2.6.38-13-generic
make[1]: Leaving directory `/usr/src/linux-headers-2.6.38-13-generic'


Mounting a yaffs image

I was planning to finish this discussion with mounting a yaffs image, but its a more complex topic than I can reasonably handle in a few lines.  Look for a discussion on the complexities of mounting a yaffs image, and maybe the methods for obtaining one, in a future post.