Xmount: When "Changing" the Evidence isn't so Bad

"Do no harm" is the modern translation of the Hippocratic Oath which is applied to physicians. But it has application in data forensics as well. It takes shape in the edicts that require write-blocking be used during the acquisition of data sources and analysis to be done on copies rather than original data. (I’ll leave the very valid discussion about triage through direct examination of data sources aside for another time. We’re talking general principles here.)

Can we ever change the evidence?

The short answer is, "No." We should never change the original data, period. But that doesn’t mean that we can’t render a copy of the data readable. After all, it is better to read the data with the programs intended to use the data… that way we know we are rendering the information as it was intended to be read. And if there is a way to repair a file or file system in a manner that doesn’t change the substantive content, should we not consider that option?

Enough vagaries. Let’s get down to brick and mortar to make this point. I’ve previously discussed using xmount to run the operating systems encapsulated in forensic images. In that situation, xmount uses a cache file to record and read changes to the file system that necessarily occur when the operating system and applications are running. Because the changes are written to the cache, the forensic image is unchanged.

Yesterday, I encountered another use for xmount when examining an image of an eMMC NAND chip from a Samsung Galaxy S3. I attempted to mount the 12GB userdata partition for analysis, but mounting failed. This may have happened to you in the past: you attempted to mount a file system the way you always do, but for unknown reasons, your command failed. What to do?

When Standard Procedure Fails

Let’s take my S3 image for example. My goal was to access the userdata partition and extract the files. The S3 is supposed to have ext4 partitions which I should be able to mount and examine in Linux.

BASH

$ mmls image
GUID Partition Table (EFI)
Offset Sector: 0
Units are in 512-byte sectors

     Slot    Start        End          Length       Description
00:  Meta    0000000000   0000000000   0000000001   Safety Table
01:  -----   0000000000   0000008191   0000008192   Unallocated
02:  Meta    0000000001   0000000001   0000000001   GPT Header
03:  Meta    0000000002   0000000033   0000000032   Partition Table
04:  00      0000008192   0000131071   0000122880   modem
05:  01      0000131072   0000131327   0000000256   sbl1
06:  02      0000131328   0000131839   0000000512   sbl2
07:  03      0000131840   0000132863   0000001024   sbl3
08:  04      0000132864   0000136959   0000004096   aboot
09:  05      0000136960   0000137983   0000001024   rpm
10:  06      0000137984   0000158463   0000020480   boot
11:  07      0000158464   0000159487   0000001024   tz
12:  08      0000159488   0000160511   0000001024   pad
13:  09      0000160512   0000180991   0000020480   param
14:  10      0000180992   0000208895   0000027904   efs
15:  11      0000208896   0000215039   0000006144   modemst1
16:  12      0000215040   0000221183   0000006144   modemst2
17:  13      0000221184   0003293183   0003072000   system
18:  14      0003293184   0028958719   0025665536   userdata
19:  15      0028958720   0028975103   0000016384   persist
20:  16      0028975104   0030695423   0001720320   cache
21:  17      0030695424   0030715903   0000020480   recovery
22:  18      0030715904   0030736383   0000020480   fota
23:  19      0030736384   0030748671   0000012288   backup
24:  20      0030748672   0030754815   0000006144   fsg
25:  21      0030754816   0030754831   0000000016   ssd
26:  22      0030754832   0030765071   0000010240   grow
27:  -----   0030765072   0030777343   0000012272   Unallocated

$ sudo mount -o ro,loop,offset=$((3293184*512)) $image /mnt
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$

Note	I’m i’ve created a link from image to the original raw device image to keep commands simple.

So, what happened here? I used the Sleuthkit mmls tool to read the partition table. I located the partition of interest - userdata - and tried to mount it read-only. I did not specify a file system type but instead let mount auto-magically determine it. I used the mount options of ro (read-only), loop (to create loopback device), and provided the offset to the partition. Since the offset required by mount is in bytes, I used shell math to translate the sector offset provided in the mmls output to bytes. But, in the end, mount did not appear to recognize the partition.

What do we do in such situation? We could use the mount -v verbose flag to try to determine what’s wrong.

BASH

$ sudo mount -v -o ro,loop,offset=$((3293184*512)) image /mnt
mount: enabling autoclear loopdev flag
mount: going to use the loop device /dev/loop0
mount: you didn't specify a filesystem type for /dev/loop0
       I will try type ext4
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
$

In this case, verbose output is not much help other than showing that mount attempted to use ext4 as the file system type. And, though ext4 is what I expected for the partition, too, maybe it is wrong. Short of a hex editor, how can we check a partition type in a disk image?

The file command is a well-known tool for providing file types by reading the file magic (the file’s hexadecimal signature). But did you know it will tell you partition types as well?

BASH

$ img_cat image -s 3293184 | file -
/dev/stdin: Linux rev 1.0 ext4 filesystem data,
UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery)
(extents) (large files)
$

The img_cat tool is another member of the Sleuthkit tool chest. It exports data from an image to stdout in blocks. Here we provide the starting sector offset to the userdata partition and pipe it to the file command. The hyphen following the file command to tells file to use stdout as input rather than a file.

What did we learn here? Well, though there is plenty of information, two values are of particular interest. First, the file system is in fact formatted ext4. Second, It appears that the file system journal is damaged. There is the smoking gun for our mounting problem.

So, we need a way to fix the journal so we can mount the partition, but we must not alter the original image. We have a few options here: * make a copy of the image * make a copy of just the partition Initially, making copies doesn’t sound too bad. After all, we’re only talking about 16GB for the image or 12GB for the partition. But what if this were a 250gb image, or larger? That sounds less palatable. Further, either action could consume a lot of resources in the form of drive space, processing power and time.

Important Note

Reader Carlos (with credit to Hal Pomerantz) correctly points out that the dirty journal issue can be avoid in the mount command by passing the noload option. In fact, according to the mount man page, noload is a good option to invoke whenever mounting ext3/4 read only to ensure no writes with dirty filesystems.

In our example, the command would be: sudo mount -o ro,noload,loop,offset=$3293184*512 image /mnt

This fact does not invalidate the rest of this discussion. You file system may need more significat repairs, such a repairing a partition table, and these repairs can still be effected with the technique that follows. As always, pick the path the meets the needs of your investigation.

Xmount to the Rescue

What if there was a way to fix the journal without taking the time and resources mentioned above? Enter xmount. Just as xmount can create a cache file to capture changes in an OS running from an image, it can capture changes when repairing a file system. We can quickly mount and repair the file system and leave the original image none-the-worse for wear. And because the blessings of the fuse file system driver on which xmount is built, we’ve only consumed new disk space for the xmount cache compared to full disk images and partitions.

BASH

$ md5sum image
15a9134d72a5590cf6589e9c9702a5ba  image
$

We start with an MD5 baseline of the image file. We’ll use this to determine if xmount allows any changes to the image.

BASH

$ sudo xmount --in dd  --out dd -o ro,allow_other \
--cache image.cache image /mnt
$ ls /mnt
image.dd  image.info
$ mount
...
xmount on /mnt type fuse.xmount (rw,nosuid,nodev,allow_other)
$

We use xmount to create a virtual image from our original image file. The --in option specifies the format of the original image. The input format can be Expert Witnes Format (ewf), Advanced Forensic Format (AFF) or raw (dd). The --out option specifies the format of the virtual image and can be raw (dd) or any one of the following virtual machine formats: vdi, vhd, or vdmk. The fuse -o allow_other option gives access to the virtual file system to all users (not just root). The final option --cache specifies the file to use for disk caching (image.cache). Then, much like the mount command, we specify the input file (image) and the mount point (/mnt).

The result of out xmount command is a virtual disk file being created in the /mnt folder that is accessible to normal users. Forget this option and you’ll have issues with listing the /mnt directory as a normal user. The name of the virtual disk image is the input file name appended with the format type. Thus, "image" became "image.dd". In the /mnt folder is also an .info file with image information.

The virtual image doesn’t consume real disk space. It is mounted read-write because we are going to fix the journal, but don’t fret, by passing the --cache image.cache option to xmount, we told xmount to capture the changes in the image.cache file. The image.cache file does not need to previously exist; xmount will create it for us.

The virtual disk image can be accessed just like the original image.

BASH

$ mmls /mnt/image.dd
GUID Partition Table (EFI)
Offset Sector: 0
Units are in 512-byte sectors

     Slot    Start        End          Length       Description
00:  Meta    0000000000   0000000000   0000000001   Safety Table
01:  -----   0000000000   0000008191   0000008192   Unallocated
02:  Meta    0000000001   0000000001   0000000001   GPT Header
03:  Meta    0000000002   0000000033   0000000032   Partition Table
04:  00      0000008192   0000131071   0000122880   modem
05:  01      0000131072   0000131327   0000000256   sbl1
06:  02      0000131328   0000131839   0000000512   sbl2
07:  03      0000131840   0000132863   0000001024   sbl3
08:  04      0000132864   0000136959   0000004096   aboot
09:  05      0000136960   0000137983   0000001024   rpm
10:  06      0000137984   0000158463   0000020480   boot
11:  07      0000158464   0000159487   0000001024   tz
12:  08      0000159488   0000160511   0000001024   pad
13:  09      0000160512   0000180991   0000020480   param
14:  10      0000180992   0000208895   0000027904   efs
15:  11      0000208896   0000215039   0000006144   modemst1
16:  12      0000215040   0000221183   0000006144   modemst2
17:  13      0000221184   0003293183   0003072000   system
18:  14      0003293184   0028958719   0025665536   userdata
19:  15      0028958720   0028975103   0000016384   persist
20:  16      0028975104   0030695423   0001720320   cache
21:  17      0030695424   0030715903   0000020480   recovery
22:  18      0030715904   0030736383   0000020480   fota
23:  19      0030736384   0030748671   0000012288   backup
24:  20      0030748672   0030754815   0000006144   fsg
25:  21      0030754816   0030754831   0000000016   ssd
26:  22      0030754832   0030765071   0000010240   grow
27:  -----   0030765072   0030777343   0000012272   Unallocated
$

We can automatically repair the journal by mounting the /userdata partition read-write. Once its repaired and mounted, we can remount as read-only for analysis.

BASH

$ mkdir mnt
$ sudo mount -o loop,offset=$((3293184*512)) /mnt/image.dd mnt
$ mount
...
xmount on /mnt type fuse.xmount (rw,nosuid,nodev,allow_other)
$ sudo mount -o remount,ro mnt
$ mount
...
/mnt/image.dd on /home/user/mnt type ext4 (ro)
$ ls -S mnt/
data
dalvik-cache
smart_stay.dmc
anr
app
app-asec
app-private
audio
backup
BackupPlus
bluetooth
bms
clipboard
dontpanic
drm
fota
fota_test
local
log
lost+found
media
misc
property
...
$

First, I created a new directory in the current working directory called "mnt". Don’t confuse this with the /mnt directory where the virtual disk image is located. Like before, we used the mount command to create a loopback device and address the partition by offset. This time, we did not set the read-only flag, and we specified a new directory for the partion since we were using /mnt to host the virtual disk. This time, we succeeded in mounting the partition, and then we immediately remounted read-only to avoid making further changes.

Wrapping Up

In summary, we tried to mount a partition in our original disk image, but it failed. We determined the partition had a damaged journal, so we created a virtual disk image to effect repairs. Then we mounted the repaired partition and listed the root directory. But did we do no harm?

BASH

$ md5sum image /mnt/image.dd
15a9134d72a5590cf6589e9c9702a5ba  image
2e16cbbeefc9e33bc754b47d2f8a4da0  /mnt/image.dd
$

The original image hash remains unchanged. The xmounted image shows its been changed. So xmount has done its job, protecting the original image while allowing us to repair the partition in the virtual image!

Oh, and what about that cache file? How much real space did we use when we created and repaired the virtual image?

BASH

$ ls -lh image.cache
-rw-r--r-- 1 root root 40M Mar 21 15:51 image.cache
$

Yep, a whole 40mb was used to repair the journal and mount a 12GB partition. That’s not too shabby, and you didn’t wait for a long copy operation of halve your storage capacity!

Xmount Cache Caveats

A quick sidebar on the xmount cache file. The cache can be reused, meaning that the changes from the last session are brought forward to the next session. In plain terms, if we unmount the userdata/ partition and then the virutal image, but later remount the image while pointing to the cache file we previous created with the --cache option, the file system will remain repaired. If we want to start afresh, we would use the overwrite cache option, or --owcache. Finally, we don’t need to specify a cache at all if changes are not necessary, and, in fact, this is the manner I usually employ xmount.

---

Last updated 2013-03-22 08:10:11 PDT

Cashing in on the Google Chrome Cache

The Google Chrome cache represents a challenge to forensic investigators. If the extent of your examination has been to open the cache folder and view the files in a file browser, you are likely missing a lot of content.

For starters, files stored in the cache are renamed from their original names on the web-server. Next, text elements (like HTML, JSON, etc.) are zlib compressed. Finally, files smaller than 16384 bytes (16k) are stored in block files which are container files that hold many smaller files. The meta-data about the cache files are stored in these container files, too, and its all mapped by a binary index file.

So, while its easy enough to point a file browser or image viewer at the cache directory and see some recognizable data structures, making sense of all that’s there can be more challenging. In the remainder of this discussion, I’ll attempt to give you more insight into the Google Chrome cache. This should be of interest to disk and mobile forensicators alike, as the structure is the same whether you are examining a desktop computer or a mobile device such as an Android phone or tablet.

Cache Structure

All the files in the Google Chrome cache are stored in a single folder called cache. The cache consists of at least five files: and index file and four data files known as block files. As I stated above, downloaded files are stored in one of the block files or directly to the cache directory, and the index keeps track of the transaction and storage location.

A cache can consist of only the five mentioned files (named index, data_0, data_1, data_2, and data_3) if all the data files in the cache are smaller than 16k. Larger files are stored outside the block files. Go ahead, check your cache if you don’t believe me… I’ll wait. In case you don’t have one handy, here’s as truncated file listing from a recent Android exam I conducted, sorted by size.

Output of ls -lSr

-rw-r--r-- 1 user user   16519 Feb 20 13:06 f_00008a
-rw-r--r-- 1 user user   16566 Feb 20 13:06 f_0000cf
-rw-r--r-- 1 user user   16604 Feb 20 13:06 f_0000cc
-rw-r--r-- 1 user user   16944 Feb 20 13:06 f_0000cd
...
-rw-r--r-- 1 user user   70659 Feb 20 13:06 f_0000f7
-rw-r--r-- 1 user user   73804 Feb 20 13:06 f_00008f
-rw-r--r-- 1 user user   74434 Feb 20 13:06 f_0000df
...
-rw-r--r-- 1 user user   81920 Feb 20 13:06 data_0
...
-rw-r--r-- 1 user user  262512 Feb 20 13:06 index
-rw-r--r-- 1 user user 1581056 Feb 20 13:06 data_1
-rw-r--r-- 1 user user 2105344 Feb 20 13:06 data_2
-rw-r--r-- 1 user user 4202496 Feb 20 13:06 data_3

Warning

When one or more of the five base files get corrupted or deleted, the entire set gets recreated. I experimented by deleting a data block file and restarting Chrome. On browser restart, the entire cache was deleted and new base files were created.

You might be wondering about the the four data blocks. How are they distinguished? The answer is: size; not size of the block-files themselves, but size of the internal data blocks. Each block file is defined to hold data in blocks, much like a file system. And like different sized file systems can be defined with different sized blocks, so the cache data block-files are defined with different block sizes, and data can be allocated to no more than four blocks at a time before being considered too large for that block-file.

Table 1. Default Data Block-file sizes

File	Block SZ	Max Data SZ
data_0	36b	rankings
data_1	256b	1k
data_2	1k	4k
data_3	4k	16K

Note	When a data block-file reaches maximum capacity (each file is only allowed to hold a defined number of objects) a new data block-file is created and pointed to in the previous data block-file header.

Cache addresses

All cached objects have an address. The address is a 32-bit integer that describes where the data is stored. Meta-data about the object is stored, too, and includes:

• HTTP Headers
  
• Request Data
  
• Entry name (Key)
  
• Other auxiliary information (e.g., rankings)
  

Examples of cache addresses:

0x00000000: not initialized
0x8000002A: external file f_0002A
0xA0010003: block-file number 1 (data_1), initial block number 3, 1 block of length.

Important

The addresses above are ordered as they read, but on disk you will find them in little-endian format, e.g, the external file appears on disk as 0x2A000080

Cache addresses are interpreted at the bit level. That means we have to convert the 32-bit integer into bits and evaluate to understand the address. The first 4 bits are the header, which consist of the initialized bit followed three file type bits.

Table 2. File Types

BINARY	INTEGER	INTERPRETATION
000	0	separate file on disk
001	1	rankings block-file
010	2	256b block-file
011	3	1k block-file
100	4	4k block-file

The remaining 28-bits are interpreted according to the file type:

Table 3. Separate File

Init	File Type	File #
1	000	1111111111111111111111111111

Table 4. Block File

Init	File Type	Reserved	Contiguous Blocks	Block File	Block#
1	001	00	11	00000000	1111111111111111

Lets take a look at the last two cache addresses above:

External File Address

0x8000002A, interpreted as a 32-bit integer, is 2147483690. In binary, it is 10000000000000000000000000101010. We interpret it as follows:

Table 5. Binary Interpretation

	Init	File Type	File #
Binary	1	000	0000000000000000000000101010
Integer	1	0	42 (0x2A)

Block File Address

0x080001A0, interpreted as a 32-bit integer, is 2684420099. In binary, it is 10100000000000010000000000000011. We interpret it as follows:

Table 6. Binary Interpretation

	Init	File Type	Reserved	Contiguous Blocks	Block File	Block#
Binary	1	010	00	00	00000001	0000000000000011
Integer	1	2	0	0	1	3

Note	The odd man out here is Contiguous blocks, which weighs in at zero but appears to be interpreted as 1 block as based on Google documentation.

There is a lot more to discuss here, but this is a good start. You now know that the cache index holds a map of cached browser data, and that the data_# files can contain cached web object you’ve may have been missing. I’ll cover more on following the cache map and extracting the file content of the data-blocks in a future article (or two)!

---

Last updated 2013-02-24 16:39:36 PST

Android Messaging: Is Android Getting Religious?

"Cleanliness is next to Godliness," it is often said. And if you believe that, then you might think the Android operating system is seeking after the divine when it comes to its messaging service. Why do I say that? Because in my quest for a thorough understanding of SQLite databases, I discovered that the mmssms.db, Android’s built-in messaging database, has the auto-vacuum option enabled! And in Full-mode at that!

SQLite Vacuum

In SQLite, Vacuum is an operation that rebuilds the entire database. Frequent updates, deletions and insertions can leave the database file fragmented. Vacuum reduces the size of fragmented databases by copying the active records to a temporary file and then overwriting the original database file. During this process, it uses the rollback journal or write-ahead log as it would for any database transaction.

SQLite has two auto-vacuum modes, full and incremental. The auto-vacuum mode can only be set when the database is created. The setting is stored in the database header (the first 100 bytes of the database file), at file offset 52. If the 32-bit, big-endian integer at offset is non-zero, it represents the address (page number) of the largest root b-tree page. For this discussion, the significance of the non-zero value is that database auto-vacuum is enabled.

The 32-bit, big-endian File offset 64 indicates the auto-vacuum mode. An non-zero value means the database is set for incremental vacuum mode, while a zero value means full mode.

Figure 1. SQLite Database Header

Note	Don’t be fooled by shortcutting your analysis by jumping straight to offset 64 to check the value. A zero value and offset 64 coupled with a zero value at offset 52 means auto-vacuum is not enabled!

SQLite Structure

Before we can really understand the SQLite vacuum operation, we have to first understand a little bit about how SQLite manages its data. SQLite organizes itself into pages. The page sizes usually match the underlying file system block size and can be determined definitively by the 16-bit, big-endian integer located at file offset 16. Each page has a single purpose and can be any one of the following types:

• Lock-byte
  
• Freelist
  
  • trunk
    
  • leaf
    
• B-tree
  
  • table interior
    
  • table leaf
    
  • index interior
    
  • index leaf
    
• Payload overflow
  
• Pointer map
  

For this discussion, we need to know about about the freelist pages and B-Tree table pages.

Freelist Pages

When data is deleted, or dropped, from the SQLite database, the database file does not get smaller (absent a vacuum operation). The database notes the location of the free space and reuses it as needed. The freelist contains the addresses, by page number, of full pages no longer being used to store data. The number of freelist pages in the database is store in the database file header as a 32-bit, big-endian integer at file offset 36.

Freelist trunk pages store the addresses—by page number, not offset—to the next trunk page, if any, and to freelist leaf pages. The freelist leaf pages are the pages that once stored data, that is, they were once B-Tree pages.

B-Tree Pages

Table records and structures are stored in B-Tree pages. B-Tree pages have headers that describe the data in the page:

• Byte offset to first freeblock (unallocated space between the records)
  
  • Free blocks are chained together, each one pointing to the next
    
• Number of cells on the page
  
  • B-Tree table leaf pages contain cells with table data
    
• Offset to the first cell on the page
  
• Number of fragmented free bytes
  
  • May not exceed 60 bytes
    

A cell pointer array follows immediately after the B-Tree page header. The array is a list of offsets to the allocated cells on the page. Cells are self describing, using integers to describe things like the cell length, the unique record index number (ROWID), and the cell payload content (by means of a record header). Not all B-Tree pages contain the table data that is the usual subject of an examination, but those that do can be identified by the page header.

The take away here is that it is B-Tree pages that contain table data. B-Tree pages can contain both allocated and unallocated space, and become fragmented when one record is dropped from the midst of other records. All records may be deleted from a B-Tree table leaf page making it subject to becoming a Freelist page.

Note	A SQLite database may reorganize, or defragment a page so there are no freeblocks or byte fragments (groups of three or less bytes), packing all the allocated cells at the end of the page. This is an internal housekeeping function independent of the vacuum function.

A Tale of Two Modes

As I already stated, auto-vacuum comes in two flavors: Full and Incremental. So, what is the difference and how does it affect our examinations?

Auto-Vacuum: Full Mode

In full auto-vacuum mode, every transaction commit to the database causes the pages in the freelist to be moved to the end of the database, and the database is truncated to remove the pages. It is important to distinguish that only the freelist pages are removed, not the fragmented B-Tree pages. Also, Full auto-vacuum does not cause B-Tree page defragmentation to occur.

Auto-Vacuum: Incremental Mode

In Incremental auto-vacuum mode, vacuuming does not occur with every commit. Instead, the database programatically receives a command to remove N pages from the freelist. The pages are moved to the end of the database, and the database is truncated. The page references are removed from the free list. If there are fewer pages in the list than required by the command, all the freepages are moved and truncated.

So What’s the Big Deal?

I started this discussion by noting that I had discovered that the Android mmssms.db was set to full auto-vacuum mode. This means that every commit to the database could cause dropped records in freepages to be moved to the end of the database and dropped off a cliff. Tools designed to recover dropped records from logigal SQLite databases won’t recover the records because they are no longer part of the database! And logical file extraction tools won’t recover the deleted pages, either.

Think of it like this: A drug dealer is seen conducting a transaction and flees when approached by police. He momentarily escapes, and takes the opportunity to delete all his text messages should he be captured. Sure enough, the good guys find him. While he’s being pat down, one of his customers texts the internationally recognized "do you have any drugs?" abbreviation: "Wuz up?"

Wuz up? You just helped the drug dealer remove all dropped records from his messing database, that’s WUZ UP!

---

Last updated 2013-01-26 17:04:53 PST