Tuesday, September 29, 2015

Compression and Android Gmail

Every registered Android mobile device has an associated Google account. Google accounts usually mean Gmail. And, for investigators interested in the Gmail content stored on Androids, that content can be found in the /data/com.google.android.gm/databases directory in a database named in the following format:


The database contains 23 tables (at least at the time of this writing), the most interesting of which is messages.

The messages table has 41 fields (or columns). To obtain the basic email content (say, for keyword searching), an investigator would likely want to export the sender’s and receiver’s addresses, the date sent or received, and the subject line, and the message body, at the very least. There is plenty more to be gleaned from the database, but your investigation will dictate the investigative needs.

Automated tools do not provide the full weath of data to be found in the mailstore database. It is always a good idea to become familiar with the database schema to learn the full potential for your investigation.

The Big Squeeze

If you have experience searching SQLite databases, you might be thinking, "Why go to the trouble of exporting messages from the database? SQLite strings are usually UTF-8, so I can just search the database with regular expressions or plain keywords." Well, there is a catch when it comes to email content in the Gmail mailstore database: zlib compression.

Short length message bodies are written to the body field in the messages table as a plain text string. In a recent exam, the longest message I found in this field in a recent exam was 98 bytes, however. Longer message bodies are compressed using the zlib algorythm and stored in the bodyCompressed field. While SQLite supports compressed databases, it has no function to decompress fields within databases. Instead, it stores such data as a blob type, and it is up to the database user to decompress the data.

The SQLite blob type is sort of a catch-all for any type or data. Data is stored in the format in which it was input.

Extracting Messages

Python is a good option for exporting messages from the Gmail messagestore database. It can both open and query databases, and it can decompress the long message bodies.

Exporting Gmail Messages
import sqlite3
import zlib

# open and query the database
conn = sqlite3.connect('messagestore.db') # database name abbreviated
c = conn.cursor
c.execute("select _id, fromAddress, datetime(dateSentMS/1000,
    'unixepoch', 'localtime'), datetime(dateReceivedMS/1000,
    'unixepoch', 'localtime'), case when body not Null then body
    else bodycompressed end from messages ")
rows = c.fetchall()

# interate through the rows and decompress the long messages
for row in rows:
    id, _from, sent, recv, body = row
        body = zlib.decompress(body)
    print('{}|{}|{}|{}|{}'.format(id, _from, sent, recv, body))
The final line can be adapted to your own needs, i.e., writing the content to a new file or database, or use python regular expressions to search the content, etc.

Some Explaination

The example above is just that: an example. It is intended, like all my posts, to remind me how to process the data and demonstrate how just a few lines of python can be leveraged to extract data. The script could have been shorter, but it would have come at the cost of clarity. That said, there is still some explanation to be had:

The SQLite query in the c.execute method might need some dissecting for you to understand what I did.

    datetime(dateSentMS/1000, 'unixepoch',   'localtime'),
    datetime(dateReceivedMS/1000, 'unixepoch', 'localtime'),
        when body not Null
        then body
        else bodyCompressed
from messages

The dateSentMS and dateReceivedMS fields are recorded in milliseconds since 1/1/1970 (Unix epoch). I let SQLite do the date converstion for me, rather and doing it python, and I converted from Unix epoch to local time. The case statement pulls a little trickery to select the body field or bodyCompressed field. Basically, the row’s body field is checked to see if it is populated. If so, it is returned. If not, the contents of the bodyCompressed field are returned.

In the body decompression section of the script, the contents of the each row are assigned to variables id, _from, sent, recv and body. The try/except clause attempts attempts to decompress the body. If it fails, as it will on the short message bodies, it just uses the contents of the body variable as is. Finally, the row is printed in a pipe-delimited fashion.

Tuesday, August 25, 2015

Android SDK on 64-bit Linux

I commonly use adb and fastboot to access Android devices.  Ubuntu has packages for those tools making installation easy:
$ sudo apt-get install android-tools-adb android-tools-fastboot
But, in recent months, I have encountered instances where the adb and fastboot binaries in the Ubuntu repository are out of date for the device with which I am attempting to connect.
$ apt-cache show android-tools-adb
Package: android-tools-adb
Priority: extra
Section: universe/devel
Installed-Size: 227
Maintainer: Ubuntu Developers
Original-Maintainer: Laszlo Boszormenyi (GCS)
Architecture: amd64
Source: android-tools
Version: 4.2.2+git20130218-3ubuntu41
Depends: libc6 (>= 2.15), libssl1.0.0 (>= 1.0.0), zlib1g (>= 1:1.1.4)
Filename: pool/universe/a/android-tools/android-tools-adb_4.2.2+git20130218-3ubuntu41_amd64.deb

$ adb version
Android Debug Bridge version 1.0.31

Android Software Development Kit

There is another way to install adb: the Android Software Development kit.  There is an issue, though.  The development kit software is 32-bit, and modern computer systems run 64-bit operating systems.  So, one needs a way to install 32-bit libraries in 64-bit Linux to support 32-bit applications.

The Android SDK can be found here. However, before you start downloading the Android Studio displayed prominently at the top of the page, consider that Studio is large collection of software for developing Android applications.  Forensically speaking, it is overkill, and fortunately, there is an alternative download that servers our purpose better: SDK Tools Only.

I prefer to install tools not found in the Ubuntu repository in the /opt directory.  After I download the SDK, I decompress the archive:
$ sudo tar xf Downloads/android-sdk_r24.3.4-linux.tgz -C /opt
The next step in installation is to execute the Android SDK Manager.  Note: You will need java installed (java-common package) to run the application.
$ /opt/android-sdk-linux/tools/android 
The only component needed to successfully run adb and fastboot are the Android SDK tools (already installed) and Android SDK Platform-tools.  Check the later and go through the installation process.  Exit the manager when you are done.

Installing 32-bit Support in 64-bit Ubuntu

The adb and fastboot executable binaries are located in the platform-tools folder of the SDK.  However, if you try to execute the tools, you will get and error stating the that the command is not found, even though you see the tools in the directory, with proper permissions for execution, too! 
$ cd /opt/android-sdk-linux/platform-tools
$ ./adb version
bash: ./adb: No such file or directory
$ ls -l adb
-rwxr-xr-x 1 root root 1221540 Aug 25 13:05 adb
Confusing?  You bet. The issue is that you are trying to execute a 32-bit application in a 64-bit-only operating system.  There is no 32-bit support installed in Ubuntu by default.  But we can solve that:
$ sudo dpkg --add-architecture i386
$ sudo apt-get update
The specific libraries to support the SDK are installed as follows:
$ sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
$ ./adb version
Android Debug Bridge version 1.0.32
Revision 57224c5cff69-android
And, voila!  The latest SDK binary tools installed.

Easy Access

There are two simple ways to gain access to the SDK platform tools: link files or path modification.  Link files are placed in a location that is part of your system path already, and path modification places the platform tools directory in your path.  Which is best?  Welcome to freedom, friend, it's your choice.

Link files

Link files are found in all modern operating systems, so I won't bother explaining them.  First, you need to know what directories are in your path so you know where to put the links:
$ echo $PATH
Most packages you install place their executable binaries in /usr/bin and /usr/sbin, depending on the need for administrative privileges.  Most software built and installed by the user are placed in the /usr/local/bin and /usr/locals/sbin folders.  Since we are installing the software, and it does not require admin rights to execute, /usr/local/bin is a good choice.

To create link files in Linux (symlinks are preferred for this type of operation), do the following:
$ ln -s /opt/android-sdk-linux/platform-tools/adb /usr/local/bin
$ ln -s /opt/android-sdk-linux/platform-tools/fastboot /usr/local/bin
Note: you will have a lot more success with link files if you get in the habit of using absolute paths.  There is a place for relative paths, but this is not it.

Modifying the PATH

The PATH system variable maintains the list of directories searched by the system for commands.  When you, as a user, type a command without it's full path, the list of directories in the PATH variable are queried for the command.  If the command is not located, an error results.  To place the adb and fastboot binaries into the path, we need to add the folder containing them to the PATH variable.

It is possible to modify the path, both temporarily and permanently.  For example:
$ echo $PATH
$ which
$ PATH=/opt/android-sdk-linux/platform-tools

$ echo $PATH
/opt/android-sdk-linux/platform-tools$ which
Command 'which' is available in the following places
 * /bin/which
 * /usr/bin/which
The command could not be located because '/bin:/usr/bin' is not included in the PATH environment variable.
which: command not found
The problem about illustrates that we have to be careful setting the path.  By setting the path to /opt/android-sdk-linux/platform-tools, we removed the directories previously stored there.  The method used, however, is only a temporary change to the PATH variable.  The path can be reset by restarting the terminal window, or by resetting the PATH variable to its original contents:
$ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
To make the change permanent, and to append to but not replace the contents of the PATH variable, we should add the following to our ~/.bashrc file (if your installation does not have the file, create it):
export PATH="$PATH:/opt/android-sdk-linux/platform-tools"
It is not necessary to have quotes in the statement above if there are no spaces in the path, but it is the safest way to execute the statement.  By placing this statement in .bashrc, the PATH variable will be expanded, have the platform-tools folder appended, and then the value will be reassigned to the PATH variable.  This will happen each time a new terminal session starts.

Monday, August 24, 2015

Riffbox and Windows 10

I decided to bite the bullet and try out Windows 10. I wanted to learn the new operating system and determine if I could run specific software/hardware combinations under the new Windows that I had been running in Windows 7, specifically Riffbox. I happy to report, that after some trial and error, I have determined a reliable way of running riffbox under Windows 10. This should work equally well under Windows 8, as the issue is the same.

Hey, What's Your Sign? 

Windows 8 and now 10 both required a driver be digitally signed by a vendor known to Microsoft of it will flat refuse to install the driver. Under Windows 7, you had the option to continue even if the driver was not digitally signed, but not so for Windows 8/10.  The only solution is to reboot the computer into "Disable device driver signing detection" mode.

Disabling Driver Signing

  1. Right-click the start menu, select Shut down or sign out.
  2. Click Restart while holding the Shift key.  The boot option will appear.
  3. Select Troubleshoot > Advanced Options > Startup Settings > Restart.
  4. In the Startup Settings menu, press F7 to disable driver signing.
The PC will now perform a one-time boot with driver signing disabled.  In this mode, it is possible to install the Riff Box software.  If the driver's do not automatically install, navigate to C:\Program Files (x86)\RIFF Box JTAG Manager\Drivers\64Bit directory and run the dpinst.exe program as Administrator.  Use Windows 7 compatibility mode if necessary.  You will be warned that the three drivers are not digitally signed, but you will have the option to install anyway.

After you install the drivers, plug in Riff Box and launch JTAG Manager.  

Common Issues

If you plug the riffbox into your Windows 10 system and the green light continually flashes, riffbox has not been properly recognized.  An easy fix for this is to right-click the Start menu icon and select 'Device Manager'.  Under 'Ports', you should find the 'RIFF BOX Control Port (COM #)' entry.

  1. Right click the entry and select Update Driver Software.
  2. Select Browse my computer for driver software.
  3. Select Let me pick from a list of device drivers on my computer.
  4. With the Show compatible hardware box checked, select RIFF BOX Control Port.
  5. Click the Next button to install the driver.  The Riff Box will now be recognized as a 'RIFF BOX Control Port.'
Note: Follow this procedure, too, if instead of the Riffbox entry, you see a generic 'USB Serial Device.'

Reboot Reincarnation

Rebooting reinstates the device driver signature requirement in Windows 8/10. Riff Box will not be recognized by JTAG Manager in normal boot mode because the unsigned drivers will not load.  Even if you follow the trouble shooting listed in the Common Issues section above, JTAG Manager will not detect Riff Box.  The only solution (to date) is to reboot into Disabled Signature Enforcement Mode.

That said, there are ways to force disabled signature enforcement mode on each boot, but do they work?

Permanently Disabling Driver Signature Enforcement

You will read that you can use the bcdedit command line tool to disable driver signing on each boot automatically.  The tool modifies the boot configuration data store which replaced the boot.ini file of windows 7 and earlier.  However, while this process will allow driver installation, it still does not allow the drivers to load when Riff Box is connected to the PC.  I include the instructions below to show you what I did in testing.

To disable driver signing mode on boot (and start in Test Mode):
  1. Right-click on the Start menu icon, select Command Prompt (admin).
  2. Run the command:
    bcdedit /set loadoptions DISABLE_INTEGRITY_CHECKS
  3. Run the command:
    bcdedit /set TESTSIGNING ON
  4. Reboot.

To reenable driver signing (and exit Test Mode):
  1. Right-click on the Start menu icon, select Command Prompt (admin).
  2. Run the command:
    bcdedit /deletevalue loadoption
  3. Run the command:
    bcdedit /set TESTSIGNING OFF
  4. Reboot
Another possible permanent solution would be to disable driver signature enforcement in the Group Policy Editor.  You will only have this tool if are running Windows Pro and higher, however.  I did not have Windows 10 pro at the time of this writing, so I have not tested the following procedure:
  1. Right-click the Start menu icon and select Run.
  2. Enter gpedit.msc and press Enter.
  3. In the editor, select User Configuration > Administrative Templates > System > Driver Installation.
  4. Double-click Code signing for device drivers.
  5. Click the Enable radio button and then select Ignore in the Options drop down.
  6. Apply the changes and reboot.
If you have success with this second method, please post a comment so others will know the solution.