Image Manipulation With *nix Commands
I decided to follow up the *nux Live Acquisition Techniques post from a few days ago with a demonstration of image or file manipulation using DD and SPLIT. This will help me get it all straight in my head while documenting it for prosterity, yours and mine. Certainly there are other tools to do this, but knowing the basics is key to being good at anything.
Here is the scenario.
- Copy the swap partition using DD. Get one big image and start manipulating it. I could just copy swap every time, but as it continuously changes I will have a problem verifying some of the commands or techniques were successful.
- Get a hash of the image.
- Chop the image of the swap partition into smaller pieces using DD and the SPLIT.
- Pull the chunks back together using CAT.
- Verify that CAT successfully rebuild the image by checking the hash.
- Delete the last two chucks to simulate that the original copy, if it had been run using SPLIT, failed at some point for any reason.
- Chop the swap partition image again but this time skip the good chunks and only re-run the bad chunks. This could end up saving A LOT of time in the imaging process. Which, in turn, saves the customer money.
- Pull the new chunks together using CAT.
- Verify that the new chucks can be used to create a valid image by checking the hash.
- Stop SCRIPT, write post, grab beer (should have remembered to do this at the beginning).
In the following output I have added my notes while also bolding interesting pieces of output. I did have a little glitch when trying to skip the good chunks, so be sure to watch for it and what I did to correct the problem.
user@cutsec:~$ script dd_stuff.txt <- Start SCRIPT so I don’t have to cut and paste
Script started, file is dd_stuff.txt
user@cutsec:~$ sudo -i <- Being root makes imaging swap easier. Plus, I like power!!
[sudo] password for user:
root@cutsec:~# cd /opt/Test
root@cutsec:/opt/Test ls
root@cutsec:/opt/Test dd if=/dev/sda5 of=./swap_orig.dd <- Copy an Image of the Swap Partition
2104452+0 records in
2104452+0 records out
1077479424 bytes (1.1 GB) copied, 96.2791 s, 11.2 MB/s
root@cutsec:/opt/Test ls -al
total 1053268
drwxr-xr-x 2 root root 4096 2008-10-01 22:19 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
root@cutsec:/opt/Test md5sum swap_orig.dd <- Hash it for verification
9a63cfbea3005551f4021aac7c287997 swap_orig.dd
root@cutsec:/opt/Test dd if=swap_orig.dd | split -d -b 200m - swap_split.dd. <- Split the Partition Image into small chunks
2104452+0 records in
2104452+0 records out
1077479424 bytes (1.1 GB) copied, 105.6389 s, 10.2 MB/s
root@cutsec:/opt/Test ls -al
total 2106548
drwxr-xr-x 2 root root 4096 2008-10-01 22:26 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.03
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.04
-rw-r–r– 1 root root 28903424 2008-10-01 22:26 swap_split.dd.05
root@cutsec:/opt/Test cat swap_split.dd.0* >>./swap_cat.dd <- Pull them back together using the CAT command
root@cutsec:/opt/Test ls -al
total 3159808
drwxr-xr-x 2 root root 4096 2008-10-01 22:28 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.03
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.04
-rw-r–r– 1 root root 28903424 2008-10-01 22:26 swap_split.dd.05
root@cutsec:/opt/Test md5sum swap_cat.dd <- Verify CAT worked properly
9a63cfbea3005551f4021aac7c287997 swap_cat.dd
root@cutsec:/opt/Test rm swap_split.dd.03 swap_split.dd.04 swap_split.dd.05 <- Remove chunks to simulate DD or SPLIT command failure
root@cutsec:/opt/Test ls -al
total 2721540
drwxr-xr-x 2 root root 4096 2008-10-01 22:33 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
root@cutsec:/opt/Test dd if=./swap_orig.dd bs=200MB skip=3 | split -d -b 200m - swap_new_split.dd. <- Start copying by Skipping first 3 chunks of 200MB
2+1 records in
2+1 records out
477479424 bytes (477 MB) copied, 38.3604 s, 12.4 MB/s
root@cutsec:/opt/Test ls -al
total 3188300
drwxr-xr-x 2 root root 4096 2008-10-01 22:37 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:36 swap_new_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:37 swap_new_split.dd.01
-rw-r–r– 1 root root 58049024 2008-10-01 22:37 swap_new_split.dd.02 <- Note the size
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
root@cutsec:/opt/Test mv swap_new_split.dd.00 swap_split.dd.03
root@cutsec:/opt/Test mv swap_new_split.dd.01 swap_split.dd.04
root@cutsec:/opt/Test mv swap_new_split.dd.02 swap_split.dd.05
root@cutsec:/opt/Test ls -al
total 3188300
drwxr-xr-x 2 root root 4096 2008-10-01 22:40 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
-rw-r–r– 1 root root 209715200 2008-10-01 22:36 swap_split.dd.03
-rw-r–r– 1 root root 209715200 2008-10-01 22:37 swap_split.dd.04
-rw-r–r– 1 root root 58049024 2008-10-01 22:37 swap_split.dd.05
root@cutsec:/opt/Test cat swap_split.dd.0* >>./swap_cat2.dd <- Pull new chunks together
root@cutsec:/opt/Test md5sum swap_orig.dd swap_cat.dd swap_cat2.dd <- Verify success
9a63cfbea3005551f4021aac7c287997 swap_orig.dd
9a63cfbea3005551f4021aac7c287997 swap_cat.dd
eec1975aed363dbd2254262594577da7 swap_cat2.dd <- FAIL!!
root@cutsec:/opt/Test rm swap_split.dd.03 swap_split.dd.04 swap_split.dd.05 <- Remove bad chunks
root@cutsec:/opt/Test ls -al
total 3803292
drwxr-xr-x 2 root root 4096 2008-10-01 23:02 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1106625024 2008-10-01 22:42 swap_cat2.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
root@cutsec:/opt/Test dd if=./swap_orig.dd bs=200M skip=3 | split -d -b 200m - swap_new_split.dd. <- Try Skip again but this time use the proper bs, MB = 1024*1024 but M = 1000*1000
2+1 records in
2+1 records out
448333824 bytes (448 MB) copied, 39.6093 s, 11.3 MB/s
root@cutsec:/opt/Test ls -al
total 4241560
drwxr-xr-x 2 root root 4096 2008-10-01 23:03 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1106625024 2008-10-01 22:42 swap_cat2.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 209715200 2008-10-01 23:03 swap_new_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 23:03 swap_new_split.dd.01
-rw-r–r– 1 root root 28903424 2008-10-01 23:03 swap_new_split.dd.02 <- Note the size, that looks better
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
root@cutsec:/opt/Test mv swap_new_split.dd.00 swap_split.dd.03
root@cutsec:/opt/Test mv swap_new_split.dd.01 swap_split.dd.04
root@cutsec:/opt/Test mv swap_new_split.dd.02 swap_split.dd.05
root@cutsec:/opt/Test ls -al
total 4241560
drwxr-xr-x 2 root root 4096 2008-10-01 23:05 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1106625024 2008-10-01 22:42 swap_cat2.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
-rw-r–r– 1 root root 209715200 2008-10-01 23:03 swap_split.dd.03
-rw-r–r– 1 root root 209715200 2008-10-01 23:03 swap_split.dd.04
-rw-r–r– 1 root root 28903424 2008-10-01 23:03 swap_split.dd.05
root@cutsec:/opt/Test rm swap_cat2.dd
root@cutsec:/opt/Test cat swap_split.dd.0* >>./swap_cat2.dd <- Pull new chunks together
root@cutsec:/opt/Test ls -al
total 4213068
drwxr-xr-x 2 root root 4096 2008-10-01 23:08 .
drwxr-xr-x 3 root root 4096 2008-10-01 11:46 ..
-rw-r–r– 1 root root 1077479424 2008-10-01 23:09 swap_cat2.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:30 swap_cat.dd
-rw-r–r– 1 root root 1077479424 2008-10-01 22:20 swap_orig.dd
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.00
-rw-r–r– 1 root root 209715200 2008-10-01 22:25 swap_split.dd.01
-rw-r–r– 1 root root 209715200 2008-10-01 22:26 swap_split.dd.02
-rw-r–r– 1 root root 209715200 2008-10-01 23:03 swap_split.dd.03
-rw-r–r– 1 root root 209715200 2008-10-01 23:03 swap_split.dd.04
-rw-r–r– 1 root root 28903424 2008-10-01 23:03 swap_split.dd.05
root@cutsec:/opt/Test md5sum swap_orig.dd swap_cat.dd swap_cat2.dd <- Verify success
9a63cfbea3005551f4021aac7c287997 swap_orig.dd
9a63cfbea3005551f4021aac7c287997 swap_cat.dd
9a63cfbea3005551f4021aac7c287997 swap_cat2.dd <- Success!!
root@cutsec:/opt/Test logout
user@cutsec:~$ exit
Script done, file is dd_stuff.txt
There you have it. The basics of using system commands to image a partition, chop it up, and pull it all back together. Hopefully this is useful in some capacity.
Go forth and do good things,
Don C. Weber
Help support my training and travel to security conferences. Get your SANS Training and GIAC Certifications through the Security Ripcord.
You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.









October 7th, 2008 at 9:03 pm
[...] Security Ripcord » Blog Archive » Image Manipulation With *nix Commands Great post on working with images in Unix (tags: image analysis unix linux) [...]