Give DD a progress bar

On 4. July 20134. July 2013 By Hans-Henry JakobsenIn Linux1 Comment

This is a short description on how you can get a progress bar to your dd command or other commands as well using the command pv.
pv is a shell pipeline element to meter data passing through and can be installed using aptitude in Ubuntu.

# aptitude install pv

As an example I create a empty file with zeroes with a block size of 1MByte and it is going to be 1GByte in total
# dd if=/dev/zero bs=1M count=1000 | pv -s 1G | dd of=/tmp/1G 1000+0 records in 165MB/s] [================================> ] 93% ETA 0:00:00 1000+0 records out 1048576000 bytes (1.0 GB) copied, 6.24504 s, 168 MB/s 1e+03MB 0:00:06 [ 160MB/s] [================================> ] 97% 2048000+0 records in 2048000+0 records out 1048576000 bytes (1.0 GB) copied, 7.90455 s, 133 MB/s
The file /tmp/1G is now filled with zeroes sizing 1GByte.

How to corrupt a disk partition using dd

On 14. November 201115. November 2011 By Hans-Henry JakobsenIn Linux, Scripting

I wanted to test corruption of a 41TB XFS filesystem and the dd command was the perfect tool for the job.
I used a block size of 512 bytes and wrote 10 blocks of data from device /dev/urandom to the partition /dev/sdb1.
The “Seek” option specifies how many blocks are skipped, counting 100 blocks from the beginning of the partiton, before writing the random/corrupted data.

# dd bs=512 count=10 seek=100 if=/dev/urandom of=/dev/sdb1

Caution!
This procedure WILL most likely corrupt your disk partition or files on the partition. Use it ONLY on test systems and with a valid backup!

Create a empty file using dd

On 13. June 200913. June 2012 By Hans-Henry JakobsenIn Linux

How to create a empty/null file with a specified size

# dd if=/dev/zero of=destinationfile.bin bs=1024k count=100

This command creates a 100MB (1024k * 100) sized file.

Backup partition table

On 19. March 200919. March 2009 By Hans-Henry JakobsenIn Linux

This post describes how you can backup your partition table for future recovery.

Backup

This example describes how you can backup the disk partition table on /dev/sda

dd if=/dev/sda of=sda.mbr bs=512 count=1

It’s also useful to keep a human readable copy of the disk layout for future reference

sudo fdisk -l > partitions.lst

Restore

This example shows how you can recover your file system using your partition table backup.

dd if=sda.mbr of=/dev/sda bs=512 count=1

This is a nice way to keep the information about your file system and will it help you in the future if you get disk corruption.
This does not only apply to linux partitions but all types of partitions.

Mount a RAID Reconstructor disk image in linux

On 19. January 200924. May 2012 By Hans-Henry JakobsenIn Linux, Misc, Scripting

This post came to life because I had gotten a image file of a 120GB linux ext2 RAID partition that I couldn’t access using Windows software. The partition had been rescued by RAID Reconstructor, a Windows software that can rebuild RAID arrays by combining the disks from the RAID and store it as one image file.

The solution was to use linux and try to mount it as a loop filesystem. My assumption was that the partition itself had an unknown offset, so I used this script to search and mount the partition. The script tries to mount the partition within the first 20000 blocks. I assume the block size is 512 bytes and I already know that the partition is formatted as ext2.

for ((i=0 ; $i < 20000 ; i=$i + 1)) ; do
    mount -t ext2 -o loop,offset=$(($i * 512)) diskimage.img /mnt/point && break
done

If it succeeds you can issue the mount command to get the offset value to the mounted partition

# mount | grep diskimage

/dev/loop0 on /mnt/point type ext2 (rw,offset=32256)

In my case the offset was 32256.

This script can also be used to mount a partition from a backup of your filesystem.

Example
This is my example disk I'm going to create an image of

# fdisk -l

Disk /dev/hda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1          14      112423+  83  Linux
/dev/hda2              15          96      658665   83  Linux
/dev/hda3              97        3394    26491185   83  Linux
/dev/hda4            3395        9729    50885887+   f  W95 Ext'd (LBA)
/dev/hda5            3395        7218    30716248+  83  Linux
/dev/hda6            7219        8750    12305758+  83  Linux
/dev/hda7            8751        9533     6289416   83  Linux
/dev/hda8            9534        9729     1574338+  83  Linux

Create a backup image of your entire disk

# dd if=/dev/hda of=imagefile.img

To mount one of these partitions as a loop filesystem you can issue a mount command and tell it the right offset. Use the script above if you don't know the offset, it will take the guesswork for you and mount the partition.

Example of mounting /dev/hda3 if the offset is known

# mount -t ext3 -o loop,offset=49664 /dev/hda3 /mnt/point

Determining the offset is easy when you know the disk geometry have access to the file table.
In this case the hda3 partition has an offset of 512 bytes * 96 blocks = 49664.

EDIT
This post could have been avoided if I had known about the linux tool TestDisk. testDisk is a powerful free data recovery software! It was primarily designed to help recover lost partitions and/or make non-booting disks bootable again when these symptoms are caused by faulty software, certain types of viruses or human error (such as accidentally deleting a Partition Table). It could also be used as a forensic tool.

Pario TechnoBlob

Tag: dd