Mounting partitions from a dd image (opens in new tab)

(assafmo.github.io)

85 pointsassafmo7y ago34 comments

34 comments

This is doing it the hard way. Make the image a loop device:

   losetup /dev/loop0 /path/to/your/image.img

Now you can use /dev/loop0 like it's a normal disk. If you're not seeing your partitions (/dev/loop0p1, etc.):

    partprobe /dev/loop0

When you're done, detach the loop device:

    losetup -d /dev/loop0

I use this trick to transparently compress ddrescue images with brtfs: https://brashear.me/blog/2017/11/14/how-to-create-a-compress...

jschwartzi7y ago

If you have a full dd image of the disk, you can just do losetup -Pf /path/to/your/image.img and that will pick a loopback device and perform the partprobe on it automatically.

strong_silent_t7y ago

Thanks, not OP but that looks useful to me. For context, I've used the strategy in OP to debug a small linux system running on an SD card, if I have one version that has a bug and one that doesn't, and I don't know the difference, I can image them both, mount them and diff the whole system (i.e. with meld.) And it can be done with the same card over time. I'd like to figure out how to do this with VM disks as well but I've never had a pressing need.

jlgaddis7y ago

> I'd like to figure out how to do this with VM disks as well but I've never had a pressing need.

Everything that you will (probably) need is here: http://libguestfs.org

If you happen to need something that isn't already included, the APIs make it pretty easy to build your own tools as well.

Conveniently, those tools are also already packaged up for your distro (unless you use some esoteric distribution, perhaps).

discreditable7y ago

I commonly use loopdevs to create and partition bootable disk images for clonezilla. I like that I can create them sparse so that they take up less space.

mrrsm7y ago

I came here to say using the loop device just makes things much easier. No math to get your offsets and fat fingering the numbers this way.

lathiat7y ago

Interesting I have used kpartx which uses device mapper instead of Partprobe.

Wonder why kpartx exists; will have to look into that. Maybe not everything can be partprobed.

discreditable7y ago

I'm not sure. All I can think is that partprobe is provided by parted and I have noticed people tend to gravitate towards using fdisk or parted and ignoring the other. Notice the author of the article is an fdisk guy.

I might be mis-remebering but I think parted used to be considered lame before fdisk lagged to support GPT.

1 more reply

jschwartzi7y ago

kpartx is an older tool that was used with 2.6.xx-era Linux systems. partprobe/losetup are the newer mechanisms for doing this.

A lot of StackOverflow questions have answers that use kpartx, and those answers are out of date.

detuur7y ago

There's just one thing I don't get with this approach. How do you make use of the space savings? You've explicitly created an outside disk image which is larger than the ddrescue image, and the ddrescue image is inside that larger file. So while the inside image is taking less space inside the mounted loopback volume, the outside disk image will always take the original amount of space. Do you trim the outside volume after you're done or do you somehow extract the lzo-encoded disk image after?

pastage7y ago

It's called sparse files, it's files with holes in them where no data is written to disk. You can have terabytes of files on small disks as long as they contain a high % of bocks of only zeros.

1 more reply

jlgaddis7y ago

Millions of zeroes in a row tend to compress pretty well.

nzmsv7y ago

There’a actually an even easier way for many use cases. Just use libguestfs. One big benefit it has over the losetup process is that it works in unprivileged containers because nothing needs to be mounted.

Skunkleton7y ago

You can make that even easier:

    # losetup -Pf /path/to/disk.img
    # mount /dev/loop0pX /mnt

maccam947y ago

Additionally, this method is not susceptible to race conditions when other loop devices are being configured.

dbolgheroni7y ago

OpenBSD uses vnconfig (also FreeBSD; on NetBSD it's vndconfig). An example:

  # vnconfig vnd0 install63.fs

Then you can treat vnd0 as if it was a disk:

  # disklabel vnd0
  # /dev/rvnd0c:
  type: vnd
  disk: vnd device
  label: fictitious
  duid: 138b4f2a2e184426
  flags:
  bytes/sector: 512
  sectors/track: 100
  tracks/cylinder: 1
  sectors/cylinder: 100
  cylinders: 7382
  total sectors: 738240
  boundstart: 1024
  boundend: 737280
  drivedata: 0 
  
  16 partitions:
  #                size           offset  fstype [fsize bsize   cpg]
    a:           736256             1024  4.2BSD   2048 16384 16142 
    c:           738240                0  unused                    
    i:              960               64   MSDOS

For instance, to mount the 'a' partition:

  # mount /dev/vnd0a /mnt

You can also associate an encryption key with the device. All data written to the diskimage will then be encrypted.

floatboth7y ago

FreeBSD does not use vnconfig, FreeBSD uses mdconfig:

    # mdconfig -f smartos-latest-USB.img
    md0
    # gpart list md0
    ...
    scheme: MBR
    Providers:
    1. Name: md0s1
       ...
    # mount -o ro -t msdosfs /dev/md0s1 /mnt
    // (or, use labels)
    # mount -o ro -t msdosfs /dev/msdosfs/SMARTOSBOOT /mnt

Or for ISOs, in one line:

    # mount -t cd9660 /dev/$(mdconfig -f image.iso) /mnt

jlgaddis7y ago

I was just about to correct your correction, because I can quite clearly recall using vnconfig on FreeBSD.

Apparently that's been a little while, however. According to vnconfig(8):

> The mdconfig utility first appeared in FreeBSD 5.0 as a cleaner replacement for the vn(4) and vnconfig(8) combo.

5.0 was released just over 15 years ago.

phaedrus7y ago

One time I tried to create a tool to do in-place conversion between Linux software RAID volumes (e.g. RAID-0 to RAID-1 (assuming data fits), or RAID-1 to RAID-5, etc.) I used disk image files and the loopback device to test it to avoid having to reformat physical partitions.

Despite reading all of the documentation I could find, and even going so far as to read kernel source code, I could never get the system to recognize my striped block data. That is, I tried to generate three files composing a RAID-5 "by hand", but mdadm would refuse to mount them. I never figured out if there was some additional id-block I was missing, or if my striping algorithm was not correct.

ckastner7y ago

Under Linux, this can also be achieved with kpartx(8).

https://linux.die.net/man/8/kpartx

rwmj7y ago

Or without needing root using guestfish (http://libguestfs.org/guestfish.1.html)

    $ guestfish -a img.dd -m /dev/sda1
    ><fs> ll /

guestmount works if you want to mount it on a local directory (implemented using libguestfs + FUSE).

dpedu7y ago

I think ZFS has the easiest way to do this sort of thing:

    # fallocate -l 256M disk.bin
    # zpool create filepool /root/disk.bin
    # zfs set mountpoint=/root/filepool filepool
    # df -h /root/filepool
    Filesystem      Size  Used Avail Use% Mounted on
    filepool        208M     0  208M   0% /root/filepool

trm427y ago

Sure, ZFS has lots of awesome features, but the point of loopback device lets you to attach and mount basically any disk image that has kernel or fuse drivers.

theonewolf7y ago

You can directly expose the partitions using the tool `kpartx`, which will create a `loop` device for you:

  > sudo kpartx -a test.raw
  > lsblk
  NAME      MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
  loop0       7:0    0     4G  0 loop
  └─loop0p1 253:0    0     4G  0 part
  > sudo kpartx -d test.raw

Ace177y ago

> mount -t -o ro,offset=$((51263)) x.dd partition_1

> mount -t -o ro,offset=$((51296390)) x.dd partition_2

Isn't the mount type "-t" value missing?

assafmoOP7y ago

Thanks! I missed this. Fixed. :-)

pixelbeat__7y ago

libguestfs > losetup > fdisk

I.E. libguestfs is safer when dealing with images of unknown origin, and losetup has inbuilt support now for partition scanning.

Here's an old fdisk/parted wrapper I used to use:

http://www.pixelbeat.org/scripts/lomount.sh

amelius7y ago

Is it also possible to repartition the image with conventional tools?

How about setting a boot flag and writing it back to e.g. an USB stick?

peterwwillis7y ago

Yes, you can use losetup.

  ~# dd if=/dev/zero of=image.img bs=1M count=100
  ~# LODEVICE=$(losetup --find --show image.img)
  ~# fdisk $LODEVICE
  ~# losetup -d $LODEVICE

^ The above creates a 100MB disk image, binds it to a loopback device, runs fdisk, and then detaches it. I'm not sure how formatting the partition would work but presumably those tools have an offset option too?

edit: this page explains using offsets for mke2fs and doesn't require losetup: https://superuser.com/questions/737072/userspace-manipulatio...

also, losetup can now scan the device for partition tables and make partition-specific loopbacks! https://stackoverflow.com/a/15200862

regularfry7y ago

Extlinux on an ext4 partition doesn't need an offset. Or at least didn't when I wrote this: http://www.blackkettle.org/posts/archive/2013-08-19-bootable...

jschwartzi7y ago

I have built block-level bootable system images for the TS-7800 using this mechanism.

https://www.embeddedarm.com/products/TS-7800

That was before I found multistrap.

bcl7y ago

You can run parted directly on the image file to make modifications.

    fallocate -l 1G disk.img
    parted ./disk.img

j / k navigate · click thread line to collapse

34 comments

discreditable7y ago

This is doing it the hard way. Make the image a loop device:

   losetup /dev/loop0 /path/to/your/image.img

Now you can use /dev/loop0 like it's a normal disk. If you're not seeing your partitions (/dev/loop0p1, etc.):

    partprobe /dev/loop0

When you're done, detach the loop device:

    losetup -d /dev/loop0

I use this trick to transparently compress ddrescue images with brtfs: https://brashear.me/blog/2017/11/14/how-to-create-a-compress...

jschwartzi7y ago

If you have a full dd image of the disk, you can just do losetup -Pf /path/to/your/image.img and that will pick a loopback device and perform the partprobe on it automatically.

strong_silent_t7y ago

jlgaddis7y ago

> I'd like to figure out how to do this with VM disks as well but I've never had a pressing need.

Everything that you will (probably) need is here: http://libguestfs.org

If you happen to need something that isn't already included, the APIs make it pretty easy to build your own tools as well.

Conveniently, those tools are also already packaged up for your distro (unless you use some esoteric distribution, perhaps).

discreditable7y ago

I commonly use loopdevs to create and partition bootable disk images for clonezilla. I like that I can create them sparse so that they take up less space.

mrrsm7y ago

I came here to say using the loop device just makes things much easier. No math to get your offsets and fat fingering the numbers this way.

lathiat7y ago

Interesting I have used kpartx which uses device mapper instead of Partprobe.

Wonder why kpartx exists; will have to look into that. Maybe not everything can be partprobed.

discreditable7y ago

I might be mis-remebering but I think parted used to be considered lame before fdisk lagged to support GPT.

1 more reply

jschwartzi7y ago

kpartx is an older tool that was used with 2.6.xx-era Linux systems. partprobe/losetup are the newer mechanisms for doing this.

A lot of StackOverflow questions have answers that use kpartx, and those answers are out of date.

detuur7y ago

pastage7y ago

It's called sparse files, it's files with holes in them where no data is written to disk. You can have terabytes of files on small disks as long as they contain a high % of bocks of only zeros.

1 more reply

jlgaddis7y ago

Millions of zeroes in a row tend to compress pretty well.

nzmsv7y ago

Skunkleton7y ago

You can make that even easier:

    # losetup -Pf /path/to/disk.img
    # mount /dev/loop0pX /mnt

maccam947y ago

Additionally, this method is not susceptible to race conditions when other loop devices are being configured.

dbolgheroni7y ago

OpenBSD uses vnconfig (also FreeBSD; on NetBSD it's vndconfig). An example:

  # vnconfig vnd0 install63.fs

Then you can treat vnd0 as if it was a disk:

  # disklabel vnd0
  # /dev/rvnd0c:
  type: vnd
  disk: vnd device
  label: fictitious
  duid: 138b4f2a2e184426
  flags:
  bytes/sector: 512
  sectors/track: 100
  tracks/cylinder: 1
  sectors/cylinder: 100
  cylinders: 7382
  total sectors: 738240
  boundstart: 1024
  boundend: 737280
  drivedata: 0 
  
  16 partitions:
  #                size           offset  fstype [fsize bsize   cpg]
    a:           736256             1024  4.2BSD   2048 16384 16142 
    c:           738240                0  unused                    
    i:              960               64   MSDOS

For instance, to mount the 'a' partition:

  # mount /dev/vnd0a /mnt

You can also associate an encryption key with the device. All data written to the diskimage will then be encrypted.

floatboth7y ago

FreeBSD does not use vnconfig, FreeBSD uses mdconfig:

    # mdconfig -f smartos-latest-USB.img
    md0
    # gpart list md0
    ...
    scheme: MBR
    Providers:
    1. Name: md0s1
       ...
    # mount -o ro -t msdosfs /dev/md0s1 /mnt
    // (or, use labels)
    # mount -o ro -t msdosfs /dev/msdosfs/SMARTOSBOOT /mnt

Or for ISOs, in one line:

    # mount -t cd9660 /dev/$(mdconfig -f image.iso) /mnt

jlgaddis7y ago

I was just about to correct your correction, because I can quite clearly recall using vnconfig on FreeBSD.

Apparently that's been a little while, however. According to vnconfig(8):

> The mdconfig utility first appeared in FreeBSD 5.0 as a cleaner replacement for the vn(4) and vnconfig(8) combo.

5.0 was released just over 15 years ago.

phaedrus7y ago

ckastner7y ago

Under Linux, this can also be achieved with kpartx(8).

https://linux.die.net/man/8/kpartx

rwmj7y ago

Or without needing root using guestfish (http://libguestfs.org/guestfish.1.html)

    $ guestfish -a img.dd -m /dev/sda1
    ><fs> ll /

guestmount works if you want to mount it on a local directory (implemented using libguestfs + FUSE).

dpedu7y ago

I think ZFS has the easiest way to do this sort of thing:

    # fallocate -l 256M disk.bin
    # zpool create filepool /root/disk.bin
    # zfs set mountpoint=/root/filepool filepool
    # df -h /root/filepool
    Filesystem      Size  Used Avail Use% Mounted on
    filepool        208M     0  208M   0% /root/filepool

trm427y ago

Sure, ZFS has lots of awesome features, but the point of loopback device lets you to attach and mount basically any disk image that has kernel or fuse drivers.

theonewolf7y ago

You can directly expose the partitions using the tool `kpartx`, which will create a `loop` device for you:

  > sudo kpartx -a test.raw
  > lsblk
  NAME      MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
  loop0       7:0    0     4G  0 loop
  └─loop0p1 253:0    0     4G  0 part
  > sudo kpartx -d test.raw

Ace177y ago

> mount -t -o ro,offset=$((51263)) x.dd partition_1

> mount -t -o ro,offset=$((51296390)) x.dd partition_2

Isn't the mount type "-t" value missing?

assafmoOP7y ago

Thanks! I missed this. Fixed. :-)

pixelbeat__7y ago

libguestfs > losetup > fdisk

I.E. libguestfs is safer when dealing with images of unknown origin, and losetup has inbuilt support now for partition scanning.

Here's an old fdisk/parted wrapper I used to use:

http://www.pixelbeat.org/scripts/lomount.sh

amelius7y ago

Is it also possible to repartition the image with conventional tools?

How about setting a boot flag and writing it back to e.g. an USB stick?

peterwwillis7y ago

Yes, you can use losetup.

  ~# dd if=/dev/zero of=image.img bs=1M count=100
  ~# LODEVICE=$(losetup --find --show image.img)
  ~# fdisk $LODEVICE
  ~# losetup -d $LODEVICE

edit: this page explains using offsets for mke2fs and doesn't require losetup: https://superuser.com/questions/737072/userspace-manipulatio...

also, losetup can now scan the device for partition tables and make partition-specific loopbacks! https://stackoverflow.com/a/15200862

regularfry7y ago

Extlinux on an ext4 partition doesn't need an offset. Or at least didn't when I wrote this: http://www.blackkettle.org/posts/archive/2013-08-19-bootable...

jschwartzi7y ago

I have built block-level bootable system images for the TS-7800 using this mechanism.

https://www.embeddedarm.com/products/TS-7800

That was before I found multistrap.

bcl7y ago

You can run parted directly on the image file to make modifications.

    fallocate -l 1G disk.img
    parted ./disk.img

j / k navigate · click thread line to collapse