I'd like to do something along the lines of

mkfs -t btrfs filedrive

mount filedrive /media/fuse

Without specifying a particular size, I'd like to be able to have the file grow in size as I write files into the mounted filesystem, and shrink when files are deleted.

Is there some mechanism for this?

I have also seen this question and I am aware that it could in theory be managed manually, but my question has nothing to do with ecryptfs and focuses on the automatic aspect. I do not particularly care about the filesystem type inside the file.

I am also aware of existing systems that do something similar: specifically VirtualBox's Dynamic Disks. If there is some way of using those - or something akin to them - without actually running a virtual machine, I would also be happy with that.

Actually, something like it is already possible

A filesystem needs to have a defined maximum size. But as the filesystem-as-file can be sparse, that size can be an arbitrary number which doesn't have much to do with how much space the filesystem-as-file takes up on the underlying filesystem.

If you can accept setting an arbitrary maximum size limit (which can be much greater than the actual size of the underlying filesystem) for the filesystem-as-file, you can create a sparse file and a filesystem on it right now:

/tmp# df -h .

Filesystem                Size  Used Avail Use% Mounted on

<current filesystem>       20G   16G  3.0G  84% /



/tmp# dd if=/dev/null bs=1 seek=1024000000000 of=testdummy

0+0 records in

0+0 records out

0 bytes copied, 0.000159622 s, 0.0 kB/s

/tmp# ll testdummy

-rw-r--r-- 1 root root 1024000000000 Feb 19 08:24 testdummy

/tmp# ll -h testdummy

-rw-r--r-- 1 root root 954G Feb 19 08:24 testdummy

Here, I've created a file that appears to be a whole lot bigger than the filesystem it's stored onto...

/tmp# du -k testdummy

0       testdummy

...but so far it does not actually take any disk space at all (except for the inode and maybe some other metadata).

It would be perfectly possible to losetup it, create a filesystem on it and start using it. Each write operation that actually writes data to the file would cause the file's space requirement to grow. In other words, while the file size as reported by ls -l would stay that arbitrary huge number all the time, the actual space taken by the file on the disk as reported by du would grow.

And if you mount the filesystem-as-file with a discard mount option, shrinking can work automatically too:

/tmp# losetup /dev/loop0 testdummy

/tmp# mkfs.ext4 /dev/loop0

/tmp# mount -o discard /dev/loop0 /mnt

/tmp# du -k testdummy 

1063940 testdummy

/tmp# df -h /mnt

Filesystem      Size  Used Avail Use% Mounted on

/dev/loop0      938G   77M  890G   1% /mnt



/tmp# cp /boot/initrd.img /mnt

/tmp# du -k testdummy 

1093732 testdummy



/tmp# rm /mnt/initrd.img

/tmp# du -k testdummy

1063944 testdummy

Automatic shrinking requires:

1.) that the filesystem type of the filesystem-as-file supports the discard mount option (so that the filesystem driver can tell the underlying system which blocks can be deallocated)

2.) and that the filesystem type of the underlying filesystem supports "hole punching", i.e. the fallocate(2) system call with the FALLOC_FL_PUNCH_HOLE option (so that the underlying filesystem can be told to mark some of the previously-allocated blocks of the filesystem-as-file as sparse blocks again)

3.) and that you're using kernel version 3.2 or above, so that the loop device support has the necessary infrastructure for this.

https://outflux.net/blog/archives/2012/02/15/discard-hole-punching-and-trim/

If you're fine with less immediate shrinking, you could just periodically run fstrim on the filesystem-as-file instead of using the discard mount option. If the underlying filesystem is very busy, avoiding immediate shrinking might help minimizing the fragmentation of the underlying filesystem.

The problem with this approach is that if the underlying filesystem becomes full, it won't be handled very gracefully. If there is no longer space in the underlying filesystem, the filesystem-as-file will start receiving errors when it's trying to replace sparse "holes" with actual data, even as the filesystem-as-file would appear to have some unused capacity left.

No, it is not supported automatically

Filesystems are created to handle a fixed amount of disk space. Some filesystems support growing, and fewer support shrinking, but not on demand without user intervention. An answer to the question you mention says that you have to do the resizing manually.

Of course what can be done manually also can be done automatically, but it would need support form the kernel that in case the file system is full it would not report the filesystem as full but enlarge the filesystem. You further want to shrink the filesystem automatically when you delete a file, this is potentially a very time consuming operation as you might have to move data around so that space at the end of the filesystem is free.

A filesystem created specifically for this task could just allocate blocks at the end on grow and free blocks in the middle on shrink, but there doesn't seem to be someone who wants to create such a file system. Of course you are free to do it yourself.

VirtualBox's Dynamic Disks are not the same. They are created with a fixed maximum size, they just don't immediately use up that amount of disk space. A block is allocated as soon as data is written to that block, and it doesn't shrink because that block is unused.

Using a sparse file + loop device + TRIM as in the accepted is nice and fine, however I'm not really convinced yours is not an XY problem: why not simply use a subdirectory/subtree? That will grow and shrink as needed, and you could also mount in another place either with mount --bind or NFS ;-)

I am also aware of existing systems that do something similar: specifically VirtualBox's Dynamic Disks. If there is some way of using those - or something akin to them - without actually running a virtual machine, I would also be happy with that.

With qemu-ndb, you can also use in the host system any disk image supported by qemu:

# modprobe nbd

# qemu-nbd --discard=unmap -c /dev/nbd1 image.qcow2

# kpartx -a /dev/nbd1

# mount /dev/mapper/nbd1p1 /mount/point

...

# fstrim -v /mount/point

...

# qemu-nbd -d /dev/nbd1

Qemu's qcow2 images not only grow and shrink to accomodate the data, they also support snapshots & "diff" images.