What's with all these hip filesystems and how are they different?

You know, ZFS, ButterFS (btrfs…its actually “better” right?), and I’m sure more.

I think I have ext4 on my home computer I installed ubuntu on 5 years ago. How does the choice of file system play a role? Is that old hat now? Surely something like ext4 has its place.

I see a lot of talk around filesystems but Ive never found a great resource that distiguishes them at a level that assumes I dont know much. Can anyone give some insight on how file systems work and why these new filesystems, that appear to be highlights and selling points in most distros, are better than older ones?

Edit: and since we are talking about filesystems, it might be nice to describe or mention how concepts like RAID or LUKS are related.

MigratingtoLemmy,

Did BTRFS fix the Raid-5 problem or no?

manifesto7473,

No, but according to this Phoronix article, they will fix the RAID56 issues soon:

The support for RAID56 is in development and will eventually fix the problems with the current implementation. This is a backward incompatible feature and has to be enabled at mkfs time.

MigratingtoLemmy,

That is taking a surprisingly long time TBH

chitak166,

I just use ext4.

duncesplayed,

The principled “old” way of adding fancy features to your filesystem was through block-level technologies, like LVM and LUKS. Both of those are filesystem-agnostic, meaning you can use them with any filesystem. They just act as block devices, and you can put any filesystem on top of them.

You want to be able to dynamically grow and shrink partitions without moving them around? LVM has you covered! You want to do RAID? mdadm has you covered! You want to do encryption? LUKS has you covered? You want snapshotting? Uh, well…technically LVM can do that…it’s kind of awkward to manage, though.

Anyway, the point is, all of them can be mixed and matched in any configuration you want. You want a RAID6 where one device is encrypted split up into an ext4 and two XFS partitions where one of the XFS partitions is in RAID10 with another drive for some stupid reason? Do it up, man. Nothing stopping you.

For some reason (I’m actually not sure of the reason), this stagnated. Red Hat’s Strata project has tried to continue pushing in this direction, kind of, but in general, I guess developers just didn’t find this kind of work that sexy. I mentioned LVM can do snapshotting "kind of awkward"ly. Nobody’s done it in as sexy and easy way to do as the cool new COWs.

So, ZFS was an absolute bombshell when it landed in the mid 2000s. It did everything LVM did, but way way way better. It did everything mdadm did, but way way way better. It did everything XFS did, but way way way better. Okay it didn’t do LUKS stuff (yet), but that was promised to be coming. It was Copy-On-Write and B-tree-everywhere. It did everything that (almost) every other block-level and filesystem previously made had ever done, but better. It was just…the best. And it shit all over that block-layer stuff.

But…well…it needed a lot of RAM, and it was licensed in a way such that Linux couldn’t get it right away, and when it did get ZFS support, it wasn’t like native in-the-kernel kind of stuff that people were used to.

But it was so good that it inspired other people to copy it. They looked at ZFS and said “hey why don’t we throw away all this block-level layered stuff? Why don’t we just do every possible thing in one filesystem?”.

And so BtrFS was born. (I don’t know why it’s pronounced “butter” either).

And now we have bcachefs, too.

What’s the difference between them all? Honestly mostly licensing, developer energy, and maturity. ZFS has been around for ages and is the most mature. bcachefs is brand spanking new. BtrFS is in the middle. Technically speaking, all of them either do each other’s features or have each other’s features on their TODO list. LUKS in particular is still very commonly used because encryption is still missing in most (all?) of them, but will be done eventually.

OmnipotentEntity,
@OmnipotentEntity@beehaw.org avatar

ZFS has encryption now, dunno about the rest

sexy_peach,
@sexy_peach@feddit.de avatar

you’re forgetting about the all new bcachefs

dunno I also wonder if it’s worth it for personal use

rutrum,
@rutrum@lm.paradisus.day avatar

Now THAT is hip

sexy_peach,
@sexy_peach@feddit.de avatar

Well I am a very cool guy 😏

Pantherina,
@Pantherina@feddit.de avatar

Fedora uses BTRFS so I get the features are the best argument for it

www.phoronix.com/news/Linux-5.14-File-Systems

But it seems F2FS is by far the fastest in many areas! Its used in Android, optimized for Flash storage.

AMDIsOurLord,

Using Btrfs you can do some pretty cool snapshotting: It’s basically like system restore of Windows but MUCH faster and pretty seamless. Even if you annihilate the whole operating system you can restore the snapshot and voila, have fun! It also has compression which can save some wear on SSDs and of course give you some more free™ storage space, which is cool [actual benefits depend on workload*]

ipkpjersi,

ZFS also has snapshotting too FWIW.

AMDIsOurLord,

ZFS has almost everything ever conceived for filesystems lol it’s a whole ass volume manager and filesystem into one

mcepl,
@mcepl@lemmy.world avatar

This is twelve years old, but it nicely illustrates what BTRFS (and ZFS on other OS) can do … youtu.be/9H7e6BcI5Fo?t=206

jlow,

I wasn’t cool enough to figure out how to “just boot into a snapshot” when I tried btrfs a while ago. I mean I did figure it out (maybe?) but somehow the read/write rights where messed up and the snapshot couldn’t actually boot/I wasn’t able to log in +___+ Just reinstalled the system eith good old ext4. It sounds really cool, though …

Pantherina,
@Pantherina@feddit.de avatar

Fedora out of the box just works

AMDIsOurLord,

Well, sounds like a setup or distro issue. It should work without problems on Debian/Ubuntu/Mint. Linux Mint even really supports it as a setup-less default with TimeShift

Pantherina,
@Pantherina@feddit.de avatar

Do you know how I could split my default /var/home/user into /var/home/user/.var, /var/home/user/Torrents and the rest?

Think that would be great for use with btrbk, when I find out how to use that.

Damn BTRFS and btrbk need an easy GUI, I have the feeling its great for backups

AMDIsOurLord, (edited )

There’s no GUI, but following the wiki pages on BTRFS subvolumes you should be able to make a subvolume for those with like 2 simple commands (take a look at the man page for BTRFS subvolumes as well)

callyral,
@callyral@pawb.social avatar

related question, although i don’t think it’s big enough for a post of its own.

if i use btrfs subvolumes, does it mean that i can have one EFI partition and one root partition, and then subdivide the root partition using subvolumes? how would that work during the installation process? or is it done after installation?

the_third,

if i use btrfs subvolumes, does it mean that i can have one EFI partition and one root partition, and then subdivide the root partition using subvolumes?

Yes.

how would that work during the installation process? or is it done after installation?

That depends on your distro. What do you plan to achieve with the separate subvolumes?

callyral,
@callyral@pawb.social avatar

I already have a partition layout in btrfs where I have a /home and a /root partition, since when I installed I didn’t know about btrfs subvolumes. I use Void Linux and I think it’s after installation, since I don’t remeber having a subvolumes step during the installation.

I’ll make sure to remind about btrfs subvolumes in case I reinstall. There’s a btrfs program that has a subvolume argument, so I’m guessing that’s what I could use.


<span style="color:#323232;">usage: btrfs [global]  [...]  []
</span><span style="color:#323232;">
</span><span style="color:#323232;">...
</span><span style="color:#323232;">Command groups:
</span><span style="color:#323232;">  subvolume         manage subvolumes: create, delete, list, etc
</span>
mhz, (edited )

One EFI + one ROOT partition is what I do on both my laptop and desktop for years, /home is a subvolume to my root partition. This setup suits my needs as I don’t have to worry about how big should my root or home (gaming) partition should be.

I use Arch on my desktop and Opensuse on my laptop. They both have options to set up subvolumes from their installer, Debian does not, and I’m not sure about other distros, but you can always set that up after installation, just make your home partition the last one (after the root partition) so you can easily delete it after and grow the root partition without much blocks relocation.

danielquinn, (edited )
@danielquinn@lemmy.ca avatar

I’ve never heard of sub volumes. What do they do for me? Why not just partition the disk or store everything on the one partition?

mhz,

I like to think a subvolume is a directory on my filesystem that:

  • Acts as an independent filesystem.
  • Shares it’s parent size (unless quotas are set in place)
  • Can be mounted/unmounted any time
  • Excluded from their parent partition’s snapshots. (a /home subvolume is exluded from / snapshots).
  • Can be snapshot-ed independently.

This is by no mead a definition for BTRFS subvolume, but I hope you get the idea.

lupec,

Pretty much, yeah. At some point I remember the recommendation being having a separate /boot as well due to incompatibilities with GRUB’s save default option iirc, not sure that’s a thing anymore.

Anyway, you usually set that up during the install process, although I’m not sure graphical installers let you handcraft btrfs subvolume mount points or even select them as such these days. Last I checked at least they either just used a default layout (@ and @home with Ubuntu, for instance) or treated it as a single volume with no further options.

RustyNova,

Exactly. But if you tend to be on the hoarder side, put a swap partition in there too. Even 32GB ram isn’t enough sometimes

lolcatnip,

ButterFS (btrfs…its actually “better” right?),

I’m still waiting to find out who the BCA Chefs are.

ReversalHatchery,

ext4 certainly has its place, it’s a fine default file system, there’s really no problems with it.

But others, like ZFS and BTRFS, have features that you may want to use, but ext4 doesn’t do: fs snapshots, data compression, built in encryption (to a degree, usually only happening for data and some of the metadata, so LUKS is often better IMHO), checking for bitrot and restoring it when possible (whether it is depends on your config), quotas per user group or project, spanning multiple disks like with RAID but safer (to a degree), and others.

possiblylinux127,

ZFS is a crazy beast that’s best for high end server systems with tiered storage and lots of RAM.

ext4 is really just a basic file system. Its superior to NTFS and fat as it does have extra features to try to prevent corruption but it doesn’t have a large feature set.

Btrfs is kind of the new kid on the block. It has strong protection against corruption and has better real world performance than ext4. It also has more advanced features like sub volumes and snapshots. subvolumes are basically virtual drives.

Another few older options include things like XFS but I won’t go into those.

List of filesystems: en.m.wikipedia.org/…/Comparison_of_file_systems

Lojcs,

and has better real world performance than ext4

Source? Most benchmarks I’ve seen it lags behind

possiblylinux127,

My personal testing

the_third,

Let’s agree on: it has a different performance for various use cases and hardware below. Sometimes better, sometimes worse.

possiblylinux127,

Well I use it pretty much exclusively now for bare hardware. For VMs it doesn’t matter so I use ext4

the_third,

Not much time, I’ll be brief with three examples that come to mind from my experience:

Great use: Large filestorage with regular changes, daily snapshots, stream snapshots offsite as backup.

Not so great use: Storage backend for qcow2 backed VMs on spinning RAID. CoW made a mess of access times.

Really not great use: Large Postgres-DB with queries that creted large ondisk temp tables.

It really depends.

AMDIsOurLord,

Benchmarks are also usually very different from real world usage, tbh

Pantherina,
@Pantherina@feddit.de avatar

www.phoronix.com/news/Linux-5.14-File-Systems

It seems actually F2FS is the best?? Thats used in Android and optimized for Flash storage, does that include SSDs?

Lojcs,

Yes, but most filesystems are already optimized for flash storage. Arch wiki says f2fs is prone to corruption on power loss. Based on that and the lack of information on its anti-corruption measures I’m inclined to think it doesn’t have one and that’s why it’s faster. I wouldn’t use it in a non-battery operated device.

Pantherina,
@Pantherina@feddit.de avatar

So basically all laptop users can safely use it.

Crazy how PC users rely on such a steady power supply. Arent there small UPS devices for a few seconds with auto shutdown?

Gabu,

Catastrophic battery failure isn’t really any less likely than catastrophic power supply failure (conceptually. If you use a brandless grey power supply, results may vary).

d3Xt3r, (edited )

That link is for kernel 5.14, so I’d say those results are pretty much invalid for most users (unless you’re actually on it, or the 5.15 LTS kernel). There have been a ton of improvements in every filesystem since then, with pretty much every single kernel release.

A more relevant test would be this one - although it talks about bcachefs, other filesystems are also included in it. As you can see, F2FS is no longer the fastest - bcachefs and XFS beat it in several tests, and even btrfs beats it in some tests. F2FS only wins in the Dbench and CockroachDB benchmarks.

Pantherina,
@Pantherina@feddit.de avatar

Thanks. Bcachefs is for SSD-HDDs isnt it?

d3Xt3r,

Not quite. Bcachefs can be used on any drive, but it shines the best when you have a fast + slow drive in your PC (eg NVMe + HDD), so the faster drive can be used as a cache drive to store frequently accessed data.

christian,
@christian@lemmy.ml avatar

I know I’m not making a helpful contribution here, but I’ve been wondering about this stuff for a while myself and this thread has some great answers. Thanks for asking this OP.

jokro,

btrfs = B-tree filesystem

B-trees are a data structure.

kpw,

Great. So what?

Chewy7324,

You know, ZFS, ButterFS (btrfs…its actually “better” right?), and I’m sure more.

tastysnacks,

I call it butterface

teawrecks,

Butterface?! That’s my wife!

Frederic, (edited )

After using ext4 for yyyeeeaaaarrrrrsss, when I upgraded my MX21 to MX23 I used btrfs, with subvolumes, especially for easy backup/snapshot/timeshift.

Just at install, super easy, create a small ext4 boot partition on the SSD, then a big LUKS partition, format with btrfs, create subvolumes for / /home /var /swap and that’s it. No hassle with sizing correctly.

btrfs seems pretty stable. I see no diff in performance compared to ext4 because my application are not that dependant to FS speed, and with SSD anyway?

oh yeah, built-in compression too!

BeefPiano,

I don’t know about the new ones, but ReiserFS was a killer back in the day.

n2burns,

Bad joke.

rotopenguin,
@rotopenguin@infosec.pub avatar

The horrible part is it was. Your other choice was ext2, which wasted so many lifetimes with its hours long fsck times. Reiserfs was a cut above the rest, we would all be using it today if it weren’t for that one teensy-weensy legal issue.

cybersandwich,

I have an old drive with it in there. The drive is going bad so I haven’t messed with it too much. I never knew at the time why the development and shine faded so quickly.

bjoern_tantau, (edited )
@bjoern_tantau@swg-empire.de avatar

I’ve started using BTRFS on my laptop with OpenSUSE and on my Steam Deck. It does two things for me, which I’m interested in. On OpenSUSE it does a snapshot before every system update. So if anything goes wrong I can easily roll back.

On the Steam Deck I love the deduplication. It’s really great for a ton of Windows games that all need their own little “Windows” environment which amounts to a GB or two per game. With BTRFS I only use that space once.

rutrum,
@rutrum@lm.paradisus.day avatar

Can you elaborate more on deduplication? Is this a feature you setup, or does it sort of work out of the box? This is a new concept to me, but sounds incredibly useful, especially in that scenario.

bjoern_tantau,
@bjoern_tantau@swg-empire.de avatar

I used a script that did everything for me, so I’m not 100 % sure. But as far as I know you enable the feature at mount time and then every time you copy something only a reference is copied until you actually do a change to the new or old file.

For everything else a cronjob runs every week or so to search for unnecessary duplicates.

Chewy7324,

And if a copied file is changed, btrfs only stores the difference instead of two complete files. E.g. if the 1GB file1 is copied to file2, they will take 1GB total. If 100MB is appended to file2, the total storage usage is 1,1GB

Discover5164,

it’s not automatic since it will eat resources while it’s running. but it’s a feature of btrfs.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • uselessserver093
  • Food
  • [email protected]
  • aaaaaaacccccccce
  • test
  • CafeMeta
  • testmag
  • MUD
  • RhythmGameZone
  • RSS
  • dabs
  • oklahoma
  • Socialism
  • KbinCafe
  • TheResearchGuardian
  • SuperSentai
  • feritale
  • KamenRider
  • All magazines
    •