Log in

No account? Create an account

Rant about filesystems - Journal of Omnifarious

Mar. 11th, 2009

03:13 pm - Rant about filesystems

Previous Entry Share Next Entry

Here is a response I wrote to the Slashdot article titled: Apps That Rely On Ext3's Commit Interval May Lose Data In Ext4 . In particular, some people are trying to say that it is unreasonable of app makers to create thousands of small files and they ought to use a database like sqlite or something instead.

Filesystems that cannot handle thousands of tiny files efficiently are completely broken. I think the Linux filesystem people have been complete idiots for years for not considering this use case to be worth it. Too many big iron database vendors whispering in their ears apparently.

I want to be able to use the filesystem to appropriately name and reference my data. I do not want to have to rely on some completely different set of tools to actually see what data I have stored on my filesystem. If that's the case, I'll just use LVM for my 'filesystem' and use something vaguely decent to actually hold my data and use those tools instead of the Unix filesystem tools.

Now, those applications that are broken because they are written incorrectly should be re-written so they are correct and coincidentally god-awful slow on ext4. Then maybe the designers of ext4 will get a clue and actually write a filesystem instead of a glorified version of LVM with fancy hierarchical namespace for partitions instead of the the flat one LVM has.

Current Location: 1500 Dexter Ave N, 98109
Current Mood: [mood icon] aggravated


[User Picture]
Date:March 12th, 2009 12:57 am (UTC)
I’m pretty sure that understanding how POSIX works is a pretty good plan, if you’re developing software for that platform. For instance, both NFS and XFS don’t provide you the same guarantees either, which is why some software just doesn’t seem to work properly when your homedir uses those filesystems.

Creating many little files that you care about means that you need to fsync(fd) them. We’ve always known that, much like we’ve always known that ptr = realloc(ptr, size) is unsafe.

That doesn’t stop people from ignoring that advice, though. With any luck, mainstream adoption of a modern filesystem for Linux will encourage software developers to do the right thing.
(Reply) (Thread)