From Reiser3 to Reiser4 and Uncertainty Beyond

Reiser3 was the first Linux filesystem with journaling and received mass adoption in the period of 2001-2006. Reiser4 is designed with lessons learned from the mistakes in Reiser3. It is still the fastest Linux filesystem.  That is without filesystem compression. Reiser4 has been around since 2001, but it has never been accepted into the Linux kernel. It is a testament to the failure of a free society to recognize greatness. The Linux kernel maintainers are still human.

Before Namesys

Reiser was about 12 years old in high school when he was accepted into Berkeley. He studied on and off, working at companies like IBM, before earning a degree at middle age. The development of Reiser3 and Reiser4 can be traced to his senior thesis, in which he studied the difference between computer science and empirical sciences. As a result, Reiser3 and Reiser4 were exceptionally fast filesystems, because ideas conventionally regarded in computer science as junk were given a second chance.

To start Namesys, Reiser went to Russia in 1993 and hired a team of programmers. Russia had just come out of a huge failure. To summarize the period:

The Russians are just taking into their hands the ashes of incompetent industry, collectivized farming, Stalinist paranoia, military machoism, the Gulags, Leninist madness…

They turned out to be the best programmers that money could buy, full of bright ideas.

Reiser3 (ReiserFS)

When I used Windows 95/98, I remember those colorful screens and the long wait. It turned out the OS was checking the filesystem, because it didn’t shut down properly. So is there a way to make it start faster and not loose any data? Reiser3 was the first Linux filesystem to implement journaling, one method to solve both problems. With Reiser3, data centers did not have to worry about loosing power, even just for 1 second.

Major Linux distributions all offer Reiser3. SourceForge was hosted using  it, and Novell made it their default filesystem. Since then, there has been a decline due to kernel locks used extensively, resulting in non-concurrent operation on multi core systems. However, there are patches, because Reiser3 users still haven’t switched.

Reiser3 development was funded by DARPA and Linspire. Another source of funding when it was included with the kernel was database implementations.

Reiser4

Reiser4 is a non-continuous increment of Reiser3. A Reiser3 filesystem cannot be converted to a Reiser4 one. This decision in itself reflects how continuous innovations are adopted quicker. For example, C/C++ are almost synonymous compared to C# and Objective-C. Switching to Reiser4 means compiling a kernel with it, formatting another partition large enough, and copying all the data.

However, the benefits of being non-continuous outweighs the costs. Reiser4 is designed to support an on-disk database, within the filesystem. This is implemented by using two separate layers, graphs for relationships, and trees for efficient searching. Reiser4 also implements extents, mapping a section of a disk to a file, which makes fragmentation non-existent.

Another feature is the plugin system, where a userland SQL interface can be placed. Kernel developers argued for moving it to VFS, while Reiser insists on keeping it as part of the filesystem. There are many other reasons why Reiser4 was not accepted into the kernel. The plugin system allows compression and encryption of the entire on-disk structure. LZO compression increases filesystem performance by two times, placing Reiser4 head and shoulders above the rest.

Whereas Reiser3 became the standard journaling filesystem,  Reiser4 had no special feature associated with it. Being the best filesystem at the time, Reiser4 was never included in the kernel. Compared with ext3, which still used an H-tree and had a double write penalty for journals, Reiser4 was better technology-wise. If Reiser3 was adopted for data-loss protection, then Reiser4 needed a similar reason. An interesting application for the graph layer is a Baynesian network. It would have high scalability and performance due to its on-disk structure, and could be sold as an out-of-box solution, like Google Search for Enterprise. Such a product would have applications in real-time business analytics, data warehousing, and as a component of a CRM system.

The End

Now as Reiser4 has been forgotten for 4 years, you may wonder how the story ended. Sadly, the story ends with a telltale heart, which I saw as a play and now in real life. Reiser must have liked Lord of the Rings a lot, since he used metaphors in his talks. It’s usually popular with Dungeons and Dragons players, or Linux fans. The terminal is another way of seeing the world.