FAQ and infos

ZFS

From a first view, ZFS is just another filesystem among others like ext4, XFS, HFS or NTFS but this is only half of the truth.

The inventor
Sun, one of the most innovative computer companies some years ago had the intention to create a complete new generation of Solaris, their Unix server operation system with a bunch of revolutionary ideas. They included projects like their own NFS and SMB server, Comstar as a framework for enterprise FC/iSCSI block storage, Dtrace for system optimization or Crossbow, a network virtualisation stack with virtual switches, virtual nics and vlan support. All included in Solaris with all what is needed for a server from one company - perfect integration.

No limits
The most important idea was ZFS as the base for some of these ideas. From the original idea ZFS should be a filesystem that is fully embedded into Solaris to overcome all traditional filesystem limits and problems like a limitation to a partition size or a fixed partition size. They created pooling and storage virtualisation where a filesystem has not a size but can dynamically grow without a reachable limit. Capacity handling is done by reservations and quotas.

No bitrot
As it was known that storage has a silent error/bitrot problem (data corruption by chance) that become a problem with larger capacities, they build in real data checksum protection end to end, from disk to disk driver to detect these corruptions and repair them from redundancy either on access or an online check of all files called scrubbing.

No Write holes
Next problem ZFS should be solve was data corruption due a crash during a write. Older systems first modify data on a write and update then affected metadata, On a crash it can happen that data is updated but metadata not resulting in a currupted filesystem. If this happens on a raid you may additionally find a corrupted raid too. This was the idea behind CopyOnWrite where an atomic write (data + metadata update) is done completely or not at all. This is achieved with a write behaviour where you never update old data but always write it newly. Only when the whole process reports success, the new state becomes valid, no danger of data curruption on a crash and therefor no checkdisk to repair corrupted metadata. After a successful write the old data is marked as available space.

Versioning
This write behaviour where you never overwrite data is the base of the ZFS versioning feature based on snaps. If you create a snap, all data blocks that are marked as available due a data update are simply marked as blocked by a snap.This is done without a delay or extra space consumption like other "Timemachine" alike features that require a copy of data.

Raid and Volumemanager
These are the filesystem features. Sun included software raid with pooling and raidmanagement/ volumemanagement into ZFS with the ability to build raid over any controller. This allows expandable datapools with hundreds of disks organized in raid-vdevs and Petabyte size with error management and hot spare functions.

Servicemanager for SMB, NFS and originally iSCSI
Last idea was to integrate sharing services like SMB, NFS or iSCSI into Solaris and ZFS. As Sun had full controll over these services including their own SMB server, the integration was perfect. As Sun wants to compete against Linux and Windows NT, they decided to open these technologies with OpenSolaris, the predecessor of the current commercial Oracle Solaris.

OpenSource
When Oracle bought Sun and closed the OpenSolaris Project, Illumos was founded to continue the OpenSolaris project with an OpenZFS development. and distributions like OpenIndiana, OmniOS, SmartOS anmd NexentaStor as a commercial storage options. Other Unix variants like BSD and OSX offere ZFS as well. Linux adds this lately because of license problems. While ZFS and Linux are under an OpenSource license, it is a problem to include ZFS as a kernel service. Therefor you must add ZFS under Linux as a seperate setup.

Everywhere
But most important, you can use ZFS now not only on Solaris and the OpenSource fork Illumos/OmniOS where it was developed for but on BSD, and Linux and OSX with real enterprise features for free.

napp-it 14.10.2024