[Ksummit-2013-discuss] [ATTEND] Kernel Logical Volume Management API

[Daniel Phillips]

Hi all,

Currently, the only way for a filesystem to implement advanced RAID 
techniques of the form that ZFS provides is to reimplement an entire 
volume manager within itself. We do not accept the argument that this is 
the only possible approach. We propose to provide kernel infrastructure 
and a kernel helper library to enable kernel filesystems to add 
customized volume management functionality using shared kernel 
infrastructure suited to the purpose.

This would be a ways and means discussion cutting across several kernel 
teams: Btrfs, which already provides advanced volume management 
capabilties that could possible be rebased upon a sufficiently powerful 
and efficient shared API; LVM team, which could contemplate this as a 
base for a next generation volume manager; Block API, which could 
seamlessly integrate with this functionality; LVM2/DM team, who built 
the nearest thing resembling an infrastructure like this; MD developers, 
who could possibly streamline their implementation; and the Tux3 team, 
which wishes to provide its own answer to the question posed by ZFS.

Discussion topics would include internal and external API style favoured 
by the various teams; required base functionality; performance goals; 
availability of developer resources and timeframe for development.

This is an area where Linux lags far behind the competition, that is, 
FreeBSD and even Windows. Little of substance has changed in the Linux 
storage stack since the days that the Device Mapper and BIO subsystems 
were invented, a long time ago. This topic cuts across several 
subsystems: various filesystems; the block layer; device mapper and md. 
This topic could also be addressed at the annual Storage and Filesystem 
summit, however that is nearly a year away. This issue is urgent enough 
that we ought to initiate a dialog on this topic and pursue it early and 
often, which is to say, both at Edinburgh and at the next FS Summit, and 
continue thereafter until we manage to bootstrap our storage 
infrastructure to a similar standard of excellence as other kernel 
subsystems.

Some preliminary patches exist as a possible discussion focus.

Regards,

Daniel