Open Source Block Storage Compared: Ceph RBD, LINSTOR/DRBD, Longhorn and ZFS (a vSAN Alternative)

The open source block storage systems that can replace VMware vSAN in 2026 are Ceph RBD, LINSTOR with DRBD, Longhorn, OpenEBS Mayastor and ZFS shared over iSCSI or NVMe-oF. Ceph is the only one that serves block, file and object storage from one cluster, and it is the default choice for OpenStack. LINSTOR is the lightest way to get replicated disks for KVM, and it runs on two data nodes. Longhorn and Mayastor only make sense inside Kubernetes. ZFS on a single storage server is the simplest of all, but that server is a single point of failure. GlusterFS still exists, but its commercial backer ended support at the end of 2024, so we would not start a new cluster on it.

Key facts, checked on 11 October 2026:

• Ceph has two supported releases: Tentacle 20.2.4 and Squid 19.2.6, both released on 19 August 2026. Squid reaches its estimated end of life on 31 October 2026 (Ceph release index).
• Ceph keeps 3 copies of every object by default (osd_pool_default_size = 3) and plans for 4 GiB of RAM per disk daemon (osd_memory_target), with total server RAM above twice that per disk (Ceph CPU and memory sizing).
• LINSTOR server 1.35.2 was released on 14 September 2026 (GitHub). The DRBD kernel module we installed from LINBIT's Ubuntu PPA was 9.3.4.
• Longhorn 1.13.0 came out on 29 September 2026 and needs Kubernetes 1.34 or later (release notes). Its documented minimum is 3 nodes with 4 vCPU, 4 GiB RAM and a 10 Gb/s network between them (Longhorn best practices).
• OpenEBS 4.6.2 (7 October 2026) ships Replicated PV Mayastor 2.12.2 (GitHub).
• OpenZFS 2.4.4 was released on 21 August 2026 (GitHub).
• Red Hat Gluster Storage reached end of life on 31 December 2024 (Red Hat life cycle). The newest upstream GlusterFS release is 11.2, from 2 July 2025 (GitHub).
• In our own lab, a 2-way DRBD mirror took sync 4K writes from 3.31 ms to 6.07 ms on average, and from 301 to 164 IOPS. The details are further down.

What vSAN did for you, and what you need back

vSAN pooled the disks inside your ESXi hosts into one datastore, kept two or more copies of each VM disk on different hosts, and let vMotion and HA restart a VM anywhere. When you leave VMware for OpenStack, plain KVM with libvirt, or Kubernetes, you need the same three things from something else:

• Shared block devices. A block device is a raw virtual disk that the guest formats itself, as opposed to a shared folder (file storage) or an S3 bucket (object storage).
• Replication, so one dead server does not lose data. Synchronous replication means a write is only confirmed once every copy has it.
• A driver for your platform: a Cinder driver for OpenStack, a CSI driver for Kubernetes, or a disk type that libvirt or Incus understands.

If you are moving virtual machines, the Cinder and libvirt columns in the table below matter more than any benchmark. Our VMware to OpenStack migration steps cover the move itself.

The seven candidates in plain words

Ceph RBD

Ceph spreads data in small objects across many disks on many servers. A placement algorithm called CRUSH decides where each object lives, so there is no central lookup table. RBD (RADOS Block Device) builds virtual disks on top of those objects. The same cluster can also export a POSIX file system (CephFS) and an S3-compatible API (RGW). It needs 3 monitor daemons to keep quorum, which means 3 servers is the honest minimum. It is licensed LGPL 2.1 or LGPL 3 (COPYING, checked 11 October 2026). Ceph has the widest support of anything here. OpenStack Cinder, Glance and Nova use it natively, Kubernetes uses it through ceph-csi or the Rook operator (Rook 1.21.0, 6 October 2026), libvirt and QEMU talk to RBD directly, and Incus has ceph, cephfs and cephobject drivers. The price is operations: placement groups, failure domains, RAM per disk and upgrades. Our three-VPS Ceph test lab builds one, and our RBD performance tests show what it costs in latency.

LINSTOR with DRBD

DRBD is a Linux kernel module that mirrors a block device to one or more other servers over TCP, much like a RAID 1 across the network. LINSTOR is the manager on top. You tell it "I want a 100 GB disk with 2 copies", and it carves LVM or ZFS volumes on two nodes and wires DRBD between them. Each volume is an ordinary LVM volume underneath, readable even with LINSTOR stopped. One node writes at a time (the Primary). Two data nodes are enough, plus a small "diskless" third node as a quorum tiebreaker. Licences: DRBD is GPL 2 and LINSTOR server is GPL 3 (both on GitHub, checked 11 October 2026). It has an in-tree Cinder driver ("LINBIT DRBD/LINSTOR" in the Cinder support matrix), a Kubernetes operator (Piraeus 2.12.0, 17 September 2026), and an Incus linstor storage driver. For plain libvirt, a DRBD device is just /dev/drbdNNNN, so you hand it to the VM as a block disk. We tested this one, see the lab below.

Longhorn

Longhorn runs as pods inside Kubernetes. Each volume gets its own small controller (the engine), and that engine writes every block to replicas on other nodes. It has a web UI, built-in snapshots and backups to S3 or NFS, and it is a CNCF incubating project (since 4 November 2021, CNCF). Licence: Apache 2.0. Shared (ReadWriteMany) volumes work through an NFS server placed in front of a block volume. There is no OpenStack or libvirt integration, so it fits Kubernetes, not VM migrations.

OpenEBS Replicated PV Mayastor

Mayastor is the replicated engine of OpenEBS. It is built on SPDK (a user-space storage toolkit) and exports volumes to pods over NVMe over TCP, the same protocol that fast all-flash arrays use. That design aims at low latency, but it reserves resources: the docs ask for 2 free CPU cores, 1 GiB RAM and 2 GiB of 2 MiB huge pages on every storage node, plus kernel 5.15 or later with the nvme-tcp module (OpenEBS prerequisites). Licence: Apache 2.0. Like Longhorn, it is for Kubernetes only.

ZFS zvols over iSCSI or NVMe-oF

This is the TrueNAS-style setup. One server with a ZFS pool creates zvols (block devices inside ZFS) and exports them over iSCSI or NVMe-oF to your hypervisors. You get checksums on every block, compression, snapshots and an asynchronous copy to a second box with zfs send. When the storage server dies, every VM on it stops until you fail over by hand. OpenZFS is CDDL licensed (META, checked 11 October 2026), which is why it ships as a separate module and not inside the Linux kernel. Cinder has no in-tree ZFS driver, so OpenStack users usually go through the LVM/iSCSI reference driver or NFS. Incus has a truenas driver, and libvirt has an iSCSI storage pool type.

GlusterFS

Gluster joins folders ("bricks") on several servers into one network file system. Upstream is not archived and commits still land, but Red Hat ended its product on 31 December 2024 and the last release is from July 2025. Kubernetes removed its in-tree GlusterFS driver in 1.26 (Kubernetes 1.26 release blog), and the Cinder matrix no longer lists one. Licence: GPL 2 or LGPL 3. If you run it today, plan the exit.

Which ones also do file and object?

Only Ceph serves block, file and object natively from one cluster. The others need a partner: an NFS server for file, and MinIO or a similar S3 server for object, as in our restic and MinIO backup guide. If S3 for Kubernetes is the real goal, read our Ceph RGW S3 guide.

The big comparison table

Versions and licences come from each project's official release page or licence file, checked on 11 October 2026. "Ops effort" is our editorial judgement, not a measured number.

What replication costs in latency

Every synchronous copy adds a network round trip and a remote disk write before the guest hears "done". A database that syncs each commit feels that directly, while bulk copies and reads barely notice. ZFS on one box adds only the iSCSI or NVMe-oF hop, but keeps no live second copy. DRBD adds one round trip to the peer, and reads stay local. Ceph writes to a primary disk daemon that forwards to two more, so the slowest of three disks sets your latency. We measured the DRBD case below. For Ceph numbers, see our RBD performance article, and for your own tests, our VPS benchmark commands.

Our lab: a two-node LINSTOR cluster in 2 GB VMs

We built this on 11 October 2026 on one of our Amsterdam test servers. Two KVM virtual machines ran Ubuntu 24.04 (kernel 6.8.0) with 2 vCPU and 2 GiB RAM each, plus an empty 8 GiB disk (/dev/sdb) per VM. The node IPs were 10.141.188.100 (wf-osbs-1) and 10.141.188.3 (wf-osbs-2). Both VMs shared one physical host with other test workloads, so treat the absolute numbers as a floor, not a target. The ratio between them is the useful part.

Step 1: install DRBD 9 and the LINSTOR satellite on both nodes

The DRBD module that ships inside the Ubuntu kernel is the old 8.4 branch, and LINSTOR needs DRBD 9. LINBIT publishes DRBD 9 for Ubuntu in a public PPA.

# as root, on both nodes
add-apt-repository -y ppa:linbit/linbit-drbd9-stack
apt update
apt install -y linux-headers-$(uname -r) drbd-dkms drbd-utils lvm2 linstor-satellite fio

The install took 145 seconds per node, mostly DKMS compiling the kernel module. On one node add-apt-repository timed out talking to Launchpad; copying the PPA's .sources file from the other node's /etc/apt/sources.list.d/ fixed it, since the file carries its signing key.

# as root, on both nodes
modprobe drbd
cat /proc/drbd

Expected output starts with version: 9.3.4 (api:2/proto:118-125). Our first try printed modprobe: ERROR: could not insert 'drbd': Key was rejected by service instead. That is UEFI Secure Boot refusing a module compiled on the machine. On our lab VMs we switched Secure Boot off in the VM settings and rebooted. On a server where Secure Boot must stay on, you enrol a machine owner key for DKMS instead, which we did not test here.

Step 2: give LINSTOR a thin pool on each node

# as root, on both nodes
pvcreate /dev/sdb
vgcreate vg_drbd /dev/sdb
lvcreate -L 7G -T vg_drbd/thin
systemctl enable --now linstor-satellite

Expected: Volume group "vg_drbd" successfully created and Logical volume "thin" created.

Step 3: the controller and the cluster (first node only)

# as root, on wf-osbs-1 only
apt install -y linstor-controller linstor-client
systemctl enable --now linstor-controller
linstor node create wf-osbs-1 10.141.188.100
linstor node create wf-osbs-2 10.141.188.3
linstor node list

Both nodes showed SATELLITE and Online. The client reported linstor-client 1.29.1 and linstor controller version reported 1.35.2.

# as root, on wf-osbs-1 only
linstor storage-pool create lvmthin wf-osbs-1 pool1 vg_drbd/thin
linstor storage-pool create lvmthin wf-osbs-2 pool1 vg_drbd/thin
linstor resource-group create rg-mirror --storage-pool pool1 --place-count 2
linstor volume-group create rg-mirror
linstor resource-group spawn-resources rg-mirror vm1disk 4G
linstor resource list

The spawn printed Resource 'vm1disk' successfully autoplaced on 2 nodes, and the resource list showed UpToDate on both. It also printed one warning worth reading: Could not find suitable node to automatically create a tie breaking resource for 'vm1disk'. With only two nodes there is nobody to break a tie if the link between them fails, so production setups add a third, diskless node. The disk now exists on both nodes as /dev/drbd1000.

# as root, on either node
drbdadm status

Before anything used the disk, both nodes were Secondary and the peer disk showed peer-disk:UpToDate.

Step 4: prove the second node has the data

# as root, on wf-osbs-1
mkfs.ext4 -q /dev/drbd1000
mkdir -p /mnt/vm1
mount /dev/drbd1000 /mnt/vm1
dd if=/dev/urandom of=/mnt/vm1/test.bin bs=1M count=100 status=none
sha256sum /mnt/vm1/test.bin
umount /mnt/vm1

# as root, on wf-osbs-2
mkdir -p /mnt/vm1
mount /dev/drbd1000 /mnt/vm1
sha256sum /mnt/vm1/test.bin

Both checksums matched (d60e0850...d47a), so the second node had every byte the first one wrote. DRBD promotes a node automatically when something opens the device. When we tried to mount it on the second node while the first still had it mounted, the mount failed with Wrong medium type. That single-writer rule protects the file system, which is what you want for a VM disk.

Step 5: one fio job, local versus mirrored

We ran the same fio job (fio 3.36) on a plain thin LVM volume and on the DRBD device. It writes 4K blocks with a sync after each one, the pattern a database commit log produces.

# as root, on wf-osbs-1
lvcreate -V 2G -T vg_drbd/thin -n baseline
fio --name=sync4k --filename=/dev/vg_drbd/baseline --rw=randwrite --bs=4k --iodepth=1 --direct=1 --sync=1 --ioengine=libaio --runtime=30 --time_based --size=1G
fio --name=sync4k --filename=/dev/drbd1000 --rw=randwrite --bs=4k --iodepth=1 --direct=1 --sync=1 --ioengine=libaio --runtime=30 --time_based --size=1G

In our lab the mirror roughly doubled sync write latency. Part of that is our setup: both VMs wrote to the same physical disk, so each mirrored write hit it twice. On two separate NVMe servers, the extra cost is mostly the network round trip. Memory was modest. The LINSTOR Java processes used about 229 MB and 315 MB (controller and satellite) on the first node and 222 MB on the second, and the first VM used 857 MiB of RAM in total.

Only LINSTOR and DRBD were lab-tested. Everything about Ceph, Longhorn, Mayastor, ZFS over iSCSI and GlusterFS here comes from official documentation and release pages.

Pick this if

Testing these on VPS and VDS servers first

You do not need a rack to choose. A KVM server runs its own kernel, so you can load and test storage modules like DRBD or ZFS on it, as we did in KVM VMs for this lab. For a LINSTOR test, 2 nodes with 2 GB RAM each are enough, as our lab shows, so a VPS Micro or VPS Mini per node works, plus a small third node as tiebreaker. For Ceph, follow the three-VPS Ceph lab; its RAM appetite makes a VDS Small (8 GB) per node the calmer choice. The VDS plans have 10 Gb/s ports, which matches the inter-node network Longhorn lists as a basic requirement. Send replication traffic between nodes through an encrypted tunnel, which our WireGuard guide sets up.

RS Computers runs KVM VPS and VDS servers in Amsterdam, Dublin and Prishtina, all on NVMe with unmetered traffic and their own IPv4 and IPv6. Availability by city is on the plans page.

Frequently asked questions

What is the best open source alternative to VMware vSAN?

For three or more servers under OpenStack or KVM, Ceph RBD is the most complete open source alternative to vSAN, because it replicates VM disks across hosts and also serves file (CephFS) and object (S3) storage. For two or three hosts, LINSTOR with DRBD is lighter and needs only two data nodes plus a small tiebreaker.

Which software-defined storage supports block, file and object storage?

Ceph is the open source system that supports all three from one cluster: RBD for block devices, CephFS for a shared POSIX file system and RGW for an S3-compatible object API. LINSTOR, Longhorn, Mayastor and ZFS are block-first and need a separate NFS or S3 server for the other two.

Can Ceph run on two nodes?

Not in a way we would recommend. Ceph needs a majority of its monitors to agree, and it keeps 3 copies of each object by default, so 3 nodes is the practical minimum. If you only have two servers, LINSTOR with DRBD plus a diskless third node for quorum is the better fit.

Is GlusterFS deprecated?

The upstream GlusterFS project is not archived and still receives commits, but Red Hat ended Red Hat Gluster Storage on 31 December 2024, the last release (11.2) is from July 2025, and Kubernetes removed its in-tree GlusterFS driver in version 1.26. It is not a good base for a new cluster in 2026.

Longhorn or Rook Ceph for Kubernetes?

Longhorn is easier: it installs as pods, has a web UI and backs up to S3, with a documented minimum of 3 nodes with 4 vCPU and 4 GiB RAM each. Rook Ceph is heavier but adds shared file systems and S3 object storage, so pick it when you need more than block volumes.

How much latency does replicated storage add?

Each synchronous copy adds at least one network round trip and a remote disk write. In our lab, a 2-copy DRBD mirror raised the average sync 4K write latency from 3.31 ms to 6.07 ms on a shared test host.

Build your test cluster

Choose your storage by testing it with your own workload, not by reading benchmark charts. Take two or three servers from our VPS and VDS plans, run the LINSTOR steps above or the Ceph lab, and put your own database or VM image on it. If you want help choosing node sizes or locations, or want us to quote the setup work, message us on Telegram.

← All articles