Recommendations for replacing a GFS cluster?

Ceph and GlusterFS is where the clustered FS technology is currently going. Since I'm not familiar with GlusterFS I'll speak on the features of Ceph.

Ceph scales horizontally; the more low-end nodes you add to it the better the performance. Unlike GlusterFS this is a primary benefit to Ceph as it shards the data.

However, Ceph is in active development (it's production ready except for Ceph FS) and requires a modern kernel (as I write this not even CentOS 6.5 default kernel can take advantage of RBD/CephFS features). To get around this I installed ELRepo kernel-lt.

To break it down for you:

• Cephs RBD is a clustered SAN replacement; you can create "virtual" devices that are in the cluster and can be mounted on servers. Note: Only one server should have an RBD image mounted at one time (you don't want multiple OS's mounting a SATA drive do you?). You would then format the RBD disk, mount it like normal, and then have NFS/CIFS make it available. If the server providing NFS/CIFS goes down no data is lost.
• Ceph FS is a clustered NAS replacement (although not production ready); it provides file locking capabilities that are needed for a clustered FS that is shared between servers (such as a web server for example).

RBD runs in kernel space; so there is no fuse performance hits. Ceph FS also runs in kernel space but can run with FUSE.

Ceph is also pretty easy to deploy:

1. pip install ceph-deploy on an admin node (your desktop/workstation).
2. Add Inktank RPM repos and ceph-deploy install node1 node2 ... nodeN to install ceph on all of the nodes.