cluster01: SDS with CEPH and Rook.io
I've implemented this cluster in August 2018, using rook-ceph v0.8.0.
I used 3 virtual disk of 50GB on each of the 3 kvm guests (aka. as k8s cluster nodes). As expected, the Rook orchestrator for Ceph created 9 OSDs, 3 MONs and 1 MGR instance. From these I could have created block devices, S3 compatible storage objects and even filesystems.
But since I don't have a deep knowledge of Ceph and it's internals, I was lost with the Ceph CLI. What I'm really missing, is a web GUI to start with basic storage management tasks. I know, there is a dashboard included with Rook. But that is for Ceph monitoring only.
Maybe SUSE is going to fill that void with it's openATTIC based new efforts. See also: openATTIC and it's docker container on github.
Although using Ceph with Rook makes perfectly sense, I felt rather overwhelmed. That's why, I started looking around and found Rancher's Longhorn. Longhorn is a storage replicator that has been built with and for Kubernetes.
I'm probably giving Rook-ceph another try, when release 1.0 comes out.