I’m looking for storage classes for a multi node cluster. I’m currently using Longhorn and NFS, but I’m not happy with the performance. My cluster doesn’t have beefy nodes, so Ceph/Rook is out of the question (for now).

Nodes:

  1. 8 GB RAM, 4 cores VM, control plane. 256 GB SSD
  2. 4 GB RAM, 2 cores, control plane, currently cordoned. 128 GB SSD
  3. 8 GB RAM, 4 cores, ARM, control plane. 512 GB SSD
  4. 8 GB RAM, 4 cores. 256 GB SSD
  5. 16 GB RAM, 6 cores. 256 GB SSD + 1 TB HD
  6. RPi 4, 4 GB RAM. 128 GB SSD
  • custard_swollower@lemmy.worldEnglish
    3·
    12 hours ago

    I’m not sure you’ll get nice performance in local network with small appliances (consumer network hardware, mini PCs and rpi 4). I’ve never got sub-ms network disk access on 1Gbps switch and router. In the end I’ve done the opposite - I’ve added one k8s host with a lot of storage, and any storage services are deployed there. All the other k8s services rely on local SSDs.