Should we move stateful services to a managed Kubernetes and watch pv perf?

[LukeUL]

I’m trying to decide if we should move our stateful services to a managed Kubernetes service, but I’m worried about persistent volume performance and cost predictability. Our current setup uses self-hosted storage, and I’m not sure if the abstraction of cloud-native storage will meet our latency requirements for the database workloads.

[Dennis17]

We moved a couple of stateful services to a managed Kubernetes service a few months back. After we picked a high performance storage class and provisioned IOPS, latency stayed in the low tens of milliseconds for reads and a bit higher for writes. It felt acceptable for our workload but the cost predictability part was rough. Once you ramp IOPS and throughput the monthly bill can swing. We added a small in app cache to smooth hot paths and take some pressure off the storage tier.

[CharlesL]

On the flip side we tried and the latency did not improve as hoped. Tail latencies would spike during backups and bursts unless we paid for more IOPS. That persistent volume setup never felt predictable, invoices jumped month to month. We even debated keeping some workloads on a traditional on premises array, but that kind of defeated the point of the move.

[JackQP]

Backup windows felt like a separate beast and restore times could be a headache in tests. Then DR planning dragged in cross region replicas and egress costs. All this while trying to keep the developers productive and not breaking SLAs.

[Oliver13]

Have you tried running a small pilot with one service to measure actual latency and cost before a full move?