Google says GKE Pod snapshots can reduce startup latency by as much as 89%, restoring a 70-billion-parameter model in 37 seconds and an 8-billion-parameter model in 15 seconds. The feature became generally available in May for sufficiently recent Google Kubernetes Engine clusters.
This is checkpoint-and-restore rather than an ordinary file cache. A snapshot preserves CPU and GPU memory, open files, threads, registers, the container filesystem and temporary volumes. A new replica resumes from that state instead of repeating model initialization, often the slowest part of launching an inference workload.
The mechanism relies on gVisor, so Pods must run in GKE Sandbox. Autopilot includes that environment; Standard clusters need a gVisor-enabled node pool. Snapshot data lives in Cloud Storage, while Kubernetes resources define the bucket, eligible Pods, trigger and retention policy. Codeway reported cutting startup on an H100 workload from about a minute with its own cache to eight seconds.
Faster restoration creates a lifecycle challenge. A saved state is useful only while its application specification, machine family, kernel and drivers remain compatible. Teams must manage invalidation, retention, storage cost and security of memory snapshots rather than treating the feature as a transparent speed switch.