Deploy Mooncake with CacheRuntime
Mooncake is a distributed KVCache store built for LLM inference workloads. Unlike cache systems such as Alluxio or JuiceFS, Mooncake does not provide POSIX mount semantics: applications talk to the cache service directly through its own client library instead of reading and writing files through a mount point.
Fluid's generic CacheRuntime supports this kind of cache system: the CacheRuntimeClass topology declares only the master and worker components, and omits the client component that would otherwise be responsible for mounting. This document walks through that minimal setup.
For background on the client-less architecture, see the Generic Cache System Integration Guide.
Prerequisites
Before running this example, follow the Installation Guide to install Fluid, and verify that its components are running:
$ kubectl get pod -n fluid-system
NAME READY STATUS RESTARTS AGE
cacheruntime-controller-58c775584-x9q47 1/1 Running 0 10m
csi-nodeplugin-fluid-74bpw 2/2 Running 0 10m
csi-nodeplugin-fluid-cjvc6 2/2 Running 0 10m
csi-nodeplugin-fluid-x4wwp 2/2 Running 0 10m
dataset-controller-58497b968b-q6ssv 1/1 Running 0 10m
fluid-webhook-74db64fdf4-7rq69 1/1 Running 0 10m
fluidapp-controller-7dbdc7696b-86pph 1/1 Running 0 10m
Typically you should see one pod each for dataset-controller, cacheruntime-controller, fluid-webhook and fluidapp-controller, plus one csi-nodeplugin pod per node. cacheruntime-controller is the one this example requires; it is enabled with --set runtime.cacheruntime.enabled=true when installing the Helm chart.
Note: this example requires a Fluid build that includes #6157. On earlier versions the controller panics when the client component is omitted from
topology.
The demo image
Fluid does not publish a Mooncake image. This example uses a demo image built from the Mooncake Python distribution plus two small scripts that Fluid invokes:
| Path | Purpose |
|---|---|
/custom-entrypoint.sh | Component entrypoint; starts the right process based on the role (master/worker) |
/reportSummary.sh | Collects cache usage and emits it as JSON in the format Fluid expects |
The build context for that image lives in the Fluid repository under samples/mooncake/docker, so you can build an equivalent image yourself:
$ docker build -t <your-registry>/mooncake:v3 samples/mooncake/docker
$ docker push <your-registry>/mooncake:v3
Note that apt-get and pip resolve to whatever versions are current at build time, so a fresh build produces a functionally equivalent image, not a bit-for-bit reproduction of the digest below.
Building your own image and substituting it in the manifests below is the recommended path. For convenience, a prebuilt copy is also available, pinned to an immutable digest so this example keeps working even if the tag is moved:
btxu/mooncake:v3@sha256:067614b70d25b496e3edc3480747d558ee8a364ef47a67f669f5d96ca5098552
An Alibaba Cloud mirror is available for networks in mainland China. It is a copy of the same manifest, so it carries the identical digest:
crpi-4hkqof7tc9brc6d5.cn-hongkong.personal.cr.aliyuncs.com/mooncake1314/mooncake:v3@sha256:067614b70d25b496e3edc3480747d558ee8a364ef47a67f669f5d96ca5098552
Note: both registries are personal accounts belonging to the author of this example, not project-controlled infrastructure, and they carry no availability guarantee. Treat them as a convenience for trying the example out, and build from
samples/mooncake/dockerfor anything beyond that.
Neither script is specific to this image: you can build an equivalent image on top of any Mooncake distribution, as long as it provides the same two entry points. See the conventions in the Generic Cache System Integration Guide.
Running the Example
Create the CacheRuntimeClass
Review the CacheRuntimeClass to be created
$ cat<<EOF >mooncake-cacheruntimeclass.yaml
apiVersion: data.fluid.io/v1alpha1
kind: CacheRuntimeClass
metadata:
name: mooncake-demo
fileSystemType: mooncakefs
topology:
master:
service:
headless: {}
executionEntries:
reportSummary:
command:
- bash
- -c
- /reportSummary.sh
timeout: 30
template:
spec:
restartPolicy: Always
containers:
- name: master
image: btxu/mooncake:v3@sha256:067614b70d25b496e3edc3480747d558ee8a364ef47a67f669f5d96ca5098552
command:
- /custom-entrypoint.sh
args:
- master
- start
imagePullPolicy: IfNotPresent
readinessProbe:
tcpSocket:
port: 50051
initialDelaySeconds: 10
periodSeconds: 5
failureThreshold: 12
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
ports:
- containerPort: 50051
name: rpc
- containerPort: 8080
name: metadata
- containerPort: 9003
name: metrics
worker:
service:
headless: {}
template:
spec:
restartPolicy: Always
containers:
- name: worker
image: btxu/mooncake:v3@sha256:067614b70d25b496e3edc3480747d558ee8a364ef47a67f669f5d96ca5098552
command:
- /custom-entrypoint.sh
args:
- worker
- start
imagePullPolicy: IfNotPresent
readinessProbe:
tcpSocket:
port: 50052
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 12
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
ports:
- containerPort: 50052
name: data
- containerPort: 9300
name: http
EOF
CacheRuntimeClass is a CRD defined by Fluid that describes how a class of cache system runs on Kubernetes — the image each component uses, how it starts, which ports it exposes, and how Fluid collects its runtime status.
A few things worth noting in this example:
- Only
masterandworkerare declared undertopology. All three components (master, worker, client) are optional in the API, and Mooncake does not need a client component. - Note that
fileSystemTypeandtopologyare top-level fields — they do not live underspec. - The master component exposes three ports:
50051for RPC,8080for the metadata service, and9003for metrics. Applications connect directly to the first two. reportSummarypoints at/reportSummary.shinside the image. Fluid executes it periodically in the component pod and writes the result to the Dataset'sstatusfield. For the required output format, see the Generic Cache System Integration Guide.
Create the CacheRuntimeClass
$ kubectl apply -f mooncake-cacheruntimeclass.yaml
cacheruntimeclass.data.fluid.io/mooncake-demo created
$ kubectl get cacheruntimeclass
NAME AGE
mooncake-demo 0s
Tip: if you modify the CacheRuntimeClass afterwards, you must delete and recreate the CacheRuntime for the change to take effect. A CacheRuntime renders the CacheRuntimeClass into workloads at creation time; later edits are not applied retroactively, and deleting the pods alone does not help.
Create the Dataset and CacheRuntime
Review the resources to be created
$ cat<<EOF >mooncake-dataset-runtime.yaml
apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
name: mooncake-demo
namespace: default
spec:
placement: Shared
accessModes:
- ReadWriteMany
mounts:
- name: mc
mountPoint: "mooncakefs:///"
---
apiVersion: data.fluid.io/v1alpha1
kind: CacheRuntime
metadata:
name: mooncake-demo
namespace: default
spec:
runtimeClassName: mooncake-demo
master:
replicas: 1
worker:
replicas: 2
tieredStore:
levels:
- emptyDir:
quota: 1Gi
high: "0.8"
low: "0.5"
EOF
The Dataset describes the dataset itself, while the CacheRuntime describes the runtime instance providing cache service for it — here, one master replica and two worker replicas.
About the mounts field: Mooncake does not mount any underlying storage (UFS); data is written directly into the cache by the client, so mooncakefs:/// here is only a placeholder mount point. Fluid passes the contents of mounts through into the CacheRuntime config ConfigMap for the component containers to read, but because this example's CacheRuntimeClass declares no mountUfs execution entry, Fluid never performs an actual mount (see the notes on skipping MountUFS in the Generic Cache System Integration Guide). In the API mounts is optional, but if you do declare it, it must contain at least one entry.
Create the resources
$ kubectl apply -f mooncake-dataset-runtime.yaml
dataset.data.fluid.io/mooncake-demo created
cacheruntime.data.fluid.io/mooncake-demo created
Check the Dataset status
$ kubectl get dataset
NAME UFS TOTAL SIZE CACHED CACHE CAPACITY CACHED PERCENTAGE PHASE AGE
mooncake-demo NotBound 6s
As shown above, PHASE is NotBound, meaning the Dataset is not yet bound to a cache runtime. Wait for the components to start:
$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mooncake-demo-master-0 1/1 Running 0 15s 10.244.2.13 fluid-mooncake-worker <none> <none>
mooncake-demo-worker-0 1/1 Running 0 15s 10.244.2.14 fluid-mooncake-worker <none> <none>
mooncake-demo-worker-1 1/1 Running 0 15s 10.244.1.7 fluid-mooncake-worker2 <none> <none>