Skip to content

vllm.v1.kv_offload.cpu.common

Classes:

CPULoadStoreSpec

Bases: BlockIDsLoadStoreSpec

Spec for loading/storing KV chunks to/from CPU memory.

The inherited block_ids field holds chunk indices into the CPU cache (not GPU block IDs). The chunk_ids alias exposes the same array under the name used by the tiering layer.

Source code in vllm/v1/kv_offload/cpu/common.py 
18
19
20
21
22
23
24
25
26
27
28
class CPULoadStoreSpec(BlockIDsLoadStoreSpec):
    """Spec for loading/storing KV chunks to/from CPU memory.

    The inherited block_ids field holds chunk indices into the
    CPU cache (not GPU block IDs). The chunk_ids alias exposes
    the same array under the name used by the tiering layer.
    """

    @property
    def chunk_ids(self) -> np.ndarray:
        return self.block_ids

CPUOffloadingInfo dataclass

Bases: ConfigInfo

Static facts about the CPU offload tier.

Source code in vllm/v1/kv_offload/cpu/common.py 
31
32
33
34
35
36
@dataclass(frozen=True)
class CPUOffloadingInfo(ConfigInfo):
    """Static facts about the CPU offload tier."""

    # Chunk slots in the tier. Chunks, not GPU blocks.
    cpu_num_chunks: int