Skip to content

vllm.renderers.chat_utils.types

Classes:

AudioURL

Bases: TypedDict

Attributes:

  • url (Required[str]) –

    Either a URL of the audio or a data URL with base64 encoded audio data.

Source code in vllm/renderers/chat_utils/types.py 
57
58
59
60
61
class AudioURL(TypedDict, total=False):
    url: Required[str]
    """
    Either a URL of the audio or a data URL with base64 encoded audio data.
    """

url instance-attribute

Either a URL of the audio or a data URL with base64 encoded audio data.

ChatCompletionContentPartAudioEmbedsParam

Bases: TypedDict

Attributes:

  • audio_embeds (MultiModalEmbedsPayload | None) –

    The audio embeddings. It can be either:

  • type (Required[Literal['audio_embeds']]) –

    The type of the content part.

  • uuid (str | None) –

    User-provided UUID of a media. User must guarantee that it is properly

Source code in vllm/renderers/chat_utils/types.py 
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
class ChatCompletionContentPartAudioEmbedsParam(TypedDict, total=False):
    audio_embeds: MultiModalEmbedsPayload | None
    """
    The audio embeddings. It can be either:
    - A single base64 string representing a serialized torch tensor.
    - A dictionary of base64 tensors or numeric JSON metadata arrays.
    """
    type: Required[Literal["audio_embeds"]]
    """The type of the content part."""
    uuid: str | None
    """
    User-provided UUID of a media. User must guarantee that it is properly
    generated and unique for different medias.
    """

audio_embeds instance-attribute

The audio embeddings. It can be either: - A single base64 string representing a serialized torch tensor. - A dictionary of base64 tensors or numeric JSON metadata arrays.

type instance-attribute

The type of the content part.

uuid instance-attribute

User-provided UUID of a media. User must guarantee that it is properly generated and unique for different medias.

ChatCompletionContentPartAudioParam

Bases: TypedDict

Attributes:

Source code in vllm/renderers/chat_utils/types.py 
64
65
66
67
68
class ChatCompletionContentPartAudioParam(TypedDict, total=False):
    audio_url: Required[AudioURL]

    type: Required[Literal["audio_url"]]
    """The type of the content part."""

type instance-attribute

The type of the content part.

ChatCompletionContentPartImageEmbedsParam

Bases: TypedDict

Attributes:

  • image_embeds (MultiModalEmbedsPayload | None) –

    The image embeddings. It can be either:

  • type (Required[Literal['image_embeds']]) –

    The type of the content part.

  • uuid (str | None) –

    User-provided UUID of a media. User must guarantee that it is properly

Source code in vllm/renderers/chat_utils/types.py 
74
75
76
77
78
79
80
81
82
83
84
85
86
87
class ChatCompletionContentPartImageEmbedsParam(TypedDict, total=False):
    image_embeds: MultiModalEmbedsPayload | None
    """
    The image embeddings. It can be either:
    - A single base64 string.
    - A dictionary of base64 tensors or numeric JSON metadata arrays.
    """
    type: Required[Literal["image_embeds"]]
    """The type of the content part."""
    uuid: str | None
    """
    User-provided UUID of a media. User must guarantee that it is properly
    generated and unique for different medias.
    """

image_embeds instance-attribute

The image embeddings. It can be either: - A single base64 string. - A dictionary of base64 tensors or numeric JSON metadata arrays.

type instance-attribute

The type of the content part.

uuid instance-attribute

User-provided UUID of a media. User must guarantee that it is properly generated and unique for different medias.

ChatCompletionContentPartPromptEmbedsParam

Bases: TypedDict

Attributes:

Source code in vllm/renderers/chat_utils/types.py 
122
123
124
125
126
127
128
129
130
class ChatCompletionContentPartPromptEmbedsParam(TypedDict, total=False):
    data: Required[str]
    """
    Base64-encoded bytes of a serialized `torch.Tensor` of shape
    `(num_tokens, hidden_size)`. The tensor's `dtype` and `hidden_size` must
    match the model's input embedding layer.
    """
    type: Required[Literal["prompt_embeds"]]
    """The type of the content part."""

data instance-attribute

Base64-encoded bytes of a serialized torch.Tensor of shape (num_tokens, hidden_size). The tensor's dtype and hidden_size must match the model's input embedding layer.

type instance-attribute

The type of the content part.

ChatCompletionContentPartVideoEmbedsParam

Bases: TypedDict

Attributes:

  • type (Required[Literal['video_embeds']]) –

    The type of the content part.

  • uuid (str | None) –

    User-provided UUID of a media. User must guarantee that it is properly

  • video_embeds (MultiModalEmbedsPayload | None) –

    The video embeddings. It can be either:

Source code in vllm/renderers/chat_utils/types.py 
106
107
108
109
110
111
112
113
114
115
116
117
118
119
class ChatCompletionContentPartVideoEmbedsParam(TypedDict, total=False):
    video_embeds: MultiModalEmbedsPayload | None
    """
    The video embeddings. It can be either:
    - A single base64 string representing a serialized torch tensor.
    - A dictionary of base64 tensors or numeric JSON metadata arrays.
    """
    type: Required[Literal["video_embeds"]]
    """The type of the content part."""
    uuid: str | None
    """
    User-provided UUID of a media. User must guarantee that it is properly
    generated and unique for different medias.
    """

type instance-attribute

The type of the content part.

uuid instance-attribute

User-provided UUID of a media. User must guarantee that it is properly generated and unique for different medias.

video_embeds instance-attribute

The video embeddings. It can be either: - A single base64 string representing a serialized torch tensor. - A dictionary of base64 tensors or numeric JSON metadata arrays.

ChatCompletionContentPartVideoParam

Bases: TypedDict

Attributes:

Source code in vllm/renderers/chat_utils/types.py 
140
141
142
143
144
class ChatCompletionContentPartVideoParam(TypedDict, total=False):
    video_url: Required[VideoURL]

    type: Required[Literal["video_url"]]
    """The type of the content part."""

type instance-attribute

The type of the content part.

ConversationMessage

Bases: TypedDict

Attributes:

  • content (str | None | list[dict[str, str]]) –

    The contents of the message

  • name (str | None) –

    The name of the function to call

  • reasoning (str | None) –

    The reasoning content for interleaved thinking.

  • reasoning_content (str | None) –

    Deprecated: The reasoning content for interleaved thinking.

  • role (Required[str]) –

    The role of the message's author.

  • task (str | None) –

    Model-specific task marker. Currently passed through for DeepSeek V4.

  • tool_call_id (str | None) –

    Tool call that this message is responding to.

  • tool_calls (list[ChatCompletionMessageToolCallParam] | None) –

    The tool calls generated by the model, such as function calls.

  • tools (list[ChatCompletionFunctionToolParam] | None) –

    The tools for developer role.

Source code in vllm/renderers/chat_utils/types.py 
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
class ConversationMessage(TypedDict, total=False):
    role: Required[str]
    """The role of the message's author."""

    content: str | None | list[dict[str, str]]
    """The contents of the message"""

    tool_call_id: str | None
    """Tool call that this message is responding to."""

    name: str | None
    """The name of the function to call"""

    tool_calls: list[ChatCompletionMessageToolCallParam] | None
    """The tool calls generated by the model, such as function calls."""

    reasoning: str | None
    """The reasoning content for interleaved thinking."""

    reasoning_content: str | None
    """Deprecated: The reasoning content for interleaved thinking."""

    tools: list[ChatCompletionFunctionToolParam] | None
    """The tools for developer role."""

    task: str | None
    """Model-specific task marker. Currently passed through for DeepSeek V4."""

content instance-attribute

The contents of the message

name instance-attribute

The name of the function to call

reasoning instance-attribute

The reasoning content for interleaved thinking.

reasoning_content instance-attribute

Deprecated: The reasoning content for interleaved thinking.

role instance-attribute

The role of the message's author.

task instance-attribute

Model-specific task marker. Currently passed through for DeepSeek V4.

tool_call_id instance-attribute

Tool call that this message is responding to.

tool_calls instance-attribute

The tool calls generated by the model, such as function calls.

tools instance-attribute

The tools for developer role.

CustomChatCompletionContentPILImageParam

Bases: TypedDict

A simpler version of the param that only accepts a PIL image.

Example: { "image_pil": ImageAsset('cherry_blossom').pil_image }

Attributes:

  • uuid (str | None) –

    User-provided UUID of a media. User must guarantee that it is properly

Source code in vllm/renderers/chat_utils/types.py 
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
class CustomChatCompletionContentPILImageParam(TypedDict, total=False):
    """A simpler version of the param that only accepts a PIL image.

    Example:
    {
        "image_pil": ImageAsset('cherry_blossom').pil_image
    }

    """

    image_pil: PILImage | None
    uuid: str | None
    """
    User-provided UUID of a media. User must guarantee that it is properly
    generated and unique for different medias.
    """

uuid instance-attribute

User-provided UUID of a media. User must guarantee that it is properly generated and unique for different medias.

CustomChatCompletionContentSimpleAudioParam

Bases: TypedDict

A simpler version of the param that only accepts a plain audio_url.

Example: { "audio_url": "https://example.com/audio.mp3" }

Source code in vllm/renderers/chat_utils/types.py 
191
192
193
194
195
196
197
198
199
200
201
class CustomChatCompletionContentSimpleAudioParam(TypedDict, total=False):
    """A simpler version of the param that only accepts a plain audio_url.

    Example:
    {
        "audio_url": "https://example.com/audio.mp3"
    }

    """

    audio_url: str | None

CustomChatCompletionContentSimpleImageParam

Bases: TypedDict

A simpler version of the param that only accepts a plain image_url. This is supported by OpenAI API, although it is not documented.

Example: { "image_url": "https://example.com/image.jpg" }

Attributes:

  • uuid (str | None) –

    User-provided UUID of a media. User must guarantee that it is properly

Source code in vllm/renderers/chat_utils/types.py 
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
class CustomChatCompletionContentSimpleImageParam(TypedDict, total=False):
    """A simpler version of the param that only accepts a plain image_url.
    This is supported by OpenAI API, although it is not documented.

    Example:
    {
        "image_url": "https://example.com/image.jpg"
    }

    """

    image_url: str | None
    uuid: str | None
    """
    User-provided UUID of a media. User must guarantee that it is properly
    generated and unique for different medias.
    """

uuid instance-attribute

User-provided UUID of a media. User must guarantee that it is properly generated and unique for different medias.

CustomChatCompletionContentSimpleVideoParam

Bases: TypedDict

A simpler version of the param that only accepts a plain audio_url.

Example: { "video_url": "https://example.com/video.mp4" }

Attributes:

  • uuid (str | None) –

    User-provided UUID of a media. User must guarantee that it is properly

Source code in vllm/renderers/chat_utils/types.py 
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
class CustomChatCompletionContentSimpleVideoParam(TypedDict, total=False):
    """A simpler version of the param that only accepts a plain audio_url.

    Example:
    {
        "video_url": "https://example.com/video.mp4"
    }

    """

    video_url: str | None
    uuid: str | None
    """
    User-provided UUID of a media. User must guarantee that it is properly
    generated and unique for different medias.
    """

uuid instance-attribute

User-provided UUID of a media. User must guarantee that it is properly generated and unique for different medias.

CustomChatCompletionContentToolReferenceParam

Bases: TypedDict

A tool reference content param that only accepts a plain tool name.

Example: { "name": "get_weather", "type": "tool_reference" }

Attributes:

  • name (str) –

    The name of the tool being referenced.

  • type (Literal['tool_reference']) –

    The content type.

Source code in vllm/renderers/chat_utils/types.py 
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
class CustomChatCompletionContentToolReferenceParam(TypedDict, total=False):
    """A tool reference content param that only accepts a plain tool name.

    Example:
    {
        "name": "get_weather",
        "type": "tool_reference"
    }

    """

    name: str
    """The name of the tool being referenced."""

    type: Literal["tool_reference"]
    """The content type."""

name instance-attribute

The name of the tool being referenced.

type instance-attribute

The content type.

CustomChatCompletionMessageParam

Bases: TypedDict

Enables custom roles in the Chat Completion API.

Attributes:

  • content (str | list[ChatCompletionContentPartParam]) –

    The contents of the message.

  • name (str) –

    An optional name for the participant.

  • reasoning (str | None) –

    The reasoning content for interleaved thinking.

  • role (Required[str]) –

    The role of the message's author.

  • task (str | None) –

    Model-specific task marker. Currently passed through for DeepSeek V4.

  • tool_call_id (str | None) –

    Tool call that this message is responding to.

  • tool_calls (list[ChatCompletionMessageToolCallParam] | None) –

    The tool calls generated by the model, such as function calls.

  • tools (list[ChatCompletionFunctionToolParam] | None) –

    The tools for developer role.

Source code in vllm/renderers/chat_utils/types.py 
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
class CustomChatCompletionMessageParam(TypedDict, total=False):
    """Enables custom roles in the Chat Completion API."""

    role: Required[str]
    """The role of the message's author."""

    content: str | list[ChatCompletionContentPartParam]
    """The contents of the message."""

    name: str
    """An optional name for the participant.

    Provides the model information to differentiate between participants of the
    same role.
    """

    tool_call_id: str | None
    """Tool call that this message is responding to."""

    tool_calls: list[ChatCompletionMessageToolCallParam] | None
    """The tool calls generated by the model, such as function calls."""

    reasoning: str | None
    """The reasoning content for interleaved thinking."""

    tools: list[ChatCompletionFunctionToolParam] | None
    """The tools for developer role."""

    task: str | None
    """Model-specific task marker. Currently passed through for DeepSeek V4."""

content instance-attribute

The contents of the message.

name instance-attribute

An optional name for the participant.

Provides the model information to differentiate between participants of the same role.

reasoning instance-attribute

The reasoning content for interleaved thinking.

role instance-attribute

The role of the message's author.

task instance-attribute

Model-specific task marker. Currently passed through for DeepSeek V4.

tool_call_id instance-attribute

Tool call that this message is responding to.

tool_calls instance-attribute

The tool calls generated by the model, such as function calls.

tools instance-attribute

The tools for developer role.

CustomThinkCompletionContentParam

Bases: TypedDict

A Think Completion Content Param that accepts a plain text and a boolean.

Example: { "thinking": "I am thinking about the answer", "closed": True, "type": "thinking" }

Attributes:

Source code in vllm/renderers/chat_utils/types.py 
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
class CustomThinkCompletionContentParam(TypedDict, total=False):
    """A Think Completion Content Param that accepts a plain text and a boolean.

    Example:
    {
        "thinking": "I am thinking about the answer",
        "closed": True,
        "type": "thinking"
    }

    """

    thinking: Required[str]
    """The thinking content."""

    closed: bool
    """Whether the thinking is closed."""

    type: Required[Literal["thinking"]]
    """The thinking type."""

closed instance-attribute

Whether the thinking is closed.

thinking instance-attribute

The thinking content.

type instance-attribute

The thinking type.

PILImage

Bases: BaseModel

A PIL.Image.Image object.

Source code in vllm/renderers/chat_utils/types.py 
147
148
149
150
151
class PILImage(BaseModel):
    """A PIL.Image.Image object."""

    image_pil: Image.Image
    model_config = ConfigDict(arbitrary_types_allowed=True)

VideoURL

Bases: TypedDict

Attributes:

  • url (Required[str]) –

    Either a URL of the video or a data URL with base64 encoded video data.

Source code in vllm/renderers/chat_utils/types.py 
133
134
135
136
137
class VideoURL(TypedDict, total=False):
    url: Required[str]
    """
    Either a URL of the video or a data URL with base64 encoded video data.
    """

url instance-attribute

Either a URL of the video or a data URL with base64 encoded video data.