Structured Decisions Models¶

Structured Decisions Models (aka JEV Models) map inputs directly to predefined options or categories and return probabilities and confidence scores, commonly used for classification and routing.

Supported Models¶

Text-only Models¶

Architecture	ReadStrategy	Task	Head	Example HF Models
`ClefFlashForJointSchema`	`ClefFlashStrategy`	`token_classify`	`JointSchemaHead`	`Cloudflare/clef-flash`
`Qwen3ForCausalLM`	`NextTokenStrategy`	`generate`	N/A	`Qwen/Qwen3-0.6B`, etc.

Multimodal Models¶

Note

For more information about multimodal models inputs, see this page.

Architecture	Inputs	ReadStrategy	Task	Head	Example HF Models
`ClefOmniForJointSchema`	T + I + V	`JointSchemaStrategy`	`token_classify`	`JointSchemaHead`	`Cloudflare/clef-omni`
`Lfm2VlForConditionalGeneration`	T + I⁺	`LiquidStrategy`	`generate`	N/A	`LiquidAI/d1-3B`
`Qwen3_5ForConditionalGeneration`	T + I^E+ + V^E+	`NextTokenStrategy`	`generate`	N/A	`Qwen/Qwen3.5-0.8B`, etc.
`Qwen3_5MoeForConditionalGeneration`	T + I^E+ + V^E+	`NextTokenStrategy`	`generate`	N/A	`Qwen/Qwen3.5-35B-A3B`, etc.
`DiffusionGemmaForBlockDiffusion`	T + I⁺ + V⁺	`DiffusionGemmaCanvasStrategy`	`generate`	N/A	`google/diffusiongemma-26B-A4B-it`, etc.

Online Serving¶

TypeSafe System One Endpoint¶

The /v1/systemone endpoint answers a set of typed questions about a state and returns a probability for every allowed answer.

The endpoint is available by default on generation models. Models without a supported read strategy return 501. Supported architectures require --logprobs-mode raw_logprobs or processed_logprobs, since label_mass needs log probabilities.

How it works¶

Each question is one read. The user message is the state, followed by the question and its options with the type's labels, and an instruction to answer with one label.
The read is the reply's first token. Each label must be one token there, or the request returns 400.
The read returns the logprobs of that question's label tokens. Softmax over the labels gives the answer's probabilities.

Thinking is off unless chat_template_kwargs turns it on.

Probabilities can differ between identical requests. They depend on which requests share a batch and on whether a read hits the prefix cache. Set VLLM_BATCH_INVARIANT=1 for reads that repeat exactly, at a cost in throughput.

Every read of a request starts with the state, so with --enable-prefix-caching the state is prefilled once and each further question prefills only its own text, then reads one token.

Question types¶

type	criteria	answer
`choice`	map of option name to a description or `null`	`choice`, `probabilities` by option name, `confidence`
`noul`	`null`, or an object with optional `true` and `false` descriptions	`noul` (probability of yes), `probabilities`, `confidence`
`score`	ordered list of level names	`score` (expected 0-indexed level), `legend`, `probabilities` by level index, `confidence`

A choice has at least one option, a score at least two levels. A question id is a non-empty string.

Example¶

curl -s http://localhost:8000/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "state": {"ticket": "My card was charged twice for one order."},
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which team should handle this ticket?",
        "criteria": {
          "billing": "payments and refunds",
          "shipping": "deliveries",
          "security": "account access"
        }
      },
      "urgent": {
        "type": "noul",
        "instructions": "Does the customer need a reply within the hour?",
        "criteria": {"true": "the account is locked or money is moving"}
      },
      "tone": {
        "type": "score",
        "instructions": "How upset is the customer?",
        "criteria": ["calm", "annoyed", "furious"]
      }
    },
    "chat_template_kwargs": {"enable_thinking": false}
  }'

Response, with probabilities that depend on the model:

{
  "id": "decision-...",
  "object": "structured_decision",
  "created": 1790000000,
  "model": "Qwen/Qwen3-0.6B",
  "answers": {
    "team": {
      "type": "choice",
      "choice": "billing",
      "probabilities": {"billing": 0.97, "shipping": 0.01, "security": 0.02},
      "confidence": 0.90
    },
    "urgent": {
      "type": "noul",
      "noul": 0.02,
      "probabilities": {"yes": 0.02, "no": 0.98},
      "confidence": 0.95
    },
    "tone": {
      "type": "score",
      "score": 0.6,
      "legend": {"0": "calm", "1": "annoyed", "2": "furious"},
      "probabilities": {"0": 0.7, "1": 0.2, "2": 0.1},
      "confidence": 0.65
    }
  },
  "usage": {"input_tokens": 142, "output_tokens": 3},
  "diagnostics": {
    "team": {"label_mass": 0.93, "argmax_is_label": true},
    "urgent": {"label_mass": 0.98, "argmax_is_label": true},
    "tone": {"label_mass": 0.91, "argmax_is_label": true}
  }
}

diagnostics.label_mass is the probability the model put on the labels across its whole vocabulary. A low value means most of the probability went to tokens that are not labels. confidence is the chosen option's probability over the whole vocabulary: its share of the labels times label_mass.

Labels¶

Labels follow the type: a choice labels its options A to Z in the order the request lists them, a noul labels its options yes and no, and a score labels its levels 0 to 9, so a level's label is its score.

Limits¶

limit	value
questions per request	64
options per `choice`	26
levels per `score`	10

Request fields¶

field	meaning
`state`	what the questions are about: a string, or JSON that is sent as its JSON text
`questions`	question id to `{type, instructions, criteria}`, asked in this order
`instructions`	optional context placed ahead of the questions
`chat_template_kwargs`	passed to the chat template, for example `{"enable_thinking": false}`
`cache_salt`	optional prefix-cache salt applied to every question
`images`, `audio`, `videos`	optional media URLs or data URLs, for models that take them (Clef-Flash, images/video; Clef-Omni; Liquid d1, images only)

A question with any field other than type, instructions and criteria is rejected with a 400.

When tracing is enabled with --otlp-traces-endpoint, the HTTP traceparent and tracestate headers are forwarded to every question's generation request.

OpenAI Decisions API¶

vLLM provides a text-only implementation of OpenAI's POST /v1/decisions API. Start a supported model:

vllm serve Qwen/Qwen3-0.6B

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
decision = client.decisions.create(
    model="Qwen/Qwen3-0.6B",
    input="The package arrived with a broken screen.",
    questions=[
        {
            "type": "predicate",
            "name": "damaged",
            "instructions": "Does the customer report a damaged item?",
        },
        {
            "type": "choice",
            "name": "department",
            "instructions": "Which department should handle this complaint?",
            "choices": [{"value": "returns"}, {"value": "billing"}],
        },
        {
            "type": "score",
            "instructions": "How severe is the damage?",
            "levels": [{"label": "cosmetic"}, {"label": "broken"}],
        },
    ],
)
print(decision.answers)

Use OpenAI Python SDK 3.26.0 or newer for client.decisions.create. input accepts a string or an array of user messages with string content or input_text parts. Answers retain question order and echo name, including null for unnamed questions. Choice values preserve strings and booleans. Scores are probability-weighted averages of zero-based level indices.

The request contract follows the OpenAI OpenAPI schemas retrieved on October 7, 2026. This MVP supports 1–64 questions, 2–26 choices, and 2–10 score levels. Upstream permits 200 questions and 255 choices; this backend returns a 400 when either model limit is exceeded. The optional safety_identifier is caller metadata, available in request body logging with --enable-log-requests and DEBUG logging; it does not authenticate callers.

Implementation and limits¶

This MVP uses the same label-reading backend as structured decisions. It supports generation architectures with a registered read strategy, including Qwen, DiffusionGemma, and Liquid d1; unsupported models return 501. Each question reads a model's label tokens. Usage reports the actual prompt and output tokens across these reads, including prefix-cache counters.

Probabilities are normalized over the supplied options. confidence is the winning label's probability over the full vocabulary; it is not calibrated to OpenAI's models. Thinking is disabled for these reads.

Image inputs, other message roles, tools, streaming, and per-question refusal scoring are outside this MVP. Unsupported request fields and input types are rejected. The /v1/systemone endpoint retains its existing request format and is available by default.

Examples¶

Clef-Flash¶

Clef-Flash uses a Qwen3.5-9B backbone and a joint schema head to score all questions in one forward pass. vLLM detects the head and serves the checkpoint as a pooling model.

vllm serve Cloudflare/clef-flash --max-model-len 16384

Send requests to /v1/systemone using state and questions, as in the examples above. choice, noul and score questions are supported.

Text and JSON states use the checkpoint's fixed prompt. JSON is compact with sorted keys. instructions returns 400; chat_template_kwargs has no effect.
The state is truncated to fit 16,384 tokens, or max_model_len if smaller. A schema that exceeds this bound returns 400.
images and videos are placed before the state. Videos are sampled at 2 fps and processed by the Qwen3.5 vision processor. Audio inputs return 400, and video soundtracks are ignored.
Each request performs one pooling read. usage.input_tokens counts the prompt once, output_tokens is 0, and label_mass is 1.

Clef-Omni¶

Clef-Omni is a decision model: a joint schema head on the Qwen3-Omni thinker scores every option of every question in one forward pass. vLLM serves it as a pooling model.

vllm serve Cloudflare/clef-omni

A request is one read. The prompt is the one the checkpoint's joint_schema_model.py builds: the state, then every question with its options. A JSON state is written as compact JSON with sorted keys.
A state too long for the model is truncated to fit, as the reference does. A request whose questions alone exceed 64,000 tokens, or max_model_len, returns 400.
instructions returns 400, since the prompt is fixed. chat_template_kwargs has no effect.
There are no labels, so label_mass is 1 and confidence is the chosen option's probability. usage.input_tokens counts the prompt once, and output_tokens is 0.
images, audio and videos are read as the reference does: placed before the state, videos sampled at 2 fps, and a video's soundtrack heard with it when every video has one. Audio clips cannot be sent together with videos that have soundtracks.

Liquid d1¶

LiquidAI/d1-3B is LFM2.5-VL-3B post-trained to answer decisions as the first token of its reply. vLLM serves it as a generative model.

vllm serve LiquidAI/d1-3B

The server writes the prompt and reads the label tokens the way the prompt.py shipped with the checkpoint does, so picks and probabilities match model.system_one(). confidence and usage follow the definitions on this page instead.
A JSON state is written as indented JSON. A null state leaves the images, or the question alone, as the prompt.
images are placed before the state and shrunk to at most 1024 x 1024 pixels, as the reference does.
A choice whose option names are all single letters is labeled with those letters.
When each request carries one question over a state that changes from request to request, --no-enable-prefix-caching lowers latency.