Why has vLLM such a strict schema definition?

[houmie]

I get 41 validation errors when posting a request with these fields to vLLM OpenAI endpoint:

41 validation errors for CompletionRequest
Extra inputs are not permitted [type=extra_forbidden, input_value='', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden

The list is long, is this normal that so many fields are actually not accepted by openAI compatible endpoint? Because some of these seem very important like max_new_tokens, rep_pen_range, top_a etc

Just making sure, I'm not doing something wrong:

api_server
api_type
ban_eos_token
custom_token_bans
do_sample
encoder_repetition_penalty
epsilon_cutoff
eta_cutoff
grammar_string
guidance_scale
legacy_api
max_new_tokens
max_tokens_second
min_length
mirostat
mirostat_eta
mirostat_mode
mirostat_tau
n_predict
negative_prompt
no_repeat_ngram_size
num_beams
penalty_alpha
rep_pen
rep_pen_range
repeat_last_n
repeat_penalty
repetition_penalty_range
sampler_priority
sampler_seed
smoothing_curve
smoothing_factor
stopping_strings
temperature_last
tfs
tfs_z
top_a
truncation_length
typical
typical_p
add_bos_token

[DarkLight1337]

I enforced the strictness of the schema in response to #4339.

[DarkLight1337]

Are there any plans to support smoothing_factor though?

Is it in the OpenAI API? I don't see it defined there.

[houmie]

Is it in the OpenAI API? I don't see it defined there.

I was considering it as an "Extra parameter". If you look at the existing supported parameters vLLM, you will find some very useful ones, such as min_p or top_k, which do not seem to be supported by the OpenAI specification. However, vLLM still supports them.

[DarkLight1337]

We do support certain vLLM-specific parameters via the Completions API.

vllm/vllm/entrypoints/openai/protocol.py

Lines 155 to 168 in e9d32d0

	# doc: begin-chat-completion-sampling-params
	best_of: Optional[int] = None
	use_beam_search: Optional[bool] = False
	top_k: Optional[int] = -1
	min_p: Optional[float] = 0.0
	repetition_penalty: Optional[float] = 1.0
	length_penalty: Optional[float] = 1.0
	early_stopping: Optional[bool] = False
	ignore_eos: Optional[bool] = False
	min_tokens: Optional[int] = 0
	stop_token_ids: Optional[List[int]] = Field(default_factory=list)
	skip_special_tokens: Optional[bool] = True
	spaces_between_special_tokens: Optional[bool] = True
	# doc: end-chat-completion-sampling-params

[houmie]

We do support certain vLLM-specific parameters via the Completions API.

Correct. So is there a chance to add smoothing_factor to this list as well? That would be amazing.
smoothing_factor is essentially Quadratic Sampling.

[DarkLight1337]

Since smoothing_factor isn't implemented in vLLM yet, I suggest you open an issue to request this feature.