Extracting VLLM Metrics

[mohanajuhi166]

Does anyone know, how can I get the exposed metrics from /metrics if I run a custom scoring script and not use the open AI compatible server ?

# load the model
    model = LLM(
        model=model_path,
        tensor_parallel_size=1,
        max_model_len=MAX_TOKENS,
        max_seq_len_to_capture=MAX_TOKENS,
        max_num_batched_tokens=MAX_TOKENS,
        dtype="float16",
        otlp_traces_endpoint="<>",    can I pass any application insights string here ?
    )
    sampling_params = SamplingParams(
        temperature=TEMPERATURE, top_p=TOP_P, max_tokens=MAX_TOKENS
    )