domains / families / devops / vllm-server

vllm-server

authentic

Deploy and manage vLLM for high-throughput LLM inference. Configure continuous batching, tensor parallelism, quantization, and OpenAI-compatible API endpoints for production LLM serving.

When to use

• Serving open-source LLMs (Llama, Mistral, Qwen, Gemma) at scale
• Building an OpenAI-compatible API endpoint for self-hosted models
• Optimizing LLM throughput and latency for production traffic
• Running multi-GPU inference with tensor or pipeline parallelism
• Deploying quantized models to reduce GPU memory requirements

Skill page

Source