domains / families / data / huggingface-local-models

huggingface-local-models

authentic

Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving.

When to use

• selecting models to run locally with llama.cpp
• finding GGUFs on Hugging Face Hub
• quant selection
• running servers
• exact GGUF file lookup
• conversion from Transformers weights
• OpenAI-compatible local serving

Skill page

Source