domains / families / media / llava

llava

separate

Large Language and Vision Assistant. Enables visual instruction tuning and image-based conversations. Combines CLIP vision encoder with Vicuna/LLaMA language models. Supports multi-turn image chat, visual question answering, and instruction following. Use for vision-language chatbots or image understanding tasks. Best for conversational image analysis.

When to use

• Building vision-language chatbots
• Visual question answering (VQA)
• Image description and captioning
• Multi-turn image conversations
• Visual instruction following
• Document understanding with images
• **23,000+ GitHub stars**
• GPT-4V level capabilities (targeted)

When not to use

• **Hallucinations** - May describe things not in image
• **Spatial reasoning** - Struggles with precise locations
• **Small text** - Difficulty reading fine print
• **Object counting** - Imprecise for many objects
• **VRAM requirements** - Need powerful GPU
• **Inference speed** - Slower than CLIP

Skill page

Source