domains / families / media / blip-2

blip-2

separate

Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.

When to use

• Need high-quality image captioning with natural descriptions
• Building visual question answering (VQA) systems
• Require zero-shot image-text understanding without task-specific training
• Want to leverage LLM reasoning for visual tasks
• Building multimodal conversational AI
• Need image-text retrieval or matching
• **Q-Former architecture**: Lightweight query transformer bridges vision and language
• **Frozen backbone efficiency**: No need to fine-tune large vision/language models

When not to use

• instruction-following multimodal chat
• improved instruction-following
• production multimodal chat
• simple image-text similarity without generation
• few-shot visual learning

Skill page

Source