Smart Vision
Bridges pixels and words: an image encoder and a language decoder trained together, so the model can write a caption or answer a question about what it can see.
Nothing analysed yet
Add an image, choose what you want the model to produce, and run it.
BLIP pairs an image encoder with a language decoder. It writes one word at a time, looking back at the image patches at every step to decide what to say next.
The image is cut into patches and encoded into vectors. A language decoder then generates words one at a time, attending back to those patches at every step to decide what to say next.
Trained largely on web image–caption pairs, so it inherits their biases in who and what gets described, and how.
- Captions drift toward the generic. 'A man riding a horse' is a safe, high-probability sentence; specific details are riskier and so get produced less often.
- It will confidently describe things that aren't there if the scene resembles a common training example. This is hallucination, and it looks exactly like a correct answer.
- Ask a question the image cannot answer and it will still answer. There is no abstention.
This page says “the model predicts”, not “the model knows”. That is deliberate. None of these systems understand the images or sentences you give them; they map inputs to outputs using patterns fixed at training time. The difference matters most exactly when the output is impressive.