language-models

Multimodal embeddings and multimodal RAG

taught

Embedding images and text into the same shared vector space so a text query can retrieve relevant images (or vice versa), and multimodal RAG as this course's RAG pipeline (ch.09-11) extended to retrieve and inject non-text content alongside or instead of text chunks.

Taught in

Multimodal embeddings and multimodal RAG
Wiring Language Models

Requires

dense-retrieval-concept required
Multimodal embeddings extend the same nearest-neighbor retrieval idea to a shared image/text vector space.
contrastive-image-text-embedding recommended
computer-vision teaches how contrastive image-text embeddings are actually trained; this course only needs to use the resulting shared vector space for retrieval.

Required by

Nothing in the catalog needs this yet.

Local knowledge map

Click to expand

Solid arrows are required prerequisites; dashed are recommended or helpful. Lower rows are more foundational. Click any box to jump to that concept.