Model Families and Capabilities
Level Up Team
This post summarizes Model Families and Capabilities, module 3.2 of Level Up’s Azure AI Fundamentals series. The lesson is on YouTube as Azure AI Fundamentals AI-901 Exam: 3.2: Model Families and Capabilities. Module 3.1 is From Training Data to Generated Output.
The task picks the family
Generative applications may use several model families: language models, multimodal models, embedding models, image-generation models, and reasoning models. The required input, output, and task determine the appropriate model.
Language models and multimodal models take different inputs
A language model can summarize a typed support ticket. A multimodal model can inspect a customer’s photo and explain the visible equipment problem. When a scenario requires understanding both an image and the accompanying instructions, choose a model that supports those modalities.
| Model family | Typical input | Typical output | Common uses |
|---|---|---|---|
| Language model | Text or code | Text or code | Question answering, summarization, classification, extraction, code generation |
| Multimodal model | Text plus images, audio, or video | Text or other supported content | Image understanding, document analysis, spoken interaction, mixed-media assistants |
Embedding models represent meaning as numbers
An embedding model converts content into a numerical vector that represents semantic meaning. A search for “reset my login credentials” can match documentation titled “Changing a forgotten password.” Applications compare vectors to find content with similar meaning. Keyword search looks for matching terms. Vector search finds conceptually related content.
Common uses are semantic search, document retrieval, recommendation, clustering, and retrieval-augmented generation. Choose an embedding model when the requirement centers on semantic similarity, vector search, or retrieval.
Image-generation models produce new visuals
An image-generation model creates or modifies images from instructions and, when supported, reference images. “Create a product-support illustration showing a technician inspecting a network rack” is text-to-image generation. Other tasks include editing, style or composition changes, and visual variations.
Image understanding interprets an existing image. Image generation produces new visual content. Some multimodal models understand images without generating them.
Reasoning models spend more computation before answering
A reasoning model allocates more computation to working through a complex problem before it produces an answer. A strong fit is a request such as: compare three support-resolution plans, apply five policy constraints, and justify the safest valid choice.
Strong use cases include multi-step analysis, mathematics and logic, complex coding, and planning under multiple constraints. The tradeoffs are higher latency, potentially higher cost, and unnecessary work for many simple requests. Choose a reasoning model when task complexity justifies the extra analysis, not merely because the prompt contains a question.
Match the capability to the requirement
Ask what content the model will receive, what output it must produce, how complex the task is, which accuracy, latency, and cost constraints apply, and whether the application needs current or private data. A single application can route different tasks to different models.
| Application requirement | Capability |
|---|---|
| Summarize a customer email | Language model |
| Interpret a photo and a written question | Multimodal model |
| Retrieve semantically similar support articles | Embedding model |
| Create a marketing illustration | Image-generation model |
| Solve a complex planning problem with constraints | Reasoning model |
| Retrieve documentation and compose an answer | Embedding model plus a language or reasoning model |
Bring this to your team
Level Up can run a personalized Azure AI Fundamentals workshop for your team. Tell us a little about your group and we’ll reply with pricing.