From Training Data to Generated Output
Level Up Team
This post summarizes From Training Data to Generated Output, module 3.1 of Level Up’s Azure AI Fundamentals series. The lesson is on YouTube as Azure AI Fundamentals AI-901 Exam: 3.1: From Training Data to Generated Output. Module 2.5 is Information Extraction Across Modalities.
A generative system has two phases
The path is training data, then model training, then a foundation model, then a prompt, then a generated response.
Training is when the model learns patterns from large datasets. Inference is when the trained model generates an output for a new input. Training creates or updates the model. Inference uses the model.
If a deployed model answers a user, classifies content, or generates text, the model is performing inference.
A foundation model learns broad patterns
A foundation model is trained on large, varied datasets. Developers adapt it through prompting, grounding, tools, or additional training. Typical uses include answering questions, summarizing content, generating text or images, extracting information, following instructions, and generating code.
Training, inference, and prediction are different steps
Generative models repeatedly predict what content should come next. The model does not retrieve a complete prewritten answer from the training data. On the exam, prediction can mean estimating the next token. It does not have to mean forecasting a future event.
| Term | What happens | Example |
|---|---|---|
| Training | The system adjusts model parameters by learning patterns from data | Learning relationships among words and concepts |
| Inference | The trained model processes a new input | Responding to a customer question |
| Prediction | The model estimates the most likely output | Selecting the next token |
Tokens are the model’s working units
Models process tokens, not raw sentences. “Generative AI is useful.” might split into pieces such as “Gener,” “ative,” “AI,” “is,” “useful,” and the period. A token may be a whole word, part of a word, punctuation, a symbol, or a fragment of code. Token boundaries depend on the model’s tokenizer.
The context window is the working context for one request
The context window limits how much information a model can consider during one inference request. It may include system and developer instructions, the user’s prompt, conversation history, grounding data or retrieved documents, tool results, and the generated response.
More context does not guarantee a better answer. Relevant context matters more than volume.
A prompt becomes a sequence of output tokens
A prompt supplies the input and the instructions. “Summarize this support ticket in two sentences and identify the affected product” can specify the task, relevant context, constraints, the requested format, and examples of acceptable results. The model produces the response by generating a sequence of output tokens: a short summary plus the extracted product name.
Generated answers are probabilistic
For each output position, the model assigns probabilities to possible next tokens. After “Your replacement order will arrive…,” one model might assign tomorrow 48%, Monday 27%, shortly 15%, and after 10%.
The same prompt can produce different answers. Fluent wording does not prove factual accuracy. The model can generate unsupported information. Model settings can change how much the wording varies. A polished response does not mean the model verified every statement against a trusted source.
Bring this to your team
Level Up can run a personalized Azure AI Fundamentals workshop for your team. Tell us a little about your group and we’ll reply with pricing.