Responsible AI in a Generative Application
Level Up Team
This post summarizes Responsible AI in a Generative Application, module 4.2 of Level Up’s Azure AI Fundamentals series. The lesson is on YouTube as Azure AI Fundamentals AI-901 Exam: 4.2: Responsible AI in a Generative Application. Module 4.1 is The Six Responsible AI Principles.
The principles become requirements around the whole application
The reference application is a customer-support assistant that accepts text, speech, images, and documents. Controls surround inputs and customer data, models, prompts, and retrieved information, generated responses and agent actions, and monitoring, escalation, and human review.
Protect what the assistant collects, stores, and reveals
A customer uploads an invoice with an account number and payment details. Those details must not appear in another user’s session or in unnecessary diagnostic logs. Collect only what the support task needs. Authenticate users before retrieving account information. Restrict access by user and application role. Remove sensitive information from logs. Protect credentials used by models and tools. Define retention and deletion. Authentication confirms identity. Authorization determines which information that identity may access.
A strong average can hide a failing group
Speech recognition can work well overall and still misinterpret one regional accent. Test languages and dialects, accents and speech patterns, assistive-technology users, document and image formats, and the customer groups the service actually serves. Measure task completion, error and refusal rates, escalation rate, and response quality by cohort, not only average accuracy.
Say what the assistant is, and what it cannot guarantee
A useful disclosure sounds like: “This AI assistant can answer support questions and summarize your account information. Verify important details before acting, or request a support representative.” Tell users they are interacting with AI, which tasks it can perform, when answers may be incomplete, when it uses customer data or external sources, and how to reach a person. Do not present generated answers as verified human decisions.
Offer more than one way to interact
Alternative access methods most directly support inclusiveness.
| User need | Interaction option |
|---|---|
| Cannot type easily | Speech input |
| Cannot hear audio | Text transcript and captions |
| Cannot see an image | Text description |
| Uses assistive technology | Keyboard and screen-reader support |
| Needs language assistance | Supported translation or human escalation |
Fail in a way the customer can trust
Graceful degradation means preserving safe, limited service when full service is unavailable.
| Failure condition | Safe behavior |
|---|---|
| Model endpoint unavailable | Display a clear status and offer human support |
| Retrieved evidence is missing | Avoid unsupported claims |
| Confidence is too low | Ask for clarification or escalate |
| Tool action fails | Report the failure without claiming success |
| Unsafe request detected | Refuse safely and explain available options |
Monitoring has to lead to a correction
The loop is detect the failure, investigate the cause, correct the system, and validate the correction. Track incorrect or unsupported responses, failed retrievals and tool calls, refusals and escalations, latency and availability, performance differences between groups, and repeated user corrections. Predeployment testing cannot represent every real-world interaction.
Name the owner before the assistant goes live
Escalate when the customer asks for a person, the assistant lacks reliable evidence, the request is sensitive or exceptional, the system detects potential harm or a policy violation, or an action exceeds the assistant’s authority. Reviewers need authority, information, and time to intervene.
| Responsibility | Example owner |
|---|---|
| Model and prompt behavior | AI application team |
| Customer-data controls | Security and privacy teams |
| Support policies | Customer-support leadership |
| Production incidents | Operations team |
| High-impact customer decisions | Authorized human reviewer |
One control can support more than one principle. Choose the principle the control addresses most directly. Preventing customer documents from entering diagnostic logs is privacy and security. Comparing speech-recognition success across accents is fairness. Identifying itself as an AI assistant is transparency. Captions and keyboard navigation are inclusiveness. Escalating when evidence is unavailable is reliability and safety. Assigning refund approval to an authorized employee is accountability.
Bring this to your team
Level Up can run a personalized Azure AI Fundamentals workshop for your team. Tell us a little about your group and we’ll reply with pricing.