AI-901 lesson 5: vision and extraction

Computer vision models read an image and say something about it. Image-generation models do the opposite and produce a new picture from a text description. Information-extraction techniques turn messy content, whether an invoice scan, a photo, a call recording or a video, into named fields an application can store. The skill the exam tests is recognising the direction of flow: image in and words out, words in and image out, or unstructured in and fields out.

What computer vision models do

Computer vision means software that interprets pixels. Scenario questions describe a task and expect its name:

Two routes reach these abilities in Foundry. Dedicated vision services, grouped under Azure Vision in Foundry Tools, handle specific jobs such as reading text from images and give predictable, narrow output. Multimodal generative models accept an image inside the prompt, so a person can ask an open question (“does this weld look cracked, and why?”) and get a flexible answer. Face-related features also raise the privacy and fairness concerns described in responsible AI principles.

What image-generation models do

An image-generation model takes a text prompt, sometimes with a reference image, and creates a new image. Some models also edit, for example by changing only a masked part of a picture. Specific descriptions of subject, style and composition give better results than vague ones. The output is new content, not a measurement of anything real, so it is the wrong choice whenever a scenario needs facts about an existing image.

Extracting information from content

Information extraction pulls specific facts out of content, and the input type decides the first step:

• Text: entity detection and keyword extraction find names, dates and topics (lesson 4).
• Images and scanned documents: OCR reads the characters, and layout analysis recognises tables, headings and fields.
• Audio: speech recognition converts the speech to text, after which text techniques apply.
• Video: speech is transcribed and the visual content is analysed, for example split into scenes.

Azure Content Understanding in Foundry Tools combines these steps behind one idea, the analyzer. An analyzer is a saved configuration that says what kind of content to process and which fields to produce. Prebuilt analyzers cover common cases, and a custom analyzer can be defined for a company’s own forms. The result is structured output, either fields matching a schema or a readable Markdown version of the content. Fields can be extracted as they appear in a document, classified into a fixed set of categories, or generated, such as a one-paragraph summary of a recording. Confidence scores, where enabled, tell the application how sure the service is about each value.

Worked example: a logistics company’s damage claims

A 200-person logistics company receives damage claims. Each claim arrives as a PDF form, one to five photos of the parcel, and sometimes a voicemail. The team wants the claim number and amount in a database, a judgement of whether the parcel looks damaged, and a short note of the voicemail.

The claim number and amount are fields in a form, so extraction applies, and Content Understanding with a document analyzer returns them with confidence scores. The damage judgement is an open question about a photo, so a multimodal model that accepts images is the natural fit. The voicemail needs a transcript and a summary, which is speech recognition followed by summarisation, or an audio analyzer that produces a generated summary field. No new picture is needed, so image generation never appears.

Easy to confuse

• Analysing vs generating an image. “Describe, check, count, read” means analysis. “Create, draw, design” means generation.
• OCR vs extraction. OCR returns all the text it sees. Extraction returns the specific fields requested, such as invoice total, with their meaning attached.
• Summarisation vs field extraction. A summary is shorter prose. A field is a named value an application can sort and total. If the requirement says “store in a database column”, summarisation is the wrong answer.

Key points

• Computer vision interprets existing images: classification, object detection, OCR, captions and face analysis.
• Image-generation models create new images from text prompts and do not measure or inspect real images.
• Content Understanding uses analyzers, prebuilt or custom, to turn documents, images, audio and video into structured fields or Markdown.
• When an application needs consistent named values, extraction beats free-form generation.

Check yourself

This lesson belongs to the concepts domain of AI-901, which carries 40-45% of the exam, and it closes the workload vocabulary before the course moves to hands-on work in Foundry. The domain is summarised in the AI concepts recap.