MLA-C02 deployment and orchestration

Deployment and Orchestration of ML and AI Workflows is worth 24% of MLA-C02. AWS frames it as choosing deployment infrastructure and endpoints, provisioning compute, configuring auto scaling, setting up CI/CD to automate workflows, and building agentic workflows.

This is the domain where the exam’s cost instinct is strongest. Nearly every question offers a technically workable answer that costs more than necessary.

Choosing an inference approach

The most tested decision in the domain. Match the pattern to the requirement:

Approach	Use when	Cost shape
Real-time endpoint	Low latency, continuous traffic	Pay while provisioned
Serverless inference	Intermittent, unpredictable traffic	Pay per request, cold starts
Asynchronous inference	Large payloads, long processing, latency tolerant	Queued, scales to zero
Batch inference	Large volumes on a schedule, latency irrelevant	Cheapest per record

The decision rule the exam expects: if latency does not matter, do not pay for a persistent endpoint. A nightly scoring job is batch. A sporadic internal tool is serverless. A customer-facing API is real time.

Provisioning and auto scaling

Right-sizing instances, and when a GPU is actually justified.
Auto scaling on a meaningful metric, so capacity follows demand.
Scaling to zero where the pattern allows it.
Multi-model or multi-container hosting when many small models would each waste an endpoint.

Building and maintaining containers

Task 3.2 of the outline asks you to build and maintain containers for ML and AI workloads (3.2.3). Every SageMaker AI training job and endpoint runs a container image, so this is less about Docker trivia and more about choosing and looking after the right image.

The decision ladder, cheapest effort first:

Use a prebuilt SageMaker AI image for a supported framework (PyTorch, TensorFlow, scikit-learn, XGBoost and others) and supply only your script. This is script mode, and it covers most scenarios.
Extend a prebuilt image when you need an extra library or system package: start your Dockerfile FROM the AWS image, add the dependency, push the result to Amazon ECR.
Build your own image only when no prebuilt one fits, for example an unsupported framework or a model trained outside AWS that you are bringing in (3.1.5).

What a custom image must respect, per the SageMaker AI documentation:

Images are pulled from Amazon ECR, so the execution role needs permission to read the repository.
A training container is started with the argument train, reads input from /opt/ml/input and writes the model to /opt/ml/model, which SageMaker AI uploads to S3 when the job ends.
An inference container is started with serve and must answer GET /ping (health check) and POST /invocations (predictions) on port 8080.

“Maintain” is the part people forget. An image is frozen at build time, so the framework, the base OS and every library age inside it. Rebuild on a schedule or when a base image is patched, tag each build with an immutable version rather than reusing latest, and scan the images in ECR with Amazon Inspector, which links this task to 4.3.1. Build and push belong in the pipeline (CodeBuild is the usual step), not on a laptop.

Exam pitfall: a scenario says the team needs one extra Python package and the options include writing a full custom container. That works, but extending the prebuilt image, or adding a requirements.txt next to a script-mode entry point, meets the need with far less to maintain. AWS’s comparison page lists bring-your-own-container as removed from C02 as a separate skill, which is a hint about the level: expect choose-and-maintain questions, not build-from-scratch ones.

For images, layers and the Dockerfile itself, see the Docker basics guide.

CI/CD for ML

ML pipelines are software pipelines with extra stages:

Source control for code and for data and model versions.
Automated build, test and deploy.
Infrastructure as code so environments are reproducible.
Deployment strategies that reduce risk: staged rollout, canary, shadow traffic, and the ability to roll back.
Automated retraining triggered by a schedule or by drift detection.

The distinctive ML addition: your pipeline must handle a model artefact as a deployable thing with its own versioning and approval, not just code.

Agentic workflows

Explicit in C02’s task list and new relative to its predecessor. At this level:

An agent uses tools and takes multi-step actions rather than returning a single response.
Orchestration means coordinating those steps, handling failures and managing state.
Because agents act, consequential actions need approval controls — a human in the loop for anything irreversible.
Observability matters more, not less: you need to see which steps ran and why.

Sample questions

Question 1. An internal tool calls a model perhaps 30 times a day, at unpredictable intervals. The team wants to minimise cost and does not object to occasional slow first responses. Which deployment fits?

A. A real-time endpoint with provisioned capacity
B. Serverless inference
C. A nightly batch inference job
D. An always-on GPU endpoint

Show answer

Answer: B

Serverless inference bills per request and scales to zero between calls, which suits low and unpredictable volume where cold starts are acceptable. A provisioned real-time endpoint bills continuously for idle capacity, batch does not suit interactive use, and an always-on GPU endpoint is the most expensive option.

Want more questions like this? Full MLA-C02 practice tests →

Question 2. A team wants to release a new model version while limiting exposure if it behaves badly, with the ability to revert quickly. Which approach is most appropriate?

A. Replace the existing endpoint with the new version directly
B. Deploy to a larger instance type to absorb any problems
C. Test thoroughly in development and then deploy to production
D. A canary rollout sending a small share of traffic to the new version, with rollback

Show answer

Answer: D

A canary or staged rollout sends a small share of traffic to the new version first and allows a fast revert, which is exactly what limiting exposure with quick rollback requires. Replacing the endpoint outright removes the safety net, a larger instance is unrelated, and testing only in development does not limit production exposure.

Want more questions like this? Full MLA-C02 practice tests →

Question 3. An agentic workflow can issue refunds as one of its tool actions. What control should the deployment include?

A. A human approval step before the refund action executes
B. Logging every refund to CloudWatch for later review
C. Automatic retry if the refund action fails
D. A more capable foundation model behind the agent

Show answer

Answer: A

An irreversible financial action requires a human approval step before execution, which is the standard control for consequential agent actions. Logging records what happened after the fact, retries repeat the action, and a larger model does not make the action safe.

Want more questions like this? Full MLA-C02 practice tests →

Question 4. A team trains with the prebuilt SageMaker AI PyTorch image in script mode. A new feature needs one additional system library that the image does not include. The team wants the least ongoing maintenance. What should it do?

A. Write a new container image from scratch that implements the training contract
B. Run the training on a self-managed Amazon EC2 instance instead
C. Build an image FROM the prebuilt PyTorch image, add the library and push it to Amazon ECR
D. Install the library with a shell command at the start of every training job

Show answer

Answer: C

Extending the prebuilt image keeps AWS’s tested framework stack and adds only the missing library, and pushing it to Amazon ECR lets SageMaker AI pull it. A container written from scratch must then implement and patch the whole stack. Installing at runtime on every job is slow and unrepeatable, and moving off SageMaker AI to EC2 adds infrastructure to manage.

Want more questions like this? Full MLA-C02 practice tests →

What to practise

Deploy the same model three ways — real-time, serverless and batch — and look at what each costs when idle. That comparison makes the inference-selection questions immediate, and they are the most common question shape in this domain.