Skip to main content
November 19, 2026

AWS Emulator: What to Verify Before Agents Test on It

AWS Emulator: What to Verify Before Agents Test on It

On 11 November 2026, AWS released AWS Emulator. It came with a guest post on Werner Vogels' blog, AWS Emulator and the engineer's job when agents write the code. The post's first claim is simple: the emulator "gives coding agents a local copy of AWS to test against." Its second claim is the bigger one. When agents write the code, the engineer's job moves to building "the specs, tests, and merge gates that decide when that work is correct."

This is a shipped product, not a concept. The code is public under CC0, and it runs on your own machine without an AWS account. This post covers what it actually does, where it differs from LocalStack, what it does not emulate, and what to check before your agents and CI start to trust it.

What AWS Released

According to the post, the emulator is "built from more than 400 API models covering more than 200 AWS services." It "runs free and offline on your own machine." The companion launch post by Matthew Goldberg adds how it was built. Agents write fidelity tests from the public AWS API models and check each test against real AWS. Other agents replay those tests against the emulator and file the failures, and a third team of agents fixes them one issue at a time. A merge gate rejects changes that do not compile, are insecure, or regress fidelity. Goldberg writes: "Our team hasn't written or reviewed any of the code." The fidelity tests are published in aws-emulator-evaluation-suite.

That build process explains a few unusual traits:

  • The README says: "We don't accept pull requests: AI agents keep the emulator up to date with AWS, with regular updates."
  • Releases arrive constantly. The releases page showed v1.0.520 on 19 November, eight days after launch.
  • Known gaps are tracked as files in issues/things-that-do-not-work/. When a gap is fixed, its file moves to things-that-now-work/.

How It Runs

The user guide describes a local process that serves HTTPS on 127.0.0.1:9297 with its own certificate. You can run it as a Linux package, from a Docker image that you build yourself (there is no prebuilt image), or embedded inside a Rust test process for per-test isolated state. The README says building from source needs at least 100 GB of free disk for target/.

The SDK setup page shows the environment a client needs:

unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_PROFILE
export AWS_ENDPOINT_URL=https://127.0.0.1:9297
export AWS_SHARED_CREDENTIALS_FILE=~/.aws/emulator/9297/emulator_credentials
export AWS_CONFIG_FILE=~/.aws/emulator/9297/config
export AWS_CA_BUNDLE=~/.aws/emulator/9297/ca.pem
export AWS_EC2_METADATA_DISABLED=true
export AWS_DISABLE_HOST_PREFIX_INJECTION=true
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

curl --cacert ~/.aws/emulator/9297/ca.pem https://127.0.0.1:9297/__admin/health
aws sts get-caller-identity   # Account 000000000000

Two details matter for CI. First, the emulator generates new credentials and a new CA certificate on every start, so a cached ca.pem from the last job will fail. Node, Java, Rust, and .NET clients do not use AWS_CA_BUNDLE: Node needs NODE_EXTRA_CA_CERTS, Java needs JAVA_TOOL_OPTIONS pointing at the emulator's truststore.p12, and Rust and .NET need SSL_CERT_FILE. Second, requests must be signed with SigV4 by default. The docs warn that if real credentials stay in the environment, the SDK signs with them and the emulator rejects the request. That is a good failure: a misconfigured job cannot quietly reach your real account through this path.

Telemetry is on by default. The README says the records never include request or response contents, code, or credentials. Turn it off with --telemetry off or EMULATOR_TELEMETRY=off if your CI policy needs that.

What It Does Not Emulate

The security page is the most important page for SREs. It says plainly: "Don't use AWS Emulator to test whether your application is secure." The details:

  • Some services evaluate IAM policies for users and roles created in the emulator, "but not the same way as AWS." The emulator "never denies a request signed with its own key."
  • It does not check KMS key policies or S3 bucket policies.
  • It uses one account, 000000000000, and does not support calls across accounts or Regions. To test several Regions, you run one emulator per Region.
  • State lives in memory and is gone when the emulator stops.
  • S3 Control requests cannot reach it.

The gaps folder adds behaviour gaps. Two examples we read: an open issue says Lambda Invoke of a deployed python3.12 function returns NotImplemented because "there is no in-process runtime and no payload is returned." Another says EMR Serverless job runs "are recorded but never executed." In both cases the control plane works and the data plane does not. Check that folder for each service you depend on, because API-level coverage does not tell you whether the work actually runs.

AWS Emulator vs LocalStack

LocalStack is the obvious comparison, and its situation changed this year. Since 23 March 2026, LocalStack for AWS ships as one image that needs an auth token. The old Community repository is archived. A free Hobby plan remains, but the pricing post says the free tier does not permit commercial use. LocalStack's own response to AWS Emulator puts it this way: "AWS Emulator emphasizes breadth," while LocalStack "emphasizes depth on the services most teams build on, including areas like IAM."

Question AWS Emulator LocalStack for AWS
Account or token needed No Account and auth token required
Free for commercial CI Yes, CC0 No, Hobby is non-commercial
Endpoint HTTPS 127.0.0.1:9297, new CA per start localhost.localstack.cloud:4566
IAM enforcement Partial, never denies its own key ENFORCE_IAM=1 on Base, Ultimate, Enterprise, and Student, not Hobby
State across restarts In memory only Persistence and Cloud Pods, not on Hobby
Accounts and Regions One account, one Region per instance See LocalStack docs per service
Known gaps Public issue files per behaviour Per-service coverage pages

Neither tool is a drop-in replacement for the other. If your tests set a custom endpoint like localhost:4566 or use the awslocal wrapper, plan a migration step for the TLS endpoint and the CA bundle.

The Failure Mode to Plan For

Here is the trap for agent-written infrastructure. An agent writes a Lambda handler, an IAM role, and a bucket policy. It runs the tests against the emulator, and they pass. The bucket policy was never evaluated, and the IAM role could not deny anything, because the emulator never denies its own key. The agent reports "tests green" in the PR, and a reviewer who skims sees a passing check.

The post's own lesson applies here. The author describes supervising an agent for four nights before realising "it had no way to check its results." An emulator gives the agent a way to check, but only for the behaviour it emulates. Everything outside that boundary needs another gate.

Adoption Checklist

  1. List your services and APIs. For each one, search things-that-do-not-work/ and the per-release fidelity scores the README points to. Record the gaps in the repo, next to the tests.
  2. Pin a release. Updates ship often and are written by agents. Bump the version in a reviewed change, not by pulling the latest build in every job.
  3. Split the test suite by trust level. Tag tests as emulator-safe, IAM or policy, and cross-account or cross-Region. Run only the first group against the emulator.
  4. Keep a real sandbox account. Run IAM, KMS, bucket policy, and multi-Region tests there, on a schedule or before release. The emulator's own build process checks its tests against real AWS. Your suite should do the same.
  5. Make CI fail closed. Unset real credentials before tests, regenerate the CA path on every start, and bind Docker to 127.0.0.1 as the docs advise, since -p 9297:9297 opens the emulator to your whole network.
  6. Tell your agents where the boundary is. Put the gap list and the trust tags in the agent's instructions, so "tests pass" in a PR means the right tests passed.
  7. Decide on telemetry before the emulator reaches shared runners.

What This Means for SRE Teams

The vision in the post is that agents build and check most software, "with engineers deciding what gets built and when the work is correct." For operations, that means the test environment becomes part of the control plane. A free, local AWS removes the account setup and cloud spend that kept many teams from testing infrastructure code at all. It also creates a new place for false confidence. We made a related point about verifying AI-generated code: a check is only as good as what it actually checks. For lighter local emulators, our Floci guide covers another option.

The same rule holds when agents work on live systems during an incident. In Akmatori, SSH connectors are read-only by default, and you allow write commands one pattern at a time. The rule for agents in production and for agents in test is the same: know exactly what the agent can verify and what it can change.

Conclusion

AWS Emulator is a real, free, and very broad local AWS. It is also honest about its limits: it does not test security, it runs one account and one Region, it keeps state in memory, and some data-plane paths still answer NotImplemented. Use it for fast feedback on API behaviour, keep a real account for permissions and cross-Region paths, and write the boundary down where your agents can read it.

If your team wants AI-assisted incident workflows on your own infrastructure, Akmatori helps SRE teams investigate alerts, coordinate response, and automate safe infrastructure actions. Powered by Gcore for global infrastructure reliability.

Automate incident response and prevent on-call burnout with AI-driven agents!