DEV Community: AWS

AI Agent Memory: What to Store and What to Throw Away

Elizabeth Fuentes L — Sat, 05 Sep 2026 02:37:13 +0000

The best AI agent memory is selective: it keeps durable facts, preferences, and events and drops the small talk. Here is how to build that in Strands, three ways.

📦 Clone and ⭐ stop-ai-agents-losing-memory-sample-for-aws

Everyone is racing to make agents remember more. Bigger context windows, longer histories, a vector store that keeps everything. But the agent that wins is not the one that remembers the most, it is the one that keeps the right things and throws the rest away. Store everything and your agent's memory becomes expensive, slow, and dirty; store nothing and it forgets its user between sessions. The earlier posts in this series covered where memory lives (key-value, vector, graph). This one is about the decision that comes before all of them:

What is worth storing, and what should you throw away?

That decision is called memory extraction (or selective memory), and this post builds it three ways against the same planted conversation with deterministic ground truth (companion demo). The first two run on Strands Agents' native memory framework: you attach a MemoryManager, and the SDK runs extraction, storage, retrieval, and injection for you. The third is fully managed by AWS:

Native MemoryManager, one store: the framework does memory; you write one selection prompt that decides what to keep.
Native MemoryManager, four typed stores: the same framework, one store and one prompt per memory type (Amazon Bedrock AgentCore Memory's partitioning, native SDK).
Managed memory: you send raw turns and a managed service extracts asynchronously (Amazon Bedrock AgentCore Memory).

The code uses Strands Agents, Amazon S3 Vectors or Amazon DynamoDB Vector Search for the vector store, and Amazon Bedrock AgentCore Memory for the managed option.

What counts as memory extraction?

Memory extraction, also called selective memory, is the step between a conversation and a memory store. It decides what to keep, which memory type it belongs to, and what to throw away. It is separate from the storage backend: extraction decides what enters memory, the backend decides where it lives.

The four memory types are the same across this whole series, and they map one-to-one to the four built-in strategies Amazon Bedrock AgentCore Memory offers (built-in strategies):

Type	What it holds	AgentCore built-in strategy
facts	durable facts about the user's world	`semanticMemoryStrategy`
preferences	likes/dislikes the user reveals	`userPreferenceMemoryStrategy`
trip_summary	rolling summary of the current task	`summaryMemoryStrategy`
episodes	notable events, one entry each	`episodicMemoryStrategy`

Strands does memory for you, natively

You don't hand-roll memory tools, and you don't put memory logic in the chat agent's system prompt. Strands ships a native MemoryManager you attach to the agent. It handles three jobs across the stores you give it: recall (a search_memory tool the agent can call), injection (folding relevant memory into the prompt before each call, without touching durable history), and extraction (a ModelExtractor that distills conversation into memories, off the turn, on a trigger). Recall and injection are on by default; extraction is opt-in.

You own exactly two things: the extractor's selection prompt (the keep/discard policy) and the store (where memories live and how they're searched). Everything else is the framework's job.

from strands import Agent
from strands.memory import MemoryManager, ModelExtractor, ExtractionConfig, IntervalTrigger
from strands.models.openai import OpenAIModel

# The selection prompt IS the keep/discard policy: the only memory logic you write.
SELECTION_PROMPT = (
    "Extract durable memories worth keeping about a traveler: identity, dietary "
    "restrictions and allergies, stated travel preferences, and confirmed bookings. "
    "Discard small talk, weather, and passing opinions. "
    'Return ONLY a JSON array of {"content": string}, or [] if there is nothing to keep.'
)

# A store implementing the native MemoryStore contract, backed by a vector index.
store = VectorMemoryStore(
    name="traveler_memory",
    extraction=ExtractionConfig(
        trigger=[IntervalTrigger(turns=1)],                 # when extraction runs (off the turn)
        extractor=ModelExtractor(                           # HOW selection happens
            model=OpenAIModel(model_id="gpt-4o-mini"),      # a separate, optionally cheaper model
            system_prompt=SELECTION_PROMPT,                 # <-- the policy you own
        ),
    ),
)

agent = Agent(
    model=model,
    system_prompt="You are a flight assistant. Be concise.",  # persona only, no memory logic
    memory_manager=MemoryManager(stores=[store]),
)
agent("Hi, I'm Sam, vegetarian with a shellfish allergy.")     # extraction happens automatically

The chat agent's system prompt stays about the agent's job. The selection policy lives in the ModelExtractor, a separate model call the framework runs off the turn, so it never bloats the conversational prompt and can even run on a cheaper model than the chat.

What each native piece does

You only touch four things, and the native memory framework handles the rest:

MemoryManager: the plugin you attach to the agent. It gives the agent a search_memory tool, runs extraction in the background, and folds relevant memories into the prompt, all at once.
ModelExtractor: the piece that decides what to keep. Its system_prompt is your keep/discard policy, and it runs as a separate model call from the chat.
ExtractionConfig: ties the extractor and a trigger to a store, and quietly strips tool-call noise so tool JSON never lands in memory.
IntervalTrigger / InvocationTrigger: decide when extraction runs (every turn, or every N turns), off the conversation path.

Injection is on by default too: before each model call the manager pulls relevant memories into the input without touching the durable history, and if retrieval fails it just skips injection instead of breaking the turn. The point: you declare what to keep (the prompt) and where (the store); the framework handles the plumbing.

The store: your data, your backend

MemoryManager needs somewhere to persist and search. That's a MemoryStore, a small contract of add(content) and search(query). The demo implements it over a vector index so recall is semantic, with Amazon Titan Text Embeddings V2:

class VectorMemoryStore(MemoryStore):
    name: str
    writable = True                        # the manager may write extracted memories here

    async def add(self, content, metadata=None):
        vector = embed(content)            # Titan V2
        self._backend.put(key, content, vector)   # S3 Vectors OR DynamoDB

    async def search(self, query, options=None):
        hits = self._backend.query(embed(query))
        return [MemoryEntry(content=t, metadata={"score": s}) for t, s in hits]

Because the store is just this contract, the vector backend is a lever you set with one env var: VECTOR_BACKEND=s3 (Amazon S3 Vectors) or dynamodb (Amazon DynamoDB Vector Search). It's separate from what gets remembered.

Which one, and why. Both use the same Titan V2 embeddings, so recall quality is identical; the choice is about where the vectors live and how often you query them (this is exactly how the AWS docs frame it):

Amazon S3 Vectors (docs): a dedicated vector bucket, separate from your operational data. AWS positions it for cost-optimized storage at massive scale with infrequent access: query latency is sub-second, around 100 ms or less for frequent queries and higher (up to a second or more) for cold ones. Pick it when memory is a standalone concern, you have a very large or archival vector corpus, and sub-second (not sub-10 ms) latency is fine.
Amazon DynamoDB Vector Search (docs): the vector index lives inside a DynamoDB table, so embeddings sit next to your operational data with no separate vector store to sync. AWS states single-digit-millisecond latency at 99%+ recall for real-time search. You create it with the same CreateTable/UpdateTable APIs (a VectorIndexes parameter) and query it with the SearchVectors API, which needs a recent boto3. Pick it when your agent already reads from DynamoDB, or you want real-time retrieval and one service for data and memory.

In short, the AWS docs draw the line at access pattern: use S3 Vectors when memory is a standalone, large, or archival concern and sub-second latency is fine; use DynamoDB Vector Search when you are already on DynamoDB or want real-time retrieval with data and memory collocated. Neither is "faster memory" in a way the user feels, since the embedding call dominates end-to-end latency for both.

You don't always have to write the store, either. The Strands integrations directory lists ready-made MemoryStore backends: Amazon Bedrock Knowledge Base and AgentCore Memory from AWS, packaged S3 Vectors (s3-vectors-memory) and DynamoDB (strands-dynamodb-storage) stores, and partner options like Mem0, Zep, Vectorize, and Neo4j graph memory. Implementing the contract yourself, as this demo does, is the way to understand it; in production you'd often drop in one of those.

The three mechanisms, side by side

Mechanism	What it is	Selection prompt	Partitions
A: native, one store	`MemoryManager` + one `MemoryStore`	one general prompt	one
B: native, four typed stores	`MemoryManager` + four `MemoryStore`s	one prompt per type	four (facts / prefs / summary / episodes)
C: Amazon Bedrock AgentCore Memory	fully managed by AWS	AWS (managed, or override)	managed

A vs B is granularity, not backend. A is the simplest native setup: one store, one prompt. B reproduces AgentCore's per-type partitioning (four stores, four specialized prompts) with the native SDK, so the part AgentCore ships built-in (the selection criteria) becomes text you can read and tune. Both A and B run on S3 Vectors or DynamoDB (the VECTOR_BACKEND lever); the backend doesn't define the mechanism. C is the fully managed counterpart to B: you send raw turns, AWS extracts.

The measured results

The test conversation mixes 5 keepers (2 facts, 2 preferences, 1 episode) with 3 decoys to throw away (small talk, a passing opinion, ephemeral weather). The score is selection recall: how many of the 5 keepers a mechanism stored, checked deterministically against that ground truth (no LLM judge). The decoys are there so a mechanism cannot win by hoarding: keeping everything would ace recall and still be useless.

From 20 runs each for A and B, and repeated runs for C (gpt-4o-mini; the extractor is an LLM, so A's and B's exact recall varies slightly run to run, C's result was consistent, and your numbers will differ):

Mechanism	Selection recall	Who owns the selection policy	Retrieval granularity	Turn latency	When queryable
A: native, one store	~3.9/5 (3-5)	you (one prompt)	one blended pool	~3.2 s/turn	when the turn returns
B: native, four typed stores	~5/5 (4.95)	you (one prompt per type)	per type (query/inject/tune each alone)	~3.5 s/turn	when the turn returns
C: Amazon Bedrock AgentCore Memory	5/5	AWS (managed, or override)	managed per strategy	~2.2 s/turn	~20-55 s later (async)

The reproducible finding, across all those runs: all three recall the keepers well. What differs is who writes the selection policy, and that is a choice, not a verdict:

B is the sharpest when you want to own every criterion. One specialized, non-overlapping prompt per type means each store keeps only its own kind of memory (facts vs preferences vs a confirmed-trip summary vs a completed action), so it lands recall ~5/5 on every run and drops the decoys. Writing four tight prompts is the work; per-type selection is the payoff. Choose B when the keep/discard rules are yours to define and tune.
A is the same idea with the least setup. One store, one prompt: you own the selection policy at a coarser grain. It rejects small talk, weather, and opinions; a single prompt covering everything recalls a touch less consistently (~3.9/5). Choose A when one flat memory and one prompt are enough.
C lets AWS do the selection for you. You send raw turns and Amazon Bedrock AgentCore Memory's managed strategies extract, embed, and index them, with no extraction pipeline to maintain. It recalls the keepers (5/5) and runs the memory lifecycle server-side. Choose C when you would rather not own the selection logic. If you do want to shape it, AgentCore supports custom strategies with prompt overrides: override a built-in strategy's default logic with your own prompt and model, so control is there on the managed path too.

Nothing here is instant. Every mechanism runs an extraction step, embeds the kept text, and writes it. A and B pay that cost inside the turn, so the memory is queryable the moment the turn returns. C pays it asynchronously on AWS: the turn is cheap (~2.2 s) but the extracted memory appears ~20-55 seconds later (measured, waiting for extraction to settle). Same work, moved off the turn, for a delay before the memory is usable.

The takeaway is not "more stores is better." It is how much of the selection policy you want to hold: A and B put the prompt in your hands (one prompt, or one per type for finer control); C hands the whole pipeline to AWS, with custom strategies as the way back in if you want it. Same goal, different amount of control, pick the one that fits your team.

A note on `flush()`: when is a memory saved?

Because extraction runs in the background, the last turn's memory might not be persisted yet when the agent finishes responding. await manager.flush() closes that gap: it forces every store to save its buffered messages (even one whose trigger hasn't fired, or one currently backed off) and waits for those writes to land. It's the synchronization point that guarantees nothing is lost on a graceful shutdown.

When you call it depends on how you drive the agent:

Synchronous agent("...") (this demo): each call runs in its own event loop, so the framework flushes for you after every invocation. Memory is persisted by the time the call returns: you never flush manually.
Async agent.invoke_async(...) / stream_async(...): these share your long-lived loop and don't flush, so extraction stays on its trigger cadence, and you await memory_manager.flush() yourself at a shutdown boundary.

Two caveats from the docs: don't call flush() every turn alongside a periodic trigger (it defeats the trigger's schedule), and a hard kill (SIGKILL, timeout) can still drop the last unsaved turn since flush never runs, so a more frequent trigger narrows that window.

What B's four typed prompts look like

Mechanism B is where owning the prompt pays off most, so it's worth seeing its policy. Each memory type maps to its own vector partition and its own selection prompt, in one table:

# {memory_type: (vector_partition, selection_prompt)}
# Each prompt keeps ONLY its own kind of memory and explicitly rejects the others,
# so the four stores never overlap: nothing lands in two stores, and no decoy slips
# in disguised as a "summary". That discipline is what makes B's per-type selection sharp.
TYPED = {
    "facts":        ("selective-facts",   "Extract ONLY durable FACTS about the traveler (name, home airport, "
                                          "dietary restrictions, allergies). DISCARD preferences, opinions, "
                                          "small talk, weather, and one-off events. If none, return []."),
    "preferences":  ("selective-prefs",   "Extract ONLY stated travel PREFERENCES (cabin, seat, layover rules, "
                                          "budget). DISCARD facts like allergies, one-off bookings, opinions, "
                                          "small talk, weather. If none, return []."),
    "trip_summary": ("selective-summary", "Maintain a one-sentence summary of the CONFIRMED current trip ONLY "
                                          "(route, airline, date once booked). DISCARD small talk, weather, "
                                          "opinions, and anything not part of the booked trip. If unchanged, return []."),
    "episodes":     ("selective-episodes", "Record ONLY a concrete completed ACTION the traveler took this turn "
                                           "(a booking, a cancellation, a confirmed change). Not a comment, question, "
                                           "opinion, weather remark, or small talk. If none, return []."),
}

# Every prompt ends with the same output contract, appended when the extractor is built:
JSON_CONTRACT = ' Return ONLY a JSON array of {"content": string}, or [] if none.'

Returning [] is a first-class answer; that's the discard half of selection. If the extractor keeps a decoy, you tune the prompt. On the managed path (C) those defaults live in the service, and you can still reach them through custom strategy overrides. (If you don't pass a prompt to the ModelExtractor, it uses Strands' sensible default, but then you inherit its generic criteria.)

Which mechanism should you pick?

Situation	Pick
You want the SDK to run memory for you with the least setup, and one selection prompt is enough	A: native, one store
You need per-type control of the keep/discard criteria (regulated domain, custom taxonomy)	B: native, four typed stores
Production multi-user; asynchronous extraction (seconds of lag) is fine; you want AWS to run the whole memory pipeline for you (with custom strategies available if you later want to shape it)	C: Amazon Bedrock AgentCore Memory

Two levers cut across all of this:

Backend: S3 Vectors vs DynamoDB Vector Search is a one-env-var choice for A and B; pick by where your operational data already lives.
Destination: a MemoryStore could just as well write facts to the knowledge graph of Demo 03 (as triples). Selection and storage compose.

Practical notes for the managed path

A few things worth knowing when you wire up Amazon Bedrock AgentCore Memory through the official Strands session manager (AgentCoreMemorySessionManager):

Give each strategy an explicit namespace at creation (/facts/{actorId}/, /preferences/{actorId}/, /summaries/{actorId}/{sessionId}/, /episodes/{actorId}/{sessionId}/). The same namespace you set on the strategy is the one you reference in RetrievalConfig.
Use RetrievalConfig(relevance_score=...) to control what comes back at recall. It keeps only records above a relevance threshold per namespace, so the agent sees the most on-point memories.
Extraction is asynchronous. In this demo the extracted memory became queryable ~20 to 55 seconds after the turn (measured, polling until extraction settled). Plan for eventual consistency: a fact written this turn may not be retrievable on the next one.
A memory in CREATING status isn't ready yet. Wait until it reports ACTIVE before sending events.
Want to shape what the managed strategies keep? Use custom strategies with prompt overrides: override a built-in strategy's default extraction/consolidation logic with your own prompt and model, so you get the managed pipeline and your own criteria.

The service evolves quickly, so treat the exact behaviors above as current observations and check the docs for the latest.

Try it

Everything runs from Demo 04 of the companion repo: the three mechanisms against the same conversation, with the deterministic scorecard and the lag measurement. AWS resources (vector indexes/tables, the managed memory) are created automatically if missing, and the README covers cleanup and the exact native Strands pieces used.

There is also one interactive chat per mechanism (chat_single_store.py, chat_typed_stores.py, chat_agentcore.py): talk to the agent and watch memory fill turn by turn, with small talk discarded and keepers stored. The AgentCore chat lets you feel the async lag: right after you speak, /memory shows nothing until extraction catches up.

This post was about throwing away noise. Next in the series, the higher-stakes version of the same instinct: what your agent must NOT remember even when it looks legitimate, and how to defend the write path against prompt injection and memory poisoning.

Research referenced

Paper	Theme
MIRIX: Multi-Agent Memory System	Typed memory (6 types, +35% accuracy, SOTA 85.4% on LOCOMO)
MemGPT: Towards LLMs as Operating Systems	Core memory concept, virtual context management

We reproduce the mechanism these papers describe (typed, selective memory), not their specific benchmark numbers.

¡Gracias!

🇻🇪🇨🇱 Dev.to Linkedin GitHub Twitter Instagram YouTube

How a Strands agent took Claude Opus 5 from 30% to 99.95% on ARC-AGI-3

Morgan Willis — Fri, 28 Aug 2026 14:07:24 +0000

AI’s most fervent and optimistic promoters promise a future where AI is innovating its way out of society’s biggest problems. AI systems will conduct scientific research, discover new drugs and materials, run engineering projects, and autonomously execute complex tasks usually requiring teams of experts working over long periods of time.

But fulfilling that promise requires more than a good prompt and basic context engineering. AI systems need to pursue goals over long horizons, encounter situations they’ve never seen before, learn from failed attempts, adapt their strategy, and keep making progress without a human telling them what to do next.

They need to solve novel problems that aren’t represented in their training data. But are today’s models actually capable of that?

That’s one of the questions ARC Prize is trying to answer with ARC-AGI-3, a benchmark that drops AI systems into unfamiliar environments without giving them the rules or even telling them what the goal is. They have to experiment, figure out how the environment works, and learn what winning means in this ambiguous context.

AWS engineers built an agent to take it on. Using Claude Opus 5 and the open source Strands Agents SDK, the agent completed all 183 levels across ARC-AGI-3’s 25 public environments with a 99.95% relative human action efficiency (RHAE) score over one 8 hour run, spending about $830 in tokens to do it. NVIDIA has recently reported a similar result, with its AVO agent, also using Opus 5, scoring 100% RHAE on the public game set.

Then compare that to ARC Prize's standard evaluation of Opus 5, which scored 30.16% on ARC-AGI-3. The model across these runs is the same, but the system surrounding it is very different.

These results give us some data showing how important agent harnesses are for getting what you want out of AI. A model by itself provides the reasoning and judgment, but the harness gives it a surrounding system for managing context, maintaining state, taking actions, observing their consequences, and generally interacting with the world around it. How you build the harness matters, so having open source examples of systems that successfully handle these kinds of long-horizon tasks gives us a chance to look beyond the benchmark score and understand which design choices actually made the difference.

I work closely with the Strands team at AWS, and the entire Strands ARC-AGI-3 harness is open source. So let's dig into it to understand how they did it and how the agent works. You can find the code for yourself on GitHub.

The public vs private game set for ARC-AGI-3

Before we get further into the details I do want to clarify one thing, especially for those of you who aren't deeply familiar with how ARC-AGI-3 works.

The public game set has 25 games that are essentially the practice exam, and humans can play these games too. But the official ARC Prize rankings come from private, held-out evaluation, and the 2026 competition runs offline on Kaggle where hosted API models aren't allowed at all. The public set gives researchers a common set of environments for developing and experimenting with their agent systems, while the competition evaluates them against new unseen environments.

That competition is still open. But that doesn't mean that the public results don't give us anything useful. Acing the public benchmark is still a strong signal about which agent architectures can successfully handle long-horizon tasks.

Inside the Strands harness

First off, it's good to know that the agent starts with zero game knowledge in its system prompt. You can check the entire system prompt out by taking a look at the code directly.

The available controls are presented as generic labels like ACTION1, ACTION2, and ACTION3, without telling the agent what those actions actually do. Even the board itself is represented as raw numeric values. If it wants to understand what anything does, it has to try something and observe what happens.

This prompt is the same across all 25 games, and everything the agent knows about a game, it learned by playing it. So if it doesn't know anything, how does the agent navigate the game board?

Before the agent takes its first turn, the harness writes the initial board state to a game log. Then the model is told to read that log, analyze the board, and decide what actions it wants to try.

To do that, the harness gives the agent a small set of tools for working with files and code. It can read files, search them with grep and regex, write and edit files, and execute Python. So even though the board isn't pasted directly into the model's prompt, the agent can inspect it through the log and use these tools to start figuring out what it's looking at.

Once the agent chooses an action, the harness executes it and records the action and resulting board state back to the log. The agent can then inspect what changed, form a theory about what its action did, and decide what it wants to try next. Each new action and resulting board state adds another piece of evidence to that growing history.

Keeping that history in a file rather than continually adding it directly to the context window is important because things would get unruly pretty fast. A single board is 64x64, and over hundreds of actions the agent can build up a massive interaction log that can grow to tens of megabytes.

The agent can use its tools to decide what parts of that history are actually useful, performing its own context engineering. It can search for previous actions, write Python scripts to compare board states or look for patterns, save notes about what it has figured out, and use those findings to decide what to try next.

Why these tools?

The specific tools the agent was given didn't come from nowhere. The approach was based on PRO-LONG, which tested how well an agent using GPT-5.5 performed on ARC-AGI-3 as it was progressively given more ways to work with its interaction history.

With read-only access to its history, the agent scored 23.1%. Adding grep and regex brought that to 27.2%. But the biggest jump came when the agent was given access to Python, pushing the score to 38.3%. Finally, adding the ability to write and edit files brought it to 41.2%.

That jump from giving the agent the Python tool suggests that giving an agent access to its history isn't enough, but giving it a way to programmatically analyze and derive insights from that history can make a significant difference in performance.

For the Strands run, across the 25 games, the agent wrote 734 scripts for itself. Some of those scripts captured pieces of what the agent had learned about how a game worked. It built parsers for interpreting board states, renderers for representing them, and even simulators of game mechanics.

The team describes some of these scripts as small world models. In 10 of the 25 games, the agent imported code it had written earlier and continued building on top of it.

Those tools also run inside a bubblewrap sandbox with no network access and writes are restricted to the agent's workspace. That means that the agent can analyze everything it has learned through its own interactions, but it can't go online and find a walkthrough or inspect the underlying game implementation.

Relative Human Action Efficiency

The agent passed all the levels and scored 99.95% relative human action efficiency (RHAE). RHAE is a measure of how efficiently the agent solves the game by comparing the number of actions it takes on each level against a baseline from humans playing the game for the first time. This is arguably a more important metric than completion alone because ARC Prize defines AGI as “a system that can match the learning efficiency of humans.”

An agent could eventually solve a game by taking thousands of actions and exhaustively trying every possibility, but that wouldn't demonstrate the same learning efficiency as a human. RHAE helps us measure that difference. This is the breakdown for the Strands agent:

Compared with human baseline	Levels	Share
Beat human baseline by enough to reach maximum score	150	82.0%
Beat human baseline	7	3.8%
Matched human baseline	3	1.6%
Used more actions than human baseline	23	12.6%

Across 160 of the 183 levels, the Strands agent matched or beat the human baseline for action efficiency. And on 150 of those levels, it performed well enough to receive ARC Prize's maximum per-level efficiency score. So even though the overall score is 99.95%, it performed better than the human baseline most of the time. You can see how scoring works in ARC Prize's methodology docs.

What this means

This result on ARC-AGI-3 is impressive, and because it's open source we can all learn from how the pattern works. The takeaways for me are: externalize the history, give the agent tools to query it, let it write its own code to turn what it learns into insights and reusable programs, and let it decide what stays in active context for the next decision.

The code is surprisingly easy to understand, and the patterns could be used directly with whatever agent you're building. Go check it out.

Strands Harness Code: https://github.com/strands-labs/benchmark-harnesses/tree/main/arc-agi-3-agent
The PR, which is a great writeup on its own: https://github.com/strands-labs/benchmark-harnesses/pull/12
Scorecard: https://arcprize.org/scorecards/8a10b024-3560-448f-ac31-becc48affe5b
PRO-LONG: https://github.com/alexisfox7/PRO-LONG
ARC-AGI-3: https://arcprize.org/arc-agi/3

Graph Memory: When Vector Search Fails

Elizabeth Fuentes L — Thu, 27 Aug 2026 22:48:51 +0000

📦 Clone and ⭐ stop-ai-agents-losing-memory-sample-for-aws

AI agents accumulate facts across conversations. Key-value memory stores them as labeled blobs. Vector memory retrieves them by meaning. Neither can answer a question that spans multiple facts connected by relationships. Graph memory closes this gap by storing memories as nodes and typed edges.

This post uses a travel assistant as the demo, but the failure is structural, not travel-specific. It shows up in any agent that accumulates facts about people, places, products, or events over time. Eventually a user asks something that can only be answered by following the edges between facts. And there are no edges to follow.

The same structural gap causes agents to hallucinate answers to counting and aggregation questions. In RAG vs GraphRAG, I measured a hotel assistant that couldn't answer "how many hotels accept pets?" without inventing statistics, because it had no graph to compute over. Here the failure is multi-hop retrieval, but the root cause is the same: no edges to follow.

Here is what that failure looks like in a real run, with the travel assistant after it accumulated four facts about its user:

Facts in memory:
  Maya Torres works at Iberia.
  Iberia flies to Madrid.
  Madrid is in Spain.
  Iberia belongs to Oneworld.

Question: "Who do I know connected to flights to Spain?"

Top-3 vector similarity results:
  - Iberia. An airline.
  - Spain. A country.
  - Madrid. A city.

Recovers the person (Maya Torres)? False

Similarity found every piece. It never found the person, because a vector index has no notion of a relationship between its entries. Graph memory fixes this by storing memories as nodes and typed edges, so the answer is reached by traversal instead of resemblance. This post builds it with Neo4j, measures the same four questions against both retrievers (1/4 vs 4/4), and shows the prompting technique that gets an AI assistant to build it right. Everything runs from the companion repo.

(Post 3 of a series; the intro maps all the memory types. This is the most advanced demo so far: it assumes the earlier posts and a Neo4j instance. The code uses Strands Agents; the pattern carries over to any agent framework.)

Why Strands Agents for this demo?

Strands makes it simple to add graph memory to an agent. Creating an agent is just a few lines of code, and tools are functions with a decorator:

from strands import Agent, tool

@tool
def recall_graph(query: str) -> str:
    """Search graph memory by traversing relationships."""
    return graph_retriever.search(query)

agent = Agent(
    model=model,
    tools=[recall_graph, recall_semantic, remember_fact],
)

That's it. No custom integrations, no framework lock-in. The @tool decorator is all you need to plug Neo4j retrievers into the agent. When book_flight executes, it writes edges directly to the graph, and the knowledge graph grows with usage.

The pattern shown here (external graph + tool-based access) works in any agent framework. Strands just makes it straightforward.

What is a multi-hop question?

A question whose answer lives in no single memory, only in the chain between several. Stored as a graph, the assistant's four facts form one:

(Maya Torres) ──WORKS_AT──▶ (Iberia) ──MEMBER_OF──▶ (Oneworld)
                                 │
                            FLIES_TO
                                 ▼
                             (Madrid) ──IN_COUNTRY──▶ (Spain)

"Who do I know connected to flights to Spain?" requires three hops: person → airline → city → country. Key-value memory can't express it (no key is "the chain"). Vector memory retrieves the three most similar fragments and stops. Only a store that keeps the edges can walk them.

How does graph memory answer it?

In two moves: similarity finds the entry point, traversal finds the answer. Both retrievers in the demo are official neo4j-graphrag classes, sharing the same graph and the same vector index. The only variable is whether edges get walked:

from neo4j_graphrag.retrievers import VectorRetriever, VectorCypherRetriever

# Before: pure similarity, returns the nearest nodes, disconnected
before = VectorRetriever(driver, "memory_embeddings", embedder=embedder)

# After: similarity finds an entry node, then Cypher walks back to the person
RETRIEVAL_QUERY = """
WITH node AS entry, score
MATCH (person:Person) WHERE person <> entry
MATCH path = shortestPath((person)-[*1..5]-(entry))
RETURN person.name AS who, [n IN nodes(path) | n.name] AS chain, max(score) AS score
ORDER BY score DESC
"""
after = VectorCypherRetriever(driver, "memory_embeddings",
                              RETRIEVAL_QUERY, embedder=embedder)

Same question, second retriever, same run:

- who='Maya Torres' chain=['Maya Torres', 'Iberia'] score=0.77
- who='Maya Torres' chain=['Maya Torres', 'Iberia', 'Madrid', 'Spain'] score=0.71

Recovers the person (Maya Torres)? True

Note what the graph adds beyond the answer: the chain. Every result carries the path that produced it (Maya → Iberia → Madrid → Spain). That receipt is what makes graph memory traceable, and it becomes the star of a later post on auditing agent decisions.

What do the measured results show?

Four multi-hop questions, both retrievers, checked deterministically against the known graph (no LLM judge, so the numbers reproduce):

Question	Vector similarity	Graph traversal
Who do I know that's connected to flights to Spain?	✗	✓
Who do I know connected to an airline that flies to Madrid?	✗	✓
Who works at the Oneworld airline I know?	✓	✓
Which person is linked to airlines in Spain?	✗	✓
Total	1/4	4/4

The one similarity got right is worth pausing on: on question 3 the person node happened to rank high by resemblance alone. Similarity isn't always wrong on multi-hop questions; it's unreliable, while traversal is consistent. That's the actual finding, and it matches what the graph-memory research measures at scale (MAGMA, GAAMA, Zep).

The agent also writes back: told "remember that Maya works at Iberia", the Strands agent calls a remember_fact tool that MERGEs the edge into Neo4j and logs it to agent.state. The memory grows as a graph, one fact per conversation.

When is a graph the wrong choice?

When your memories are independent notes. A graph of disconnected nodes is a slow key-value store with extra steps, plus a database to run and a schema to think about. Skip a graph when nothing in your questions crosses more than one fact. The honest decision line, extending the series' table:

You need	Pick	Why
Facts under known keys	Key-value (post 1)	Exact, instant, zero infrastructure
Search by meaning over independent notes	Vector (post 2)	Similarity is enough when nothing connects
Questions that hop across relationships	Graph (this post)	Only edges answer chain questions, with receipts

Two more honest costs: you design the schema (every edge type must earn a real question: model "who do I know at X?", not everything), and connectivity cuts both ways, because one wrong fact contaminates every traversal that crosses it. That blast-radius problem gets its own post (memory hygiene).

How do you run the demo?

git clone https://github.com/elizabethfuentes12/stop-ai-agents-losing-memory-sample-for-aws
cd stop-ai-agents-losing-memory-sample-for-aws/03-graph-memory-demo
uv venv && uv pip install -r requirements.txt
cp .env.example .env   # OPENAI_API_KEY + your NEO4J_* values
uv run python test_graph_memory.py

Needs a running Neo4j (Desktop, Docker, or the free Aura tier) and OPENAI_API_KEY for model + embeddings (or swap to Amazon Bedrock; the README shows how). The repo's README also documents a real version-churn gotcha (neo4j-graphrag 1.18 emits Cypher 25's SEARCH clause, which fails on servers still defaulting to Cypher 5) and how the demo handles it automatically.

FAQ

What is graph memory for AI agents?

Agent memory stored as a knowledge graph: entities as nodes, facts as typed edges, with a vector index for finding entry points. It answers relationship questions ("who do I know connected to X?") that key-value lookup and vector similarity structurally cannot, and every answer carries the chain of facts that produced it.

Knowledge graph vs vector memory: which does an agent need?

Vector memory when questions match individual memories by meaning; graph memory when answers span several memories connected by relationships. Measured here: vector similarity solved 1 of 4 multi-hop questions, graph traversal 4 of 4. Most production agents eventually want both, similarity to enter the graph and traversal to answer.

Is this the same as GraphRAG?

Same mechanism, different corpus. GraphRAG builds a graph over your documents; graph memory builds one over the user facts the agent accumulated across conversations. The retrieval pattern (vector entry point, then traversal) is identical, which is why the official graph-RAG retriever classes work unchanged here.

Do I need an LLM to build the graph?

Not for this pattern. I write facts as explicit MERGE statements from a tool the agent calls, which keeps results reproducible. LLM entity extraction, such as SimpleKGPipeline, automates graph construction from raw text at the cost of determinism: a production option, not a requirement here.

How do you benchmark agent knowledge-graph memory?

Deterministically: fix a known graph, write multi-hop questions whose answers you can verify by construction, run each retriever, and count. An LLM judging its own retrieval adds noise. The demo's 1/4 vs 4/4 scorecard reproduces run after run because the check is structural, not judged.

Resources

Companion repo, demo 03 with the scorecard and notebook
RAG vs GraphRAG: When Agents Hallucinate Answers, how graph structure prevents hallucinated connections in RAG retrieval
neo4j-graphrag for Python, the two retriever classes used here
MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents, Jiang et al., 2026
GAAMA: Graph Augmented Associative Memory for Agents, Paul et al., 2026
GRAVITY: Structured Anchoring for Long-Horizon Conversational Memory, Sun et al., 2026
Zep: A Temporal Knowledge Graph Architecture for Agent Memory, Rasmussen et al., 2025

Which of the five prompting rules surprised you most? Share in the comments.

Gracias!

🇻🇪 Dev.to Linkedin GitHub Twitter Instagram Youtube

Elizabeth Fuentes LFollow

I help developers build production-ready AI applications through hands-on tutorials and open-source projects.

Build a Full-Stack Music Station with OpenRouter, Amazon Bedrock, and Nuxt

Erik Hanchett — Wed, 26 Aug 2026 00:48:03 +0000

Have you ever been coding and then gotten into that flow state? You know where hours pass by , and it feels to you it's only been a few minutes? Me too. One thing that really helps me get into that state is music. So I create my own music Lo-Fi server called compile and chill.

As a part of this project, I created three radio stations. Each station can generate a 16:9 scene with Amazon Bedrock, compose an instrumental loop with ElevenLabs, and turn an illustration into a six-second video through OpenRouter. Generated files live in private Amazon S3 storage and return to the browser through the Nuxt server.

I also added a Stream Deck API interface!

This tutorial shows how to build this radio station from start to finish.

The complete source code is available in the Compile & Chill repository.

Watch the full video on YouTube.

Prerequisites

You need the following tools for the complete build:

Node.js 22.19 or newer. The locked Nuxt 4.5.2 release requires Node 22.19+, 24.11+, or 26+.
npm 10 or newer.
An AWS account and a configured AWS Command Line Interface (AWS CLI) profile.
The AWS Serverless Application Model (AWS SAM) CLI for the private storage stack.
Access to Stability AI Stable Image Ultra through Amazon Bedrock in us-west-2.
An ElevenLabs API key for music generation.
An OpenRouter API key for animated scenes.

The provider credentials are optional. Without them, the UI, bundled scene, station switching, player, and Focus Block timer still work.

The identity running the app needs bedrock:InvokeModel plus bucket-scoped permissions for s3:GetObject, s3:PutObject, s3:DeleteObject, s3:DeleteObjectVersion, and s3:ListBucketVersions. Use a role or profile scoped to the station bucket rather than an administrator identity.

For this project I included infrastructure as code with SAM to help setup the AWS parts. It's also included in the repo.

Steps

1. Run the station without credentials

Pull down the repo and get started!

git clone https://github.com/ErikCH/compile-and-chill.git
cd compile-and-chill
npm ci
cp .env.example .env
npm run dev

Open http://127.0.0.1:8231. You should see the station UI with the bundled placeholder scene.

The environment file separates each feature:

ELEVENLABS_API_KEY=
BEDROCK_IMAGE_REGION=us-west-2
BEDROCK_IMAGE_MODEL_ID=stability.stable-image-ultra-v1:1
AWS_PROFILE=default
STATION_S3_BUCKET=
STATION_S3_REGION=us-west-2
OPENROUTER_API_KEY=
STATION_BIND_HOST=127.0.0.1
NUXT_CONTROL_TOKEN=

Do not add NUXT_PUBLIC_ to these names. Nuxt exposes public runtime configuration to browser code.

Keep STATION_BIND_HOST on loopback unless you need remote control. For Stream Deck access from another machine, bind to a private VPN interface address, never 0.0.0.0 or a public IP, and set a long random NUXT_CONTROL_TOKEN. The server refuses a non-loopback binding without that token.

2. Put provider calls behind Nuxt server routes

The browser should call your application. Compile & Chill keeps provider credentials in Nuxt runtimeConfig and places provider code under server/api/ and server/utils/.

The relevant section of nuxt.config.ts looks like this:

export default defineNuxtConfig({
  runtimeConfig: {
    elevenLabsApiKey: process.env.ELEVENLABS_API_KEY || '',
    imageGenerationRegion: process.env.BEDROCK_IMAGE_REGION || 'us-west-2',
    imageGenerationModel:
      process.env.BEDROCK_IMAGE_MODEL_ID || 'stability.stable-image-ultra-v1:1',
    stationStorageBucket: process.env.STATION_S3_BUCKET || '',
    stationStorageRegion: process.env.STATION_S3_REGION || 'us-west-2',
    openRouterApiKey: process.env.OPENROUTER_API_KEY || '',
  },
})

This boundary gives you one place to validate requests, clamp paid parameters, redact signed URLs, and translate provider errors into useful HTTP responses.

You can inspect the complete routes in server/api.

3. Generate a scene with Amazon Bedrock

Authenticate with your AWS profile first. This example uses AWS IAM Identity Center:

aws sso login --profile your-profile

Add the profile and model settings to .env:

AWS_PROFILE=your-profile
BEDROCK_IMAGE_REGION=us-west-2
BEDROCK_IMAGE_MODEL_ID=stability.stable-image-ultra-v1:1

The image route validates the station mode, visual style, developer presentation, and optional seed. It then calls generateStationScene() in server/utils/station-generation.ts.

The provider payload is small:

const payload = {
  prompt: buildScenePrompt(mode, direction, {
    visualStyle,
    developer,
    diversitySeed: seed,
  }),
  // Shortened here. The source contains the complete negative prompt.
  negative_prompt: [
    'logos, readable text, watermark, rear view, motion blur, extra fingers',
    visualStyle === 'illustrated'
      ? 'photograph, live action, photorealistic skin, 3D render'
      : 'cartoon, anime, cel shading, flat illustration',
  ].join(', '),
  mode: 'text-to-image',
  aspect_ratio: '16:9',
  output_format: 'png',
  seed,
}

const response = await client.send(
  new InvokeModelCommand({
    modelId: config.imageGenerationModel,
    contentType: 'application/json',
    accept: 'application/json',
    body: JSON.stringify(payload),
  }),
)

The server normalizes the seed before constructing the payload. This matters because Number(undefined) becomes NaN, and JSON serializes NaN as null. Stable Image Ultra expects an integer.

The prompt also ties visual style to output type. Static scenes use a realistic style. Animated scenes use a clearly illustrated 2D style so the source is visibly non-photorealistic. I added that distinction after a provider's person-likeness classifier refused one of the illustrated anchors.

Checkpoint: restart the app, open Visual settings, choose Realistic static, and generate one scene. If the model is unavailable in the configured region, the route returns the provider error and model ID instead of silently switching models.

4. Add looping music with ElevenLabs

The UI sends a 60-second duration, and the server should reject or normalize nonnumeric input before applying its 10-to-60-second bounds. This hardened version avoids sending null to the provider if another client calls the route with an invalid value:

const requestedDuration = Number(input.durationSeconds ?? 60)
const durationSeconds = Number.isFinite(requestedDuration)
  ? Math.min(60, Math.max(10, requestedDuration))
  : 60

const response = await fetch(
  'https://api.elevenlabs.io/v1/music?output_format=mp3_44100_128',
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'xi-api-key': config.elevenLabsApiKey,
    },
    body: JSON.stringify({
      prompt: buildMusicPrompt(input.mode, input.direction),
      music_length_ms: durationSeconds * 1000,
      model_id: 'music_v2',
      generation_mode: 'loop',
      force_instrumental: true,
      sign_with_c2pa: true,
    }),
  },
)

generation_mode: 'loop' is one part of making the repeat sound natural. The prompt also asks for no intro pickup, ending cadence, or fade-out. A track written like a normal song sounds broken when it jumps from the ending back to the first beat.

Compile & Chill keeps one generated track per station mode. Switching from Deep Work to Rainy Debug and back reuses the existing track instead of making another paid request.

Checkpoint: add ELEVENLABS_API_KEY to .env, restart the server, and generate one 60-second track. Switch modes and confirm that returning to the original mode reuses its audio.

5. Deploy the private media library

The repository includes an AWS SAM template managed by AWS CloudFormation. Validate and deploy it:

sam validate \
  --template-file infra/compile-and-chill-private-station.yaml \
  --region us-west-2

sam deploy \
  --region us-west-2 \
  --stack-name compile-and-chill-private-station \
  --template-file infra/compile-and-chill-private-station.yaml \
  --no-confirm-changeset \
  --no-fail-on-empty-changeset

Read the media bucket output:

aws cloudformation describe-stacks \
  --region us-west-2 \
  --stack-name compile-and-chill-private-station \
  --query "Stacks[0].Outputs[?OutputKey=='StationMediaBucketName'].OutputValue" \
  --output text

Copy that value into .env:

STATION_S3_BUCKET=YOUR_STACK_OUTPUT
STATION_S3_REGION=us-west-2

The stack creates two versioned buckets. One stores generated station media. The other stores AWS CloudTrail data-event logs encrypted with AWS Key Management Service (AWS KMS). An Amazon CloudWatch alarm watches delete requests against the media bucket.

The application assigns each browser profile a random station ID in an HttpOnly, SameSite cookie. Objects use this layout:

stations/<station-id>/manifest.json
stations/<station-id>/<mode>/scene.png
stations/<station-id>/<mode>/scene.mp4
stations/<station-id>/<mode>/music.mp3

The browser receives same-origin URLs such as /api/library/assets/deepWork/scene. The route reads the matching S3 object into server memory and returns it. The current implementation is a proxy, not a streaming pass-through, so account for memory when increasing the 50 MiB video limit.

The storage implementation is in server/utils/station-library.ts.

Checkpoint: regenerate a scene or track, reload the page in the same browser profile, and confirm that the media returns. Open a private browsing window and confirm that it starts with a separate station.

6. Turn one illustration into a seamless video loop

Animated mode generates a fresh illustrated anchor with Amazon Bedrock, saves it to S3, and creates a 300-second presigned read URL. OpenRouter needs that temporary URL because its video provider must download the input image.

The trick is to submit the same anchor as both frame constraints:

body: JSON.stringify({
  model: modelId,
  duration: 6,
  resolution: '720p',
  aspect_ratio: '16:9',
  generate_audio: false,
  prompt: [
    'Animate this 2D cartoon illustration of a fictional character.',
    'Use a locked-off camera and keep the illustrated composition unchanged.',
    'The provided illustration is both the first and last frame.',
    'Allow only tiny two-hand typing motion and one gentle blink.',
    'No zoom, pan, cut, identity change, new objects, or geometry changes.',
    'Create a silent seamless six-second ambient loop.',
  ].join(' '),
  frame_images: [
    {
      type: 'image_url',
      image_url: { url: sourceUrl },
      frame_type: 'first_frame',
    },
    {
      type: 'image_url',
      image_url: { url: sourceUrl },
      frame_type: 'last_frame',
    },
  ],
})

The model can move away from the anchor, but the matching frame constraints request a return to the same composition at the end. The prompt limits movement to a small typing motion and one blink, with no camera movement, reframing, cuts, new objects, or identity changes.

The app discovers compatible video models at runtime. A candidate must support six seconds, 720p, first_frame, and last_frame. It prefers Seedance 2.0 Fast, then checks a short fallback list.

The fallback policy is intentionally narrow:

HTTP 402 stops because credits are unavailable.
HTTP 429 stops because the account is rate limited.
An input-image moderation refusal may try the next compatible model because no video job was created.
Other failures stop rather than risk submitting and billing a duplicate job.

See the complete policy in server/utils/openrouter-video.ts.

Video generation takes minutes rather than seconds, and provider timing and prices change. Treat the request as a paid background job even if the first version runs inside one long HTTP request.

7. Report real progress during a long request

A spinner cannot tell the user whether a three-minute provider job is moving or stuck. Compile & Chill keeps a process-local job registry with six phases:

creating anchor
preparing source
submitting video
rendering loop
downloading loop
saving privately

The browser creates a job ID, starts the generation request, and polls the Nuxt status endpoint every 750 milliseconds. Nuxt polls the provider every 15 seconds, up to 48 times. These are separate loops.

The registry also rejects a second active job for the same station with HTTP 409. That protects the user from duplicate clicks, but it is not a distributed queue. A process restart loses status, and multiple Nuxt instances would each have their own registry. For a multi-instance deployment, move job state and concurrency control to a shared data store and worker queue.

8. Validate the build

Run the same checks used for the source project:

npm test
npm run typecheck
npm run build

The current repository has 26 passing tests plus a clean type check and production build.

Then test one provider at a time:

Start with no credentials and verify the UI.
Add Amazon Bedrock and generate one static scene.
Add ElevenLabs and generate one track.
Deploy the storage stack and verify reload persistence.
Add OpenRouter last, acknowledge the paid operation, and generate one animated loop.

This order keeps failures small. If the video path fails, you already know that image generation, storage, and browser identity work independently.

Cleanup

The cleanup steps remove data permanently. Download anything you want to keep before continuing. The buckets and KMS key use retention policies, so they can remain after stack deletion and may continue to incur charges.

First, use Delete saved station inside each browser profile whose generated media should be removed. This purges object versions and delete markers under that profile's station prefix.

Before deleting the stack, record the retained resource names:

MEDIA_BUCKET=$(aws cloudformation describe-stacks \
  --region us-west-2 \
  --stack-name compile-and-chill-private-station \
  --query "Stacks[0].Outputs[?OutputKey=='StationMediaBucketName'].OutputValue" \
  --output text)

AUDIT_BUCKET=$(aws cloudformation describe-stacks \
  --region us-west-2 \
  --stack-name compile-and-chill-private-station \
  --query "Stacks[0].Outputs[?OutputKey=='CloudTrailLogBucketName'].OutputValue" \
  --output text)

KMS_KEY_ID=$(aws kms describe-key \
  --region us-west-2 \
  --key-id alias/compile-and-chill-cloudtrail \
  --query KeyMetadata.KeyId \
  --output text)

Delete the non-retained stack resources:

sam delete \
  --stack-name compile-and-chill-private-station \
  --region us-west-2

The media bucket, audit bucket, and KMS key are retained by design. Empty all versions and delete markers from both versioned buckets in the S3 console, then delete the buckets:

aws s3api delete-bucket --bucket "$MEDIA_BUCKET" --region us-west-2
aws s3api delete-bucket --bucket "$AUDIT_BUCKET" --region us-west-2

Schedule the retained KMS key for deletion after confirming that you no longer need the encrypted audit logs:

aws kms schedule-key-deletion \
  --region us-west-2 \
  --key-id "$KMS_KEY_ID" \
  --pending-window-in-days 7

KMS key deletion has a waiting period. After deletion, data encrypted only by that key cannot be recovered.

For local cleanup:

rm -rf node_modules .nuxt .output
rm .env

Finale

This has been a very fun project. Let me know if you try it out!

You can explore every route, prompt, infrastructure resource, and test in the Compile & Chill source code. For another Nuxt project that keeps AI actions behind explicit user approval, read How to Build an AI Agent That Asks Permission First. Leave a comment on what you think! Thank!

Observability for AI Agents with OpenTelemetry

Elizabeth Fuentes L — Mon, 24 Aug 2026 23:32:49 +0000

AI agent observability means capturing your agent's reasoning cycles, tool calls, and token usage as metrics, traces, and logs. In this guide I build it in three layers with OpenTelemetry (OTEL), then take the same agent to production on Amazon Bedrock AgentCore.

Your AI agent is in production. A user asks it a question, and it takes thirty seconds, calls five tools, and gives an answer you can't explain. What did it actually do? Which tools did it call? How many times did it "think" before answering? If you can't answer that, you're running agents blind. Traditional monitoring won't help you here: CPU, RAM, and uptime watch the machine, not the reasoning.

In this post I make a travel-booking agent's normal behavior visible. No injected failures, no chaos experiments. A real agent doing its job, seen through four increasingly capable lenses:

Agent metrics: what the run cost, with zero extra configuration
OpenTelemetry traces: the path the agent took, step by step
Custom trace attributes: your business context, on the same trace
Production: the same visibility in Amazon CloudWatch via Amazon Bedrock AgentCore

Everything comes from a runnable sample repository: observability-for-agents-sample-for-aws. Each demo is keyed to a specific section of the Strands Agents observability documentation.

A note on the stack. The demos use Strands Agents, an open-source SDK that emits OpenTelemetry natively. Metrics, hierarchical traces, and span attributes are general agent-observability concepts. The same patterns carry over to other agent frameworks, and Strands is model-agnostic: works with any LLM provider (Amazon Bedrock, Anthropic, local models via Ollama, or others).

What agent are we observing?

All four demos instrument the same travel agent: it searches real sandbox flight fares (Duffel API), checks real weather (Open-Meteo), and books flights into a local SQLite ledger. The only thing that changes, demo to demo, is how much of the agent's internal behavior becomes visible, and where that visibility lives:

Layer 1 / What metrics do you get with zero configuration in Strands?

Every Strands agent run already carries its own metrics: reasoning cycle count, token usage, and per-tool call counts and timings, exposed through result.metrics.get_summary(). No extra install, no exporter, no setup. Every AI agent run has a shape, and that shape is captured before you configure anything.

Compare two lenses on the same run. First, traditional logging:

DEBUG | strands.tools.executors._executor | tool_use=<...name': 'search_flights'...> | streaming
DEBUG | strands.tools.executors._executor | tool_use=<...name': 'get_weather'...> | streaming
DEBUG | strands.tools.executors._executor | tool_use=<...name': 'book_flight'...> | streaming
John Doe's flight from JFK to MIA has been successfully booked ... booking reference BK-JSFPJ5 ...

Useful for "did this run". Useless for "how much did it cost". Now the built-in metrics, one method call:

result = agent("Book a one-way flight from JFK to MIA...")
print(result.metrics.get_summary())

{
  "total_cycles": 3,
  "total_duration_s": 5.13,
  "accumulated_usage": {"inputTokens": 2520, "outputTokens": 209, "totalTokens": 2729},
  "tool_usage": {
    "search_flights": {"call_count": 1, "success_count": 1, "average_time_s": 0.721},
    "get_weather":     {"call_count": 1, "success_count": 1, "average_time_s": 1.434},
    "book_flight":     {"call_count": 1, "success_count": 1, "average_time_s": 0.006}
  }
}

This is real output from an agent run, and every field answers a question a log line can't:

total_cycles: 3. An agent is not a single function call, it's a loop: the model calls a tool, thinks again with the result, calls another. Three cycles here. If this number is ever ten for a basic question, something's wrong, and now you can see it.
accumulated_usage. 2,729 tokens for the whole booking. Notice input is roughly ten times output; that's typical for agents, because every tool result gets fed back into the model. This is the number that tells you how heavy each request really is.
tool_usage. Three tools, three completely different performance profiles: search_flights at 0.7 s (a real API call), get_weather at 1.4 s (another API), book_flight at 6 milliseconds (a local write). Without this breakdown, "the agent is slow" is a mystery. With it, it's a diagnosis.

One more habit worth building from day one: the demo also queries the booking database directly, so you can cross-check what the agent said ("booked!") against what actually persisted. In this run, the agent's claim and the ground truth agreed.

Honest caveat: in Strands 1.47.0, accumulated_metrics.latencyMs reads 0 for some LLM providers. It ships as a TODO in the provider streaming code (I verified this by reading the installed SDK source). Token counts and per-tool timings are accurate everywhere; treat the top-level latencyMs as not-yet-implemented.

Layer 2 - How do you trace an AI agent with OpenTelemetry?

Metrics are a flat snapshot, traces are the path. A trace records the full hierarchy of one request: which reasoning cycle called which model invocation, which invocation triggered which tool, in what order, with timestamps. In Strands, turning on OpenTelemetry tracing is two lines:

from strands.telemetry import StrandsTelemetry

strands_telemetry = StrandsTelemetry()
strands_telemetry.setup_console_exporter()   # print the span tree to stdout
# strands_telemetry.setup_otlp_exporter()    # or send it to a collector (Jaeger, CloudWatch, ...)

StrandsTelemetry wires up the OpenTelemetry SDK and registers it as the global tracer provider. Every Agent(...) call after this is automatically instrumented; there is no manual span-wrapping of your own agent loop. Run the same travel query, and the console prints the documented span hierarchy:

invoke_agent Strands Agents      # the whole run (top-level span)
  execute_event_loop_cycle       # one reasoning cycle
    chat                         # the model invocation for that cycle
    execute_tool search_flights  # one span per tool call
    execute_tool get_weather
    execute_tool book_flight

Each span carries attributes. The invoke_agent span holds the totals (gen_ai.usage.total_tokens: 2725, gen_ai.request.model), and each execute_tool span holds that one call's gen_ai.tool.name, gen_ai.tool.call.id, tool.status, and the formatted tool result. That's enough to answer "did book_flight fail, and what did it return?" from the trace alone, without re-running anything.

And because this is standard OpenTelemetry, the console exporter is interchangeable with any OTEL backend. Want a visual UI locally? One Docker command starts Jaeger, one environment variable points the exporter at it, and the agent code doesn't change.

Layer 3 — How do you add business context to agent traces?

Out of the box, spans carry technical attributes: tool name, token counts, status. None of those answer "was this a high-value booking?". That context is yours to add, and the Strands traces guide documents two mechanisms. The demo uses both.

Static context. Agent-level trace_attributes attach metadata (session ID, user ID, tags) to every span the agent produces:

agent = Agent(
    tools=[search_flights, get_weather, book_flight],
    trace_attributes={"session.id": "demo-03-custom-trace-attributes"},
)

Dynamic context. A hook tags the active span at the exact moment a business rule fires. An AfterToolCallEvent callback runs right after each tool call finishes; at that moment, the currently open span is that tool's execute_tool span, so trace.get_current_span() reaches it directly:

from opentelemetry import trace
from strands.hooks import AfterToolCallEvent, HookProvider, HookRegistry

VIP_THRESHOLD = 50.0  # low on purpose, so sandbox fares cross it

class TagVipBookings(HookProvider):
    def __init__(self, threshold: float):
        self.threshold = threshold

    def register_hooks(self, registry: HookRegistry, **kwargs) -> None:
        registry.add_callback(AfterToolCallEvent, self._tag_if_vip)

    def _tag_if_vip(self, event: AfterToolCallEvent) -> None:
        if event.tool_use.get("name") != "book_flight":
            return
        amount = float(event.tool_use.get("input", {}).get("amount", 0))
        span = trace.get_current_span()
        span.set_attribute("business.booking_amount_usd", amount)
        span.set_attribute("business.vip_booking", amount >= self.threshold)

Run the agent, find the execute_tool book_flight span, and the custom attributes sit right alongside the SDK's own:

{
  "name": "execute_tool book_flight",
  "attributes": {
    "gen_ai.tool.name": "book_flight",
    "gen_ai.tool.status": "success",
    "business.booking_amount_usd": 88.73,
    "business.vip_booking": true
  }
}

The detail that matters: this lives on the trace, not in the conversation. The model never sees it. Trace attributes are OpenTelemetry span metadata, entirely separate from the message list, so they add exactly zero tokens to the agent's context. But six months from now, "show me every VIP booking this quarter" is a search on your traces.

Production — where does agent observability live when you deploy?

Everything so far lived in your terminal. That's fine while you're developing, but your agent isn't going to run in your terminal, and you won't be there watching console output. The payoff of building on an open standard: everything we made (metrics, traces, attributes) is OpenTelemetry data, and OTEL data is portable. Swap the exporter, and the agent code doesn't change.

Demo 04 deploys the same travel agent to Amazon Bedrock AgentCore Runtime. The production architecture:

The agent runs on AgentCore Runtime (the code change is one decorator: @app.entrypoint).
The three tools become AWS Lambda functions served through an AgentCore Gateway (a Model Context Protocol endpoint with IAM auth).
book_flight writes to Amazon DynamoDB instead of SQLite. Same tool, same booking, real storage.
One added dependency, aws-opentelemetry-distro (the AWS Distro for OpenTelemetry), ships the OTEL data to CloudWatch. The Runtime runs your agent under its auto-instrumentation automatically.
One-time account setup: turn on CloudWatch Transaction Search. Without it, traces don't appear in the console (official guide).

After invoking the deployed agent, open CloudWatch GenAI Observability and you get three views:

Agents View: every AgentCore agent in your account, with invocations, latency, and error rates.
Sessions View: every conversation. Remember the session.id from Layer 3? This is where it pays off: it's how you go from "something went wrong" to "here's the exact conversation".
Traces View: the same span tree you learned to read in your terminal (invoke_agent → cycles → chat + execute_tool), now rendered as a visual timeline, with every attribute searchable, including business.vip_booking.

The repo ships the deployment two ways: an AWS CDK stack (cdk deploy, and cdk destroy tears down everything, DynamoDB table included) and a step-by-step boto3 notebook if you want to see every API call.

Frequently asked questions

What's the difference between logs, metrics, and traces for an AI agent?
Logs are timestamped text records of what happened ("tool X was called"). Metrics are measurements of those events (how many times, how long, how many tokens). Traces are the hierarchical timeline connecting them. A log tells you that something happened, a metric tells you how much it cost, a trace shows you the path.

Do I need OpenTelemetry for basic agent metrics?
No. In Strands, result.metrics.get_summary() is part of the base SDK: no [otel] extra, no exporter, no collector. OpenTelemetry comes in when you want traces (Layer 2 onward).

Do I need a collector to see traces?
No. setup_console_exporter() prints the full span tree to your terminal. Use setup_otlp_exporter() when you want a real backend: Jaeger locally, or CloudWatch in production.

Do custom trace attributes cost extra tokens?
No. They're OpenTelemetry span metadata, entirely separate from the message list the model sees. The model never reads them.

Does this only work with Strands Agents or AWS?
No. An agent loop, hooks, metrics, and OpenTelemetry tracing are general agent-observability concepts. The demos use Strands because these primitives are built in, and Strands is model-agnostic: works with any LLM provider with no change to the agent code. The same patterns carry over to other agent frameworks.

How does Strands' built-in observability compare to manual instrumentation?
Strands emits OpenTelemetry spans natively with no manual wrapping. In frameworks without native OTEL support, you'd instrument each tool call and reasoning cycle yourself using the OpenTelemetry SDK directly. The data structure is identical — only the setup differs.

Wrap-up: three layers, one standard

Agent observability, as built here, is three layers:

Metrics tell you what your agent did and how efficiently: cycles, tokens, tool timings. Free with the SDK.
Traces show you the path it took: every decision, in order, with full context. Two lines to turn on.
Trace attributes add your context to that path, so you can search it by what matters to your business. A dictionary and a hook.

You build all three once, they travel on OpenTelemetry, and a managed runtime takes them to production with minimal configuration.

One deliberate boundary: this post is about observability, seeing what an agent already does. It is not about resilience or chaos testing (injecting failures and recovering from them); that's a different, related story. And once you can see what your agent does, the natural next step is to validate it. Evaluation builds on exactly this data. You can't validate what you can't see.

Try it yourself

The travel agent, all four demos (each self-contained, with a script and a Jupyter notebook), and both production deployment paths are in the sample repository:

→ observability-for-agents-sample-for-aws

You need Python 3.10+, uv, an API key for your LLM provider (the demos support multiple providers), and a free Duffel sandbox token. Demo 01 runs in under a minute:

git clone https://github.com/elizabethfuentes12/observability-for-agents-sample-for-aws.git
cd observability-for-agents-sample-for-aws/01-agent-metrics
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt
cp .env.example .env   # fill in your LLM provider API key and DUFFEL_API_KEY
uv run python test_agent_metrics.py

Clone it, run it, and stop running your agents blind. Which of your agents would surprise you most if you could see every cycle? Tell me in the comments.

References: Strands Agents observability docs · OpenTelemetry · AgentCore Observability · CloudWatch GenAI Observability

Gracias!

🇻🇪 Dev.to Linkedin GitHub Twitter Instagram Youtube

Elizabeth Fuentes LFollow

I help developers build production-ready AI applications through hands-on tutorials and open-source projects.

Amazon DynamoDB Vector Search. No Separate Vector Store

Elizabeth Fuentes L — Fri, 21 Aug 2026 22:40:38 +0000

📦 Clone and ⭐ stop-ai-agents-losing-memory-sample-for-aws

Part 1 of this post showed how keyword search misses semantic questions and measured two vector backends: FAISS and Amazon S3 Vectors (managed), on the same traveler memories. Both found the answer. The difference was deployment: local vs cloud-managed.

This part adds a third vector backend: Amazon DynamoDB Vector Search, generally available since 2025. The question and the memories are identical. Only the backend changes.

stored:   dietary_notes: "Vegetarian; severe shellfish allergy, strictly no
          crustaceans or mollusks."

asked:    "What should I avoid eating when I go out for dinner on this trip?"

DynamoDB Vector Search: top hit (score 0.231)  answer found: True

What is Amazon DynamoDB Vector Search?

It is a vector index added to an existing DynamoDB table. Not a separate service. You define a VectorIndexes block when you create (or update) the table, and DynamoDB stores the embeddings as a List attribute on each item. Queries use the SearchVectors API.

The key difference from S3 Vectors: the vectors live in the same table as your operational data. If your agent already reads user preferences or travel records from DynamoDB, you can add a vector index to that same table and query by meaning without provisioning another service.

	Amazon S3 Vectors	Amazon DynamoDB Vector Search
Where vectors live	Dedicated vector bucket	Inside a DynamoDB table
Operational data collocated	No	Yes
Query latency	~100-200 ms	Single-digit ms
Billing model	Per-query + storage	On-demand (PAY_PER_REQUEST)
Accuracy	✅ same (same embeddings)	✅ same (same embeddings)
Survives restart	Yes	Yes
Infrastructure to manage	None	None
Best for	Dedicated vector memory, no operational data to manage	Agents that already use DynamoDB, or want one service for data + embeddings

Both are valid choices. S3 Vectors is purpose-built for dedicated vector workloads and the right fit when you want memory completely separate from your operational data. DynamoDB Vector Search is the right fit when your agent data is already in DynamoDB and you want one service for both.

(This demo uses Strands Agents.)

How does the embedding comparison look?

Same question, same Titan V2 embeddings, four backends side by side:

Store	Finds answer	cos_sim
Key-value (keyword scan)	No	n/a
FAISS	Yes	0.231
Amazon S3 Vectors	Yes	0.231
Amazon DynamoDB Vector Search	Yes	0.231

All three vector backends return the same top hit with the same score, because they use the same Amazon Titan Text Embeddings V2 model. The embedding call (~510 ms) still dominates end-to-end latency for all of them. What changes is the query after the embedding.

How do you add a vector index to a DynamoDB table?

DynamoDB Vector Search requires on-demand billing (PAY_PER_REQUEST). The vector index is declared when creating the table:

client.create_table(
    TableName="agent-memory-demo-ddb",
    BillingMode="PAY_PER_REQUEST",           # required for vector indexes
    KeySchema=[{"AttributeName": "memory_key", "KeyType": "HASH"}],
    AttributeDefinitions=[{"AttributeName": "memory_key", "AttributeType": "S"}],
    VectorIndexes=[{
        "IndexName": "memory-vector-index",
        "VectorAttribute": {"AttributeName": "embedding"},
        "Dimensions": 1024,
        "DistanceFunction": "COSINE",
        "Projection": {"ProjectionType": "ALL"},
    }],
)

The demo self-provisions the table and index if missing: no console steps, no CDK required.

How do you write and query vectors?

Embeddings are stored as a DynamoDB List attribute alongside the rest of the item:

client.put_item(
    TableName="agent-memory-demo-ddb",
    Item={
        "memory_key":  {"S": "dietary_notes"},
        "text":        {"S": "Vegetarian; severe shellfish allergy..."},
        "embedding":   {"L": [{"N": str(float(f))} for f in vector]},  # 1024 floats
    },
)

Querying uses the SearchVectors API with the same AttributeValue format:

resp = client.search_vectors(
    TableName="agent-memory-demo-ddb",
    IndexName="memory-vector-index",
    SearchVector=[{"N": str(float(f))} for f in question_vector],
    TopK=3,
)

Score note: SearchVectors returns a cosine distance (lower = more similar). The demo converts it to cosine similarity (1.0 − score) so the output is directly comparable to FAISS and S3 Vectors.

Does the index survive a restart?

Yes. It's DynamoDB. A fresh client instantiated after the demo runs still sees every item:

fresh = DynamoDBVectorStore()
fresh.count() == 10   # True, all 10 memories are there

This is the same restart test run in Part 1 for S3 Vectors. Both pass.

How do you run Test 4?

Test 4 runs as part of the existing test_vector_memory.py in the companion repo:

git clone https://github.com/elizabethfuentes12/stop-ai-agents-losing-memory-sample-for-aws
cd stop-ai-agents-losing-memory-sample-for-aws/02-vector-memory-demo
uv venv && uv pip install -r requirements.txt
uv run python test_vector_memory.py

Needs AWS credentials (aws configure) for Titan embeddings (Bedrock), S3 Vectors, and DynamoDB. The demo creates the DynamoDB table and vector index automatically if they don't exist. Requires boto3>=1.43.72 (SearchVectors was added in that release).

When do you pick DynamoDB over S3 Vectors?

You have	Pick
No existing DynamoDB table; memory is the only use case	S3 Vectors, purpose-built for dedicated vector workloads
An existing DynamoDB table with user data	DynamoDB Vector Search, add the index to the same table; one service, one billing model
Need sub-100 ms query latency after the embedding call	DynamoDB Vector Search, single-digit ms where S3 Vectors is subsecond
High QPS, hybrid search, or advanced filtering	Dedicated vector database (OpenSearch, Qdrant, etc.)

FAQ

Can I add a vector index to an existing DynamoDB table?
Yes. Use update_table with VectorIndexUpdates to add the index to a table that already has data. Existing items without the embedding attribute won't appear in vector queries until you backfill their embeddings and update the items.

Does DynamoDB Vector Search work in all regions?
Check regional availability; the feature is GA but not in every region on launch day.

What's the cost compared to S3 Vectors?
DynamoDB Vector Search uses on-demand billing: you pay for read/write capacity units and storage on the table. S3 Vectors charges per query and per stored vector. For agent memory workloads (infrequent queries, small number of vectors per user) both are low cost; the deciding factor is architecture, not price.

Why does SearchVectors return a distance and not a similarity?
SearchVectors returns cosine distance (1 − cosine_similarity), where 0 means identical and 1 means opposite. The demo converts with 1.0 − score to get cosine similarity for easy comparison with FAISS (which returns inner product of normalized vectors, equivalent to cosine similarity) and S3 Vectors (which also returns 1 − distance).

Resources

Companion repo, demo 02 with the full 4-backend test
Amazon DynamoDB Vector Search, Developer Guide
Amazon DynamoDB Vector Search GA announcement
Amazon S3 Vectors, User Guide
Amazon Titan Text Embeddings V2
Part 1, FAISS and S3 Vectors

Which surprised you more: the single-digit millisecond DynamoDB latency, or the fact that the cosine similarity score is identical across all four backends? Share in the comments.

Gracias!

🇻🇪 Dev.to Linkedin GitHub Twitter Instagram Youtube

Elizabeth Fuentes LFollow

I help developers build production-ready AI applications through hands-on tutorials and open-source projects.

I Watched My Coding Agent Refuse to Leak a Secret

Jonathan Vogel — Fri, 21 Aug 2026 22:07:05 +0000

A coding agent with AWS access grabs a secret the obvious way and drops the plaintext into its context. AWS Secrets Manager shipped a skill that blocks that and hands the agent a safe path instead. I ran it against a live account in Claude Code and pushed on the edges.

Give a coding agent shell access with AWS credentials, ask it to hit a database and watch what it does. It reaches for the password the obvious way.

$ aws secretsmanager get-secret-value --secret-id agent-toolkit-blog-demo/api-token --query SecretString --output text --region us-east-1
{"username":"demo_user","host":"demo-db.example.internal","password":"FAKE-pw-asdf","token":"FAKE-token-asdf"}

That value is now definitely in the model's context window and conversation history. It could be sitting in a session transcript on disk and end up passing into whatever tool call the agent makes next. You thought you were using Secrets Manager because you wanted to follow best practices and not have a password sitting around.

I'll admit I've done this. I've even pasted secrets straight up into the agent. When you gotta move fast, sometimes you let security take a backseat. Don't be me. Now when you're using agents to build on AWS, you can be fast and secure!

AWS Secrets Manager shipped this nifty feature: a secret safety skill in the aws-core plugin of the Agent Toolkit for AWS. The idea here is simple. An agent should be able to use a secret without ever seeing it. I spent an afternoon running it against a throwaway secret in a real account, then trying to get around it. Here is what happened.

Before I installed the plugin I told Claude Code the secret was a fake throwaway and asked it to fetch the value and reply in chat. It did it without blinking. With the plugin enabled the same prompt died on the spot, and the part I did not expect is that the skill shut it down before the hook even ran.

What the toolkit is, in one screen

If you're doing stuff with AWS using an agent and have not yet set up the Agent Toolkit for AWS, you need to get it set up. Below is some of the "what" around the toolkit:

AWS MCP Server. A managed endpoint your agent talks to for all things AWS. Model training data can be stale, this gives you the capability of running the right AWS CLI commands, searching current AWS docs and logging calls in CloudTrail.
Skills. Task runbooks in markdown, loaded on demand, written by people who ran the workflow and found where agents get stuck. The secret safety skill is one of these.
Plugins. A single install that bundles the MCP Server config and a set of skills. aws-core is the one to start with.

Using the toolkit is free. Skills and doc search work without credentials. API calls need AWS credentials. You pay for any AWS resources your agent deploys.

The demo

I created a throwaway secret with fake values so nothing real was ever at risk.

aws secretsmanager create-secret \
  --name agent-toolkit-blog-demo/api-token \
  --secret-string '{"username":"demo_user","host":"demo-db.example.internal","password":"FAKE-pw-asdf","token":"FAKE-token-asdf"}' \
  --region us-east-1

You already saw the before. get-secret-value hands back the plaintext and it enters the agent's world.

Here's the command to install the toolkit in Claude Code:

/plugin install aws-core@claude-plugins-official

If that comes back with Plugin not found, your local marketplace index is stale. Run /plugin marketplace update claude-plugins-official and install again. If the marketplace is missing rather than stale, update will not help and you need /plugin marketplace add anthropics/claude-plugins-official first.

The skill activates on its own. So does a second piece I will come back to. One thing to know before you try it. Hooks load when the session starts, so restart your agent after installing or the block will not be there yet. Restart, ask the agent to fetch the secret and the request does not run.

The automatic block

A PreToolUse hook sits in front of the agent's tool calls. When the call would fetch a secret value, the hook denies it before it executes and hands the agent a message that points at the safe path.

Direct secret fetching is blocked. Use {{resolve:secretsmanager:secret-id:SecretString:key}} with asm-exec instead. Run /aws-secrets-manager for details.

Worth being precise about which layer does what here. When I asked in plain language, the agent never tried at all. The skill had already told it not to, so it declined on its own and pointed me at the safe path, and the hook never fired. I only saw the message above once I insisted it actually run the command. The skill is the part that changes the agent's mind and the hook is the backstop for when it tries anyway.

I wanted to know how much this actually covers. I tested ten different scenarios to see where the agent might want to reach for a secret and where I might get the skill to trip up when it shouldn't be flagging at all.

STATUS | expected | got   | case
-------+----------+-------+-----------------------------------------
  OK   | deny     | deny  | Bash: aws secretsmanager get-secret-value
  OK   | deny     | deny  | Bash: aws secretsmanager batch-get-secret-value
  OK   | deny     | deny  | use_aws: secretsmanager GetSecretValue
  OK   | deny     | deny  | run_script: boto3 get_secret_value(...)
  OK   | deny     | deny  | Bash: direct SMA daemon curl localhost:2773
  OK   | deny     | deny  | Bash: python3 -c inline boto3 get_secret_value
  OK   | allow    | allow | Bash: asm-exec with {{resolve:...}}
  OK   | allow    | allow | Bash: grep that only mentions get-secret-value
  OK   | allow    | allow | use_aws: secretsmanager CreateSecret (a write, not a fetch)
  OK   | allow    | allow | use_aws: s3 ListBuckets

Results are what I expected, well done agent toolkit. It catches the CLI fetch, the batch fetch, the structured API call, a boto3 call buried in a Python script, a curl straight at the local Secrets Manager Agent daemon and an inline python3 -c one-liner. It does not trip on a grep for the bare string get-secret-value, and it leaves writes and unrelated calls alone. The allowlist for read-only tools is narrower than I assumed though. Search for the full phrase aws secretsmanager get-secret-value and you get denied whether you run it through grep, rg or echo, because the CLI pattern is checked before the allowlist ever applies. I tripped that one myself while grepping my own notes for this post.

What sold me is that both layers push the same direction. The skill has already taught the agent what to do instead, so whether it stops itself or gets stopped by the hook, it does not stall. It rewrites its own command to use the safe path.

Resolve without seeing

The safe path is a dynamic reference and a small wrapper called asm-exec.

asm-exec -- curl -sS \
  -H "Authorization: Bearer {{resolve:secretsmanager:agent-toolkit-blog-demo/api-token:SecretString:token}}" \
  https://api.example.com/data

The command the agent constructs holds a placeholder, not a value. asm-exec scans the arguments, resolves each {{resolve:...}} reference inside its own process, then hands the resolved arguments to the real command. The plaintext never lands in the model's context window or the session transcript.

I pointed it at a local listener that records what it receives, so I could see both sides at once.

Agent-visible command:
  asm-exec -- curl -sS -H 'Authorization: Bearer {{resolve:secretsmanager:agent-toolkit-blog-demo/api-token:SecretString:token}}' http://127.0.0.1:8799/data

Response the agent sees:
  {"authenticated": true, "msg": "token accepted"}

Header the listener actually received:
  Bearer FAKE-token-asdf

The token reached the target. The agent saw a placeholder going in and an API response coming back. It never saw the token.

To be sure I was not fooling myself, I resolved the reference and piped it straight into shasum instead of a command, then compared against the hash of the known fake token.

$ printf %s "FAKE-token-asdf" | shasum -a 256
4953fa5206a3d4a298e6aab9f9c2f74ae50ec2d6f07f374a2df478386674593e  -

$ asm-exec -- sh -c 'printf %s "{{resolve:secretsmanager:agent-toolkit-blog-demo/api-token:SecretString:token}}" | shasum -a 256'
4953fa5206a3d4a298e6aab9f9c2f74ae50ec2d6f07f374a2df478386674593e  -

Same hash. The real value was resolved, used and never printed.

Then the part that closes the loop. I grepped the agent-visible transcript from both runs for the fake values.

-- unguarded path (aws secretsmanager get-secret-value) --
   FAKE-pw-asdf     found
   FAKE-token-asdf  found
-- safe path (asm-exec + {{resolve}}) --
   FAKE-pw-asdf     not found
   FAKE-token-asdf  not found

One path writes your secret into a file on disk. The other runs the exact same task without ever writing it down.

How the pieces fit

One thing I like about the design. asm-exec never shells out to aws secretsmanager get-secret-value itself, so the plaintext is never written to a local process's stdout where something could scrape it.

Where this bites

I've talked about how cool this is but I also promised to go over some rough edges. I want to be clear the skill is useful but it is also a best-effort defense. Even AWS says so in the docs. It stops the common leak but it doesn't mean you should stop thinking about security as high priority in your day-to-day actions.

It reduces exposure, it does not prove a negative. The block covers the fetch shapes I threw at it and more. A determined agent or a creative prompt can still find a path the hook does not model. Treat this as one layer. For real production stuff, you still need to keep IAM least-privilege underneath it, keep CloudTrail on and scope who can read which secret. The skill lowers the odds of a dumb leak. It does not turn the agent into a trusted process.

The safe path still calls GetSecretValue. It is not read-free. asm-exec resolves by calling Secrets Manager, it just does it in its own process instead of handing the value back to the agent. Your identity still needs secretsmanager:GetSecretValue on the secret, and the read still shows up in CloudTrail. When it goes through the MCP endpoint the event lists invokedBy, sourceIPAddress and userAgent all as aws-mcp.amazonaws.com, so you can tell an agent read from one you ran yourself. The difference is where the plaintext lands, not whether the API gets called.

asm-exec times out on slow connections. I got Failed to resolve on a fresh network even with valid creds. The read timeout is 10 seconds. A retry cleared it.

Try it on something you were going to build anyway

To make sure you get the automatic block you saw in the demo above, install the plugin in Claude Code with this command: /plugin install aws-core@claude-plugins-official, then restart your session so the hook loads.

From the Agent Toolkit for AWS site, you'll see this one shot command that works for most agents:

aws configure agent-toolkit

This is the easy way to install the toolkit but it does require AWS CLI 2.35+. This method will auto-detect your agents, install skills, and configure the MCP Server. One catch I hit: it does not install the secret-safety skill you saw here. That one ships with the aws-core plugin (it lives under plugins/aws-core/skills).

Repo and docs below:

Start read-only. Use the aws:CalledViaAWSMCP condition key to keep the agent to reads through the MCP Server so people can get used to it, then open up write access once you trust the pattern. Loosening later is easy. Walking back a broad grant is not.

If you have any feedback on this or general thoughts on the toolkit, drop a comment below. Feel free to connect/reach out on LinkedIn.

3 Ways to Manage Files in CloudShell

Sean Boult — Thu, 20 Aug 2026 16:48:35 +0000

So you've probably had to open CloudShell from time to time to poke at something quickly. Say I wanted to test pulling down an artifact from a private CodeArtifact registry. To do that, I'd need to tweak my ~/.npmrc so npm points at the right registry.

Which raises the question: what's the best way to edit a file once you're in CloudShell? Let's look at a few options.

Using vim

I use Neovim btw (sorry emacs users), so naturally I'm starting with vim.

vim ~/.npmrc

If you're already comfortable with Vim, this is probably the fastest option. You're already in the terminal, you get all the navigation and search you're used to, and there's nothing else to open.

Make your changes, hit ESC, and :wq (write + quit) to save the file.

Using nano

Ah the old classic. I've been using this for decades to edit files on remote boxes.

nano ~/.npmrc

Unlike Vim, there's not much to learn before you can start typing. Nano even shows you the important shortcuts at the bottom of the screen.

Make your changes, hit CTRL+O to write them to disk, and CTRL+X to exit.

If you just need to make a quick edit and get out, nano is hard to beat.

Using edit

Okay, but what if you don't want to use a terminal editor at all?

CloudShell has an edit command that opens the file directly in its built-in editor.

edit ~/.npmrc

Instead of editing inside the terminal, CloudShell opens the file in a familiar graphical editor where you can point, click, type, and save with CMD+S (CTRL+S on Windows/Linux) or the save icon.

If you're used to editing files in VS Code or another graphical editor, this is probably going to feel the most familiar.

No Vim modes. No terminal keybindings to remember.

Which one should you use?

Honestly, whichever gets you back to solving the actual problem fastest.

Editor	Use it when...
vim	You're comfortable with Vim and want to stay in the terminal
nano	You want a simple terminal editor without learning Vim
edit	You'd rather use a familiar graphical editor

So the next time you find yourself debugging something from CloudShell, you've got a few ways to quickly read or edit a file without leaving your browser.

As always, happy coding 😄!

Follow AWS for more articles like this and follow me for all things tech.

AWS Follow

Articles written by current and past AWS Developer Advocates to help people interested in building on AWS. Opinions are each author's own.

Sean BoultFollow

Developer. Hacker. Creator.

How to Build an AI Agent That Asks Permission First (Nuxt + AI SDK 7)

Erik Hanchett — Tue, 18 Aug 2026 18:38:25 +0000

Introduction

I did something stupid. I built a superhero-themed Nuxt app, connected it to an Anthropic model through Amazon Bedrock, and gave it a tool that deletes files from my computer. In fact, if I wasn't careful, it could have deleted all my files!

The first time I tried it, I didn't use any sort of approval mechanism. And as you expected it just deleted things. Then I looked into how my coding agent works, and I learned about tool approvals.

I learned that AI SDK 7 has a tool approval at the model-call level. It works by pausing for an approval, showing an approval window, and then deleting it. I then put Kiro CLI behind the same interface using Agent Client Protocol (ACP).

Watch the full video on YouTube.

Prerequisites

You need:

Node.js 22 or later. AI SDK 7 requires Node.js 22 and uses ECMAScript modules (ESM).
npm 11 or another package manager that works with Nuxt 4.
AWS credentials available through the standard provider chain.
Access to an Amazon Bedrock model in your AWS Region.
The AWS CLI if you want to list the inference profiles available to your account.
An authenticated Kiro CLI installation for the optional ACP section.

Step 1: Create the Nuxt app

Create the project and install the versions used in the recorded demo:

npx nuxi@latest init nuxt-agent-approval
cd nuxt-agent-approval

npm install \
  nuxt@4.5.2 \
  vue@3.5.41 \
  ai@7.0.66 \
  @ai-sdk/vue@4.0.66 \
  @ai-sdk/amazon-bedrock@5.0.57 \
  @aws-sdk/credential-providers@3.1111.0 \
  @nuxt/ui@4.10.0 \
  zod@4.4.3

npm install -D @iconify-json/lucide@1.2.123

// nuxt.config.ts
export default defineNuxtConfig({
  modules: ['@nuxt/ui'],
  css: ['~/assets/css/main.css'],
  runtimeConfig: {
    awsRegion: process.env.AWS_REGION ?? 'us-west-2',
    bedrockModelId: process.env.NUXT_BEDROCK_MODEL_ID
  }
})

Add the two Nuxt UI imports:

/* app/assets/css/main.css */
@import "tailwindcss";
@import "@nuxt/ui";

You can compare your setup with the complete companion project.

Step 2: Keep the file tool inside a fixture directory

The video uses two fixture files, old-draft.md and keep-me.md. Create them before adding the tools:

mkdir -p fixtures
printf '# Old draft\n' > fixtures/old-draft.md
printf '# Keep me\n' > fixtures/keep-me.md

The agent can list or remove files in that directory, but it should not accept a path such as ../../package.json.

Approval decides whether a tool runs. It does not decide what the tool can reach after it starts.

// server/utils/file-tools.ts
import { lstat, readdir, realpath, rm } from 'node:fs/promises'
import { resolve, relative, isAbsolute } from 'node:path'
import { tool } from 'ai'
import * as z from 'zod'

function resolveInsideFixtures(inputPath: string): string {
  const root = resolve(process.cwd(), 'fixtures')
  const target = resolve(root, inputPath)
  const rel = relative(root, target)

  if (rel === '' || rel.startsWith('..') || isAbsolute(rel)) {
    throw createError({
      statusCode: 400,
      statusMessage: `Path escapes the fixtures directory: ${inputPath}`
    })
  }

  return target
}

export const listFiles = tool({
  description: 'List the files in the project fixtures directory.',
  inputSchema: z.object({}),
  execute: async () => {
    const entries = await readdir(resolve(process.cwd(), 'fixtures'), {
      withFileTypes: true
    })
    return { files: entries.filter(entry => entry.isFile()).map(entry => entry.name) }
  }
})

export const deleteFile = tool({
  description: 'Permanently delete one file from the fixtures directory.',
  inputSchema: z.object({ path: z.string() }),
  execute: async ({ path }) => {
    const target = resolveInsideFixtures(path)
    const info = await lstat(target).catch(() => null)

    if (!info?.isFile() || info.isSymbolicLink()) {
      return { deleted: false, path, reason: 'Not a regular file' }
    }

    const root = await realpath(resolve(process.cwd(), 'fixtures'))
    const canonicalTarget = await realpath(target)
    const rel = relative(root, canonicalTarget)

    if (rel === '' || rel.startsWith('..') || isAbsolute(rel)) {
      throw createError({
        statusCode: 400,
        statusMessage: 'File resolves outside fixtures'
      })
    }

    await rm(canonicalTarget)
    return { deleted: true, path }
  }
})

The second check happens after realpath(). That catches a path that looked local before resolution but points outside the fixture directory through a symbolic link.

Step 3: Connect the route to Amazon Bedrock

I used Amazon Bedrock, but you can use provider with AI-SDK

Create the provider in server/utils/bedrock.ts:

import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock'
import { fromNodeProviderChain } from '@aws-sdk/credential-providers'

export function useBedrock() {
  const { awsRegion } = useRuntimeConfig()

  return createAmazonBedrock({
    region: awsRegion,
    credentialProvider: fromNodeProviderChain()
  })
}

fromNodeProviderChain() uses the AWS credentials already available to your local environment, including AWS IAM Identity Center sessions, named profiles, environment variables, and instance roles. You do not need to put a long-lived access key in the Nuxt project.

Do not copy a model ID from this post. Available IDs vary by account and AWS Region. List the active Amazon Bedrock inference profiles for your account:

aws bedrock list-inference-profiles \
  --region us-west-2 \
  --query 'inferenceProfileSummaries[?status==`ACTIVE`].inferenceProfileId'

Set one of the returned profile IDs before starting Nuxt:

export AWS_REGION=us-west-2
read -r -p "Inference profile ID: " NUXT_BEDROCK_MODEL_ID
export NUXT_BEDROCK_MODEL_ID
npm run dev

Now add an unguarded chat route. Starting without approval makes the failure visible:

// server/api/chat.post.ts
import {
  streamText,
  stepCountIs,
  convertToModelMessages,
  toUIMessageStream,
  createUIMessageStreamResponse
} from 'ai'
import type { AmazonBedrockProvider } from '@ai-sdk/amazon-bedrock'

type BedrockModelId = Parameters<AmazonBedrockProvider>[0]

export default defineEventHandler(async event => {
  const { messages } = await readBody(event)
  const { bedrockModelId } = useRuntimeConfig()
  const bedrock = useBedrock()

  const result = streamText({
    model: bedrock(bedrockModelId as BedrockModelId),
    instructions:
      'Manage files in the fixture project. List files before deleting. Never guess a filename.',
    messages: await convertToModelMessages(messages),
    tools: { listFiles, deleteFile },
    stopWhen: stepCountIs(5)
  })

  const stream = toUIMessageStream({ stream: result.stream })
  return createUIMessageStreamResponse({ stream })
})

stopWhen is important. AI SDK 7 stops after one step by default. The model can call deleteFile, receive the result, and then stop before it tells the user what happened. Five steps leave room to list, delete, and summarize while keeping the loop bounded.

At this point, delete old-draft.md removes the file as soon as the model selects the tool. That is what happened in the first minute of the video.

Step 4: Add approval to the delete tool

Add one option to the streamText() call (toolApproval):

const result = streamText({
  model: bedrock(bedrockModelId as BedrockModelId),
  instructions:
    'Manage files in the fixture project. List files before deleting. Never guess a filename.',
  messages: await convertToModelMessages(messages),
  tools: { listFiles, deleteFile },
  stopWhen: stepCountIs(5),
  toolApproval: {
    deleteFile: 'user-approval'
  }
})

The option lives on streamText(), not inside the tool definition. The same deleteFile tool might run unattended in a maintenance job and require a person in a customer-facing chat.

AI SDK 7 supports more than a yes-or-no policy. A policy function can approve a call, deny it without asking, or send it to the user. This example uses the direct user-approval status because every delete should stop.

Step 5: Render the approval request in Nuxt

Nuxt UI's chat documentation follows the same AI SDK message-part model. The recorded app uses useChat() from @ai-sdk/vue and the isToolApprovalPending() helper from Nuxt UI:

<script setup lang="ts">
import {
  DefaultChatTransport,
  getToolName,
  isTextUIPart,
  isToolUIPart,
  lastAssistantMessageIsCompleteWithApprovalResponses
} from 'ai'
import { useChat } from '@ai-sdk/vue'
import { isToolApprovalPending } from '@nuxt/ui/utils/ai'

const input = ref('')

const {
  messages,
  status,
  sendMessage,
  addToolApprovalResponse
} = useChat({
  transport: new DefaultChatTransport({ api: '/api/chat' }),
  sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses
})

function onSubmit() {
  if (!input.value.trim()) return
  sendMessage({ text: input.value })
  input.value = ''
}
</script>

sendAutomaticallyWhen resumes the interrupted turn after the user answers. Without it, the approval state changes in the browser but the agent does not continue on its own.

Render tool parts and attach the two decisions while approval is pending:

<template>
  <div v-for="message in messages" :key="message.id">
    <template
      v-for="(part, index) in message.parts"
      :key="`${message.id}-${part.type}-${index}`"
    >
      <div v-if="isToolUIPart(part)" class="tool-card">
        <strong>{{ getToolName(part) }}</strong>

        <pre v-if="part.input">{{ part.input }}</pre>

        <div v-if="isToolApprovalPending(part)">
          <button
            type="button"
            @click="addToolApprovalResponse({
              id: part.approval!.id,
              approved: true
            })"
          >
            Do it
          </button>

          <button
            type="button"
            @click="addToolApprovalResponse({
              id: part.approval!.id,
              approved: false,
              reason: 'The user declined this file deletion.'
            })"
          >
            Nope
          </button>
        </div>
      </div>

      <p v-else-if="isTextUIPart(part)">{{ part.text }}</p>
    </template>
  </div>

  <form @submit.prevent="onSubmit">
    <input v-model="input" placeholder="delete old-draft.md">
    <button :disabled="status !== 'ready'">Send</button>
  </form>
</template>

FYI, An approval button is not useful when the person cannot see which path the agent wants to remove.

The denial reason is also useful. approved: false tells the model it cannot run the tool. The reason gives it enough context to explain what happened rather than trying the same call again.

Run the prompt twice. Deny it once and confirm that old-draft.md remains. Reset the fixtures, repeat the prompt, approve it, and confirm that the file disappears. The model received the same request both times.

Step 6: Put Kiro CLI behind the same Nuxt UI

The video switches the backend for the final demo. Instead of sending the prompt directly to an Anthropic model through Amazon Bedrock, the Nuxt app talks to Kiro CLI over Agent Client Protocol.

ACP gives the app a common way to start an agent session, send a prompt, receive tool events, and answer permission requests. Kiro runs as a separate process and keeps its existing agent tools and Model Context Protocol (MCP) integrations. The Nuxt app remains responsible for the interface and the host tools it exposes.

Install the AI SDK harness packages:

npm install \
  @ai-sdk/harness@1.0.73 \
  @ai-sdk/harness-acp@1.0.11

The Kiro route creates a HarnessAgent instead of calling streamText() directly. The createKiroHarness() ACP preset and createUnsafeLocalSandbox() development adapter come from the companion project, so treat this as the route configuration rather than a standalone file:

import { execFileSync } from 'node:child_process'
import { HarnessAgent } from '@ai-sdk/harness/agent'

const kiroExecutable = execFileSync('which', ['kiro-cli'], {
  encoding: 'utf-8'
}).trim()

const agent = new HarnessAgent({
  harness: createKiroHarness({ port: 4100 }),
  sandbox: createUnsafeLocalSandbox({
    ports: [4100],
    hostBins: [
      { harnessId: 'kiro', name: 'kiro-cli', target: kiroExecutable }
    ]
  }),
  permissionMode: 'allow-reads',
  instructions:
    'Manage files in the fixture project. List files before deleting. Never guess a filename.',
  tools: { listFiles, deleteFile },
  toolApproval: {
    deleteFile: 'user-approval'
  }
})

The complete Kiro route and ACP preset include session creation and approval continuation. When a response comes back from the browser, the route gathers pending approval responses and calls continueStream() against the Kiro session.

My recorded run asked more than once before deleting keep-me.md. Kiro confirmed the target, its permission flow asked to run the tool, and the host deleteFile policy asked for the final approval. It was a little repetitive, but it exposed an important boundary. Kiro's built-in permissions and AI SDK's host-tool approval are separate systems.

The local sandbox in this sample is a development adapter. It limits file API paths to a temporary root, but processes still run as the current operating-system user. Replace it with an isolated sandbox provider before exposing a coding agent to untrusted prompts.

Step 7: Keep approval in its lane

Tool approval is a product DX improvement, it isn't security.

If a person approves the wrong path, the tool still removes the wrong path. If the tool can reach the rest of the filesystem, approval does not narrow that access. Keep the path checks from Step 2, apply authorization inside the tool, validate inputs on the server, and use an isolated runtime for agents that can run commands.

The sample app also keeps its recording controls in development mode. The browser can turn approval off for the first demo, but the built app ignores that flag and requires approval. A client-controlled switch that disables confirmation should not ship.

Cleanup

Stop the Nuxt development server. If you deleted either fixture during the demo, recreate both files before your next run:

printf '# Old draft\n' > fixtures/old-draft.md
printf '# Keep me\n' > fixtures/keep-me.md

This tutorial does not provision AWS resources. Amazon Bedrock requests can still incur charges, so stop sending test prompts when you finish.

If you delete the local project directory, its files and any local session state are removed. Copy anything you want to keep before deleting it. AWS credentials loaded through the provider chain remain in their original profile or identity-center cache; this app does not write them into the project.

Finale

I learned a lot by adding this tool approval process. And even though I normally use Strands Agents, the AI SDK 7 worked really well.

Make sure to leave a comment below if you got this far!

References

Why Can't We Just Use Claude for Everything? Agent Creation vs. Agent Engineering

Matheus Guimaraes — Tue, 18 Aug 2026 16:36:11 +0000

Recently, I was talking to a friend who, much like the rest of the world, has started using AI extensively at work to create agents even though she's not a software developer or from a technical background.

She has been experimenting with Claude a lot, building useful workflows for herself and her team, and getting increasingly excited about what these systems can do.

At some point, our conversation moved onto the work I've been doing around agentic systems: agent architectures, context engineering, memory, orchestration, sub-agents and frameworks such as Strands Agents.

And her reaction was:

Why do you need all of that? I'm already creating agents.

"Great question", I thought to myself.

And what made it particularly interesting was that I'd heard almost the same question from the opposite direction.

After giving a talk about agentic architecture and building harnesses with Strands Agents SDK, a junior developer came up to me and asked:

What's the point of all of this? Why can't we just use Claude for everything?

Two people approaching the subject from completely different directions had arrived at the same question.

And I think that question exposes something important about where we are with agents today. We are using the same word "agent" to describe two very different activities with very different use cases.

The way I see it, there is a big difference between creating agents and engineering agents.

Agent Creation vs. Agent Engineering

Applications such as Claude, ChatGPT, Amazon Quick and others increasingly allow people to create surprisingly sophisticated agentic workflows without building the underlying infrastructure themselves.

There is still a learning curve, though. You need to provide instructions and give the system knowledge and context in a way that produces results that are within expectation (and, hopefully, without hallucinations!). You may also need to learn about tools, set up integrations, schedule work or even create subagents so multiple pieces of functionality can work together.

From the user's perspective, they have created an agent. And indeed they have! But when a software engineer looks at the same system... they see something different.

They see an application providing a highly abstracted, opinionated agent runtime operating as a black box. You have little control over how execution happens, how context is assembled and maintained, how the system scales, or even where memory lives, among other architectural concerns.

As an application user, you are creating an agent inside somebody else's agentic architecture.

And that's not a problem. In fact, this could be exactly what you need!

Developers and software architects make trade-offs between abstraction and control all the time. We don't choose the lowest-level technology simply because it gives us the most control; we use abstractions because they remove decisions we don't want to make and complexity we don't need to own.

The same principle applies here. If Claude gives you everything you need to create an agent, why wouldn't you use it?

As usual with system design, the important thing is understanding the trade-offs. The abstraction works because somebody else has made a set of architectural decisions for you. If those decisions suit your requirements, then awesome. If they don't, you may need to take ownership of some of them yourself.

That's where I find the distinction between Agent Creation and Agent Engineering useful.

For me, three things make that distinction particularly clear.

Is the agent a feature or a component?

This is perhaps my favourite way of thinking about the distinction.

Let's say I create an agent in Claude to research holidays for me. It knows my preferences, researches destinations, compares options and perhaps uses tools to build an itinerary. It could be incredibly sophisticated... but Claude is still the application. I go to Claude, interact through Claude's interface and rely on Claude's infrastructure to make everything work.

My agent is a feature of an application I'm using.

Now imagine I want to offer that same experience through my own travel website. A customer comes to my application and asks for a holiday; somewhere behind the scenes an agent researches flights and hotels, applies their preferences, interacts with booking systems and returns the result.

The capability sounds almost identical, but architecturally something fundamental has changed. The agent is now a component of a system I'm building.

Suddenly I need to decide where it runs, how my application communicates with it, how it scales, how it authenticates against other systems, what happens when it fails, how it fits into traditional application flows, and how I observe what it's doing. The AI is no longer the destination; it's now part of the system I'm responsible for.

That's very clearly Agent Engineering.

The point is really about ownership. If an agent is a feature of Claude or another application, you're limited, quite deliberately, to what that application allows you to do. If your requirements fit inside those boundaries, great. When they don't... you need another abstraction.

Do you need control over the economics?

Another reason you might choose to engineer the agent is cost.

There is a lot of conversation right now about just how many tokens agentic systems can consume, particularly as we build long-running workflows with large context windows, tool calls and multiple agents.

Moving from Agent Creation to Agent Engineering doesn't automatically make any of this cheaper, by the way. Engineering has its own cost, and it would be rather ironic to spend £100,000 in developer time to save £500 in tokens.

What engineering gives you is control.

If you own the harness around your agents, architectural decisions become available that simply aren't exposed when you're operating inside somebody else's application.

You can decide that one task doesn't need the full conversation history, cache an expensive result, retrieve memory only when it's relevant, summarize context between steps, or route simpler tasks to cheaper models.

You can start treating context and tokens as resources that you deliberately engineer.

There's a much larger conversation here around context engineering, but that's a story for another post. For now, the important point is that the abstraction determines which levers are available to you.

If you don't need those levers, there's little value in taking responsibility for them. If the economics of your system mean you do, Agent Engineering gives you that control.

Do you need to own where your data goes?

Then there is a question that tends to become important very quickly in enterprise environments: where does everything actually live?

When I create an agent inside an application, I'm accepting that application's architecture and its mechanisms for handling context, state, memory, integrations and data. Again, that may be completely acceptable.

But what if particular data cannot leave a certain environment? What if I need to know exactly where long-term memory is persisted, control which model receives which information, maintain my own audit trail, or guarantee that certain data never crosses a particular boundary?

Now the black box really matters.

This is partly a compliance question, partly a data ownership question and, yes, partly a vendor lock-in question. The more behaviour, knowledge and state I build around one application's abstractions, the more dependent I become on that platform's capabilities and constraints.

But vendor lock-in isn't, by itself, a reason to engineer everything from scratch. We accept varying degrees of lock-in throughout modern software architecture because managed abstractions can provide enormous value.

The question is simply whether that trade-off is acceptable for the requirements in front of you.

So... Can't We Just Use Claude For Everything?

A useful spreadsheet can eventually become an application, but that doesn't mean the spreadsheet was a mistake. It may have been exactly the right abstraction for the problem at the time, and it may even have helped us understand the problem well enough to know what was worth engineering later.

I think Agent Creation and Agent Engineering should be seen in much the same way: not as levels on a maturity ladder, and certainly not as a journey where every agent should eventually become an engineered one, but as different architectural choices about how much of the underlying system we need to own. We might happily create an agent inside Claude for one task while engineering another from the ground up, perhaps even within the same system.

Sometimes an agent created inside Claude, ChatGPT, Amazon Quick or another application will remain there because it does everything we need. Sometimes our requirements will mean taking ownership of the architecture ourselves and we'll find ourselves reaching for Strands Agents SDK or another agent harness.

And perhaps the next time someone asks me, "Why can't we just use Claude for everything?", my answer will simply be...

You can! For as long as its decisions are compatible with yours.**

Understanding Async Programming

Matheus Guimaraes — Mon, 17 Aug 2026 16:01:25 +0000

Async programming, short for asynchronous programming, is one of those concepts that might sound intimidating at first but becomes surprisingly intuitive once you understand the problem it is trying to solve.

And that problem is mostly waiting.

Modern applications spend a lot of time waiting for things to happen. They wait for APIs to respond, databases to return queries, files to be read or written, and data to travel across a network. Async programming gives us a way to avoid blocking while that happens, allowing other useful work to make progress instead.

In this post, I’ll cover the basic concepts behind async programming rather than focusing on a particular language. The exact implementation differs between languages and runtimes, but the underlying ideas are largely the same. So if you’re new to async, or have used async and await without ever being completely sure what they’re doing, this is a good place to start.

What is Async Programming?

At its core, async programming allows your program to start an operation that may take some time and avoid blocking while it waits for that operation to complete.

This contrasts with synchronous programming, where execution waits for an operation to finish before continuing past that point.

Imagine you’re making an API call, a very common task in modern applications. Your application sends a request across the network and then has to wait for a response. Even if that response takes only a fraction of a second, that's a long time in computer terms.

The important thing is that your application isn't necessarily doing useful computation during all that time. A lot of it is simply waiting for something external to happen.

This is where async programming shines. Rather than blocking an execution resource while waiting for the API call to complete, asynchronous execution can allow other work to make progress and then continue the original operation when its result becomes available.

Async Doesn't Necessarily Mean Faster

One of the most common misconceptions about async programming is that it's about improving performance. A lot of people think that using async makes their code run faster. However...

Async does not necessarily make the operation itself any faster.

Your API call might take exactly the same amount of time to complete whether you call it synchronously or asynchronously. What async changes is what your application can do while it waits. Instead of being blocked by that wait, it can potentially make progress on other work.

When is Async Programming Useful?

There are three main benefits to using async programming:

Improved throughput: Because your application doesn't have to block on one operation before making progress on other work, it can often handle more work over the same period of time. This is particularly valuable in applications such as web servers that may be handling many requests concurrently.
Better resource utilization: Waiting for I/O doesn't necessarily require a thread to sit blocked until the operation completes. With asynchronous I/O, execution resources can potentially be used for other work while the application waits for the database, network, filesystem or another external resource.
Better responsiveness: Async programming can also help applications remain responsive while longer-running operations are taking place. In a user interface, for example, you generally don't want the entire application to freeze while it waits for a network request or file operation to complete.

The common thread between all three is the same: making better use of time that would otherwise be spent waiting.

Sync vs. Async: When to Use Each

Async programming sounds great, but that doesn't mean everything should be asynchronous.

The distinction becomes clearer when we think about I/O-bound and CPU-bound work.

I/O-bound operations spend a significant amount of their time waiting for something outside the CPU: network requests, database calls, file operations and similar tasks. These are often excellent candidates for async programming because there is useful time to reclaim while the application waits.

CPU-bound operations are different. If you're performing a heavy calculation, for example, the CPU is actively doing the work rather than waiting for something else to happen. Making that code asynchronous doesn't suddenly make the computation faster.

CPU-bound work may instead benefit from parallelism, where computation is divided so that multiple pieces of work can execute at the same time, potentially across multiple CPU cores.

Async programming can also introduce additional complexity and some runtime overhead, so there's little benefit in making code asynchronous simply for the sake of it. It is most useful when your application would otherwise spend meaningful time blocked waiting for I/O.

Async Isn't the Same as Parallelism

The distinction between async and parallelism is an important one. Async is primarily about allowing other work to make progress while something is waiting. Parallelism is about performing multiple pieces of work at the same time.

And async doesn't necessarily mean multithreaded either. Different languages and runtimes implement asynchronous execution in different ways, so you shouldn't assume that an async operation means another thread is somewhere doing the work.

This is why async can provide concurrency without necessarily providing parallelism: multiple operations can make progress over the same period of time without their code actually executing at the exact same moment.

How Does Async Work in Code?

The exact mechanics vary between programming languages, but many modern languages provide the familiar async and await keywords to make asynchronous code easier to write and reason about.

At a conceptual level, async identifies code that participates in asynchronous execution, while await marks a point where that code depends on the result of an asynchronous operation.

If that result is already available, execution may simply continue. If it isn't, the function can yield rather than blocking there, allowing other work to make progress. Once the awaited operation completes, execution can continue from that point.

You can think of await as saying: "I need the result of this operation before I can continue from here, but there's no reason to block everything else while I'm waiting for it."

The precise details of what happens underneath, whether that involves an event loop, tasks, promises, futures, state machines or something else, depend on the language and runtime. But you don't need to understand all of those implementation details to understand the basic async mental model.

Ready to Dive Deeper?

If async still feels a little abstract, I also explain the same ideas visually in the companion video below. Sometimes seeing the synchronous and asynchronous flows side by side is what finally makes the concept click.

AI Agent Memory: Add Semantic Search Without a Vector Database

Elizabeth Fuentes L — Thu, 13 Aug 2026 01:31:01 +0000

📦 Clone and ⭐ stop-ai-agents-losing-memory-sample-for-aws

The agent's memory holds the answer. The user asks the question. And retrieval returns nothing.

stored:   dietary_notes: "Vegetarian; severe shellfish allergy, strictly no
          crustaceans or mollusks."

asked:    "What should I avoid eating when I go out for dinner on this trip?"

keyword scan: 4 hits, answer found: False

That's a real run, not a thought experiment. The question names no key and shares no words with the stored note, so the key-value memory from the previous post never finds it. The answer was in the store the whole time.

This is the dividing line for semantic search: do you know the key, or only the intent? When questions stop matching keys, you retrieve by meaning: embed each memory once, embed the question, return the nearest neighbors by cosine similarity. This post measures two things (whether semantic search finds what keyword search misses, and which vector store fits your deployment) using the same embeddings and the same memories in the companion repo.

(Post 2 of a series; the intro maps all the memory types. The code uses Strands Agents, an open source SDK; the pattern carries over to any agent framework.)

Why Strands Agents for this demo?

Strands makes comparing vector backends straightforward. The demo tests three vector stores (FAISS, S3 Vectors, DynamoDB Vector Search) against the same memories and same embeddings, so the comparison isolates storage and retrieval performance, not the agent framework.

Adding semantic search to an agent is just a tool:

from strands import Agent, tool

@tool
def recall_memory(query: str) -> str:
    """Search memory by meaning, not keywords."""
    # Embed the query, find nearest neighbors
    results = vector_store.search(query, top_k=3)
    return "\n".join(results)

agent = Agent(
    model=model,
    tools=[search_flights, recall_memory],
)

The recall_memory tool wraps the vector store. Swap FAISS for S3 Vectors or DynamoDB, and the agent code stays the same.

The pattern shown here (semantic recall as a tool) works in any agent framework. Strands just makes it simple to plug different backends and measure them.

Why does key-value memory miss the question?

Because a key-value read is a lookup someone designed in advance, and this question maps to no key. The demo stores 10 memories about a traveler (profile facts, notes, episodes) and asks the dinner question against three stores. The key-value store has exactly two moves, and both fail honestly:

Keyword scan: match question words against keys and values. It returns 4 hits, none of them the allergy note, because "avoid eating at dinner" shares no words with dietary_notes or "shellfish". Answer found: False.
Dump-all fallback: give the model the entire memory and let it read. It works, at a price that grows with every memory you add. For these 10 memories that's 647 characters per question; for hundreds of notes it's thousands of tokens, every single question, forever.

This isn't a bug in key-value memory. Profile lookups ("what's my preferred cabin?") stay exact, instant, and free of embedding costs, which is why the previous post built them that way. The limit only appears when the question is semantic. That's the signal to add a second way in, not to replace the first.

How does semantic search find it?

By comparing meanings instead of words. Every memory is embedded once at write time into a vector (here: Amazon Titan Text Embeddings V2, 1,024 dimensions). At query time, the question is embedded and the store returns the nearest neighbors by cosine similarity:

top hit: "Vegetarian; severe shellfish allergy, strictly no crustaceans
          or mollusks."  (score 0.231)
answer found: True

No shared words between question and note. They're close in meaning, and meaning is what got indexed. Both backends below return this same top hit, because they use the same embeddings; what differs is everything around the query.

Two implementations: FAISS to prototype, S3 Vectors to persist

Both are embedding vector stores. They use the same model (Titan V2), the same algorithm (cosine similarity), and they return the same top hit with the same score. The accuracy is identical, this is not a quality trade-off.

Store	Finds the answer	Similarity score
Key-value (keyword scan)	No	keyword miss
FAISS (Facebook AI Similarity Search, Meta's in-process vector index	Yes	0.231
Amazon S3 Vectors (managed cloud)	Yes	0.231

These are two implementations of the same idea for two different moments. FAISS is an in-process library: zero infrastructure, a pip install, running local to the process. It is how you prototype semantic search on your machine (in this demo the index is rebuilt from scratch each run; FAISS can persist to disk with faiss.write_index, but that is still a file you manage). Amazon S3 Vectors is the managed step: the index lives in a cloud bucket, reachable from any process with AWS credentials, surviving restarts with no cluster to run or scale. You reach for it when the memory has to outlive the process.

The demo uses the same AWS credentials for both: Titan embeddings via Bedrock and S3 Vectors via boto3; the same aws configure setup powers both, which is why this requires no extra setup inside a Strands Agents workflow. The demo self-provisions the bucket and index on first run: create_vector_bucket → create_index (1,024 dims, cosine) → put_vectors / query_vectors.

The cost both implementations share: embedding the question costs ~510 ms with Titan V2 in this demo. The vector query itself is small next to that, so the embedding call, not the store, is what to budget for in any latency-sensitive path.

So, do you need a vector database?

It depends on the query pattern. AWS positions S3 Vectors as "ideal for workloads where queries are less frequent", which describes agent memory exactly: an agent queries a user's memories a handful of times per conversation, not thousands of times per second.

	FAISS	Amazon S3 Vectors	Dedicated vector database
Type	In-process library	AWS vector storage	Full database engine
Examples	n/a	n/a	OpenSearch, Qdrant, Weaviate, Milvus, pgvector, Chroma
Semantic accuracy	✅ same	✅ same	✅ same
Infrastructure	None (pip install)	None (fully managed)	Self-hosted or managed
Max vectors	Process memory	Up to 2 billion per index	Depends on deployment
Persists across restarts	No (in-process)	Yes	Yes
Hybrid search	❌	❌	✅ most support it
Best for	Prototype / local agent	Cloud agent, infrequent queries	High QPS, advanced filtering, production search

The decision:

You need	Pick	Why
Facts under known keys (profile, preferences)	Key-value (post 1)	Exact and instant; don't pay ~510 ms of embedding for a lookup
Semantic search, local / prototype	FAISS	Zero infrastructure, pip install, in-process
Semantic search, cloud / infrequent queries	S3 Vectors	Purpose-built AWS vector storage, subsecond latency, up to 2 billion vectors, no infrastructure to manage
High QPS, hybrid search, or advanced filtering	Dedicated vector DB	OpenSearch, Qdrant, Weaviate, Milvus, pgvector, Chroma
Multi-hop questions over relationships	Graph (next post)	Semantic search finds pieces; it can't follow edges between them

What this demo does not cover: FAISS and S3 Vectors are storage backends. They store vectors and retrieve by similarity. Building what to remember (extracting specific facts from conversations, deduplication, structured memory across sessions) is handled by managed memory services like Amazon Bedrock AgentCore Memory. That technique is the topic of a future post in this series.

How does the agent choose between key lookup and semantic search?

From the tool docstrings, on its own. The demo's last test attaches both recall tools to one Strands agent:

@tool
def recall_by_key(key: str) -> str:
    """Recall a memory when the question maps to a known identifier.
    Use when the user asks about a stored field: "my preferred cabin",
    "my home airport"..."""

@tool
def recall_semantic(question: str, top_k: int = 3) -> str:
    """Recall memories by meaning when no key is obvious.
    Use for open questions: "what should I avoid eating on this trip?"..."""

Asked the dinner question, the agent calls recall_semantic; asked "what cabin do I prefer?", it calls recall_by_key. No routing logic, no prompt engineering. The when to use this sentence at the top of each docstring is what the model reads to decide. Write that sentence carelessly and the agent pays embedding latency for profile lookups.

How do you ask an AI coding assistant to build this?

The quality of the semantic search implementation your assistant builds depends on the decisions you name in the prompt. Unnamed, it will default to embedding everything and querying one big index. These five instructions encode what this post measured:

"Add semantic search only for questions that don't map to keys; keep profile facts in key-value state." Otherwise the assistant defaults to embedding every query, including exact lookups that already have a known key.
"Embed each memory once, at write time; only the question gets embedded at query time." Assistants love re-embedding the whole store per query.
"Use one embedding function for storage and queries, and state the model and dimensions." Mixed embedders produce silent garbage similarity scores.
"Give me two recall tools with 'when to use' docstrings: by key, and by meaning." The agent routes per question from those sentences; no router code.
"Make persistence explicit: in-process index for a prototype, managed vector storage for anything that must survive a restart, and prove it with a fresh-client test that still sees every vector."

The companion repo implements and measures all five. Run it to see each decision play out.

How do you run the demo?

git clone https://github.com/elizabethfuentes12/stop-ai-agents-losing-memory-sample-for-aws
cd stop-ai-agents-losing-memory-sample-for-aws/02-vector-memory-demo
uv venv && uv pip install -r requirements.txt
uv run python test_vector_memory.py

Needs AWS credentials (aws configure) for Titan embeddings and S3 Vectors. The demo creates the vector bucket and index automatically if they don't exist. OPENAI_API_KEY is only needed for the agent conversation in the notebook (or swap one line for Amazon Bedrock); the retrieval measurements run without any LLM.

FAQ

Is a vector database the same as AI agent memory?
No. A vector database is one possible backend for one memory type (retrieval by meaning). Agent memory is the whole system: key-value state, vector or graph storage, selection rules, and hygiene. Many production agents need vector retrieval without a vector database.

Can I use a vector database as agent memory?
Yes, for memories you'll query by meaning. But route keyed facts (preferences, settings) to key-value storage first: a direct lookup costs nothing, while every vector query pays the question-embedding call (~510 ms with Titan V2) before the index is even touched.

When do I need something beyond S3 Vectors?
When your query pattern changes. Dedicated vector databases such as OpenSearch, Qdrant, Weaviate, Milvus, pgvector, and Chroma are built for high QPS, hybrid keyword+vector search, aggregations, and advanced filtering. S3 Vectors is purpose-built for infrequent queries: it handles up to 2 billion vectors per index with subsecond latency, which covers agent memory workloads well past prototype scale.

Is the vector store my latency bottleneck?
No. In this demo the vector query is small next to embedding the question (~510 ms with Titan V2), which both implementations pay. Whether you prototype with FAISS in-process or persist to S3 Vectors, budget for the embedding call, not the index lookup.

Why did my semantic search return the wrong memories?
The most common causes: the store and the queries use different embedding models or dimensions, memories were embedded with stale text, or keyed facts polluted the index. Keep one embedder for everything, embed at write time, and keep profile facts out of the vector store.

Resources

Companion repo: demo 02 with the measured tests and notebook
Amazon S3 Vectors User Guide and limitations
Amazon Titan Text Embeddings V2
FAISS, Meta's similarity search library
Zep: A Temporal Knowledge Graph Architecture for Agent Memory, Rasmussen et al., 2025
From RAG to Memory: Non-Parametric Continual Learning for LLMs (HippoRAG 2), 2025

Gracias!

🇻🇪 Dev.to Linkedin GitHub Twitter Instagram Youtube

Elizabeth Fuentes LFollow

I help developers build production-ready AI applications through hands-on tutorials and open-source projects.

DEV Community: AWS

AI Agent Memory: What to Store and What to Throw Away

What counts as memory extraction?

Strands does memory for you, natively

What each native piece does

The store: your data, your backend

The three mechanisms, side by side

The measured results

A note on flush(): when is a memory saved?

What B's four typed prompts look like

Which mechanism should you pick?

Practical notes for the managed path

Try it

Research referenced

How a Strands agent took Claude Opus 5 from 30% to 99.95% on ARC-AGI-3

The public vs private game set for ARC-AGI-3

Inside the Strands harness

Why these tools?

Relative Human Action Efficiency

What this means

Graph Memory: When Vector Search Fails

Why Strands Agents for this demo?

What is a multi-hop question?

How does graph memory answer it?

What do the measured results show?

When is a graph the wrong choice?

How do you run the demo?

FAQ

What is graph memory for AI agents?

Knowledge graph vs vector memory: which does an agent need?

Is this the same as GraphRAG?

Do I need an LLM to build the graph?

How do you benchmark agent knowledge-graph memory?

Resources

Elizabeth Fuentes LFollow

Build a Full-Stack Music Station with OpenRouter, Amazon Bedrock, and Nuxt

Prerequisites

Steps

1. Run the station without credentials

2. Put provider calls behind Nuxt server routes

3. Generate a scene with Amazon Bedrock

4. Add looping music with ElevenLabs

5. Deploy the private media library

6. Turn one illustration into a seamless video loop

7. Report real progress during a long request

8. Validate the build

Cleanup

Finale

Observability for AI Agents with OpenTelemetry

What agent are we observing?

Layer 1 / What metrics do you get with zero configuration in Strands?

Layer 2 - How do you trace an AI agent with OpenTelemetry?

Layer 3 — How do you add business context to agent traces?

Production — where does agent observability live when you deploy?

Frequently asked questions

Wrap-up: three layers, one standard

Try it yourself

Elizabeth Fuentes LFollow

Amazon DynamoDB Vector Search. No Separate Vector Store

What is Amazon DynamoDB Vector Search?

How does the embedding comparison look?

How do you add a vector index to a DynamoDB table?

How do you write and query vectors?

Does the index survive a restart?

How do you run Test 4?

When do you pick DynamoDB over S3 Vectors?

FAQ

Resources

Elizabeth Fuentes LFollow

I Watched My Coding Agent Refuse to Leak a Secret

What the toolkit is, in one screen

The demo

The automatic block

Resolve without seeing

How the pieces fit

Where this bites

Try it on something you were going to build anyway

3 Ways to Manage Files in CloudShell

AWS Follow

Sean BoultFollow

A note on `flush()`: when is a memory saved?