Residents of a crisis-affected region in southern Israel describe what they need in their own words, in one of five languages. The chat asks only what it must (where they live, how old they are, what kind of help), then returns up to three real services from a catalog of 354, filtered by geography and age in code rather than by the model.
After a regional crisis, dozens of organizations opened therapy, rehabilitation, legal, financial and community programs for local residents. Each one has its own rules. Some serve one local authority only, some the whole region, some the whole country. Many have age limits. A resident looking for physiotherapy for herself, or emotional support for a teenage son, had to know which organization to call and whether she was even eligible.
The client wanted one place to ask. It had to answer in the resident's language, never invent a service, and stay correct as organizations kept adding and changing programs.
The diagram is wider than the screen: swipe sideways to see all of it →
The CMS is the only source of truth. Every service change writes an outbox row. The sync step rebuilds that service's document and, after a full reindex or CSV import, writes one catalog snapshot that the agent reloads on a 5-minute TTL. The old engine stays behind a configuration switch, so the team could compare the two paths and fall back without a deploy.
The managed agent decided by itself how often to search the knowledge base, usually 3 or more times per turn. The new runtime runs a fixed sequence: triage, filter, rank. Nothing is retrieved until town, age and need are known. Trade-off: a Python service next to a .NET backend, and the orchestration is ours to maintain.
Which local authority a settlement belongs to, whether a service is local, regional or national, and its age range are all checked by a deterministic filter. The model only ranks by professional domain. Trade-off: the settlement-to-authority table has to be maintained by hand.
With 354 short, structured records, the filter cuts the list to a median of ≈70 candidates before the model sees anything. A JSON snapshot in S3 removed chunking, ingestion jobs and citation-token problems. Trade-off: this works while the catalog fits in memory; a much larger catalog would bring retrieval back.
The rank call can only pick from the candidate list it was given. Any ID outside that list is dropped before the backend loads service cards. If nothing valid is left, the user gets an honest “no match in your area” and a suggestion.
The emergency button skips the agent and returns a configured list of emergency services in the user's language. The most urgent answer never waits on a model or depends on one.
Most of the hard problems came from running the matching on real catalog data and real questions. A selection:
The first engine was a Lambda that sent the full compressed catalog with every request: about 245K characters of service data, cached but resent on each turn and growing with every new service.
Kept it as a working baseline and moved on to a retrieval design built on a managed knowledge base.
Default chunking separated each service's ID line from its text. The model returned the knowledge base's citation tokens instead of real IDs, the backend couldn't parse them, and valid matches disappeared.
Recreated the data source with no chunking, enriched each document with treatment type, sector, scope and age, and added a prompt rule for numeric IDs.
The agent ran at temperature 1. Answers drifted out of the JSON contract, repeated services already shown, and sometimes showed the user raw {"answer": …}.
Temperature 0, a hardened parser on the backend, and the list of already-shown service IDs sent on every turn as the source of truth.
Residents of one regional council were offered services that serve only a neighbouring council. The model was resolving geography from a lookup table written into the prompt.
Moved geography into a deterministic filter: settlement → authority, then service scope. The model never sees out-of-area services.
Traces showed several autonomous knowledge-base iterations plus pre- and post-processing on every turn, with throttling and retries on top.
Replaced the managed agent with a controlled AgentCore Runtime agent: one call to triage, one to rank, a cached system prompt, and count or name lookups answered from the catalog with no retrieval.
When no candidates were left, the agent asked for the town again, and the conversation looped.
Two explicit endings: “your town is outside the area we cover” or “I've shown everything relevant in your area”, each with a next step.
The ranker sees at most 60 candidates. Ordered by ID, national programs could fill the window and push out a local service that fit better.
Sort local first, then by keyword match on the need, and only then cap. Added a “never pad the list” rule: fewer results, or none, rather than an off-domain one.
The runtime shipped with a bundled CSV, so every catalog update meant redeploying the agent.
The backend writes a catalog snapshot to S3 on reindex and CSV import. The agent reloads it every 5 minutes and keeps the bundled CSV only as a cold fallback.
A simplified replay of one request, “physiotherapy for a 30-year-old woman in a town in the region”. The steps are illustrative. The counts come from the code and the 354-service catalog. No client data is shown.
| Area | Before | After | Change | Source |
|---|---|---|---|---|
| Context per ranking call | Whole catalog, ≈245K chars | ≤60 filtered candidates, ≈17K chars | −93% | derived |
| Model calls per search turn | 3+ autonomous iterations | 2 (triage + rank), 1 to ask | −33% or more | derived |
| Typical answer time | Minutes | ≈10 s | ≈10×+ faster | estimate |
| Invalid service IDs shown | Citation tokens, matches dropped | Only IDs from the candidate list | 0 by design | derived |
| Catalog change to live chat | Agent redeploy | Snapshot reload, TTL 300 s | ≤5 min | measured |
| Out-of-area results | Geography resolved by the model | Deterministic authority + scope filter | 0 in validation | derived |
| Catalog onboarding | Service-by-service entry | One CSV import (30 fields) + auto-reindex | days → minutes | estimate |
| Emergency answer | Through the model | Configured list, no model call | 0 model calls | derived |
Free scoping call · 30 minutes · No commitment
Write a few lines: the process or system, what hurts today (time, errors, cost or an AI pilot that stalled) and who uses it. After a short call you get a written plan and a fixed quote within ≈3 days.
Opens a pre-filled email · reply within one business day
The model reads. The code counts.
All projects