Volunteer organizers at nonprofits ask in Teams or Microsoft 365 Copilot Chat and get a short, personalised dossier for each volunteer on a shift: what they have done, what they are signed up for and how to thank or re-engage them. Every fact comes from that organization’s own volunteer records, and organizations never see each other’s data.
“Estimate” numbers are engineering estimates to confirm with pilot data. All others are counted from the code and configuration.
The volunteer platform already holds every registration, shift and logged hour. Before an event, though, an organizer who wants to thank a long-standing volunteer or spot a likely future lead has to click through records one person at a time. Most don’t, so recognition stays generic.
The product was going to be sold to many nonprofits from one shared deployment. That made tenant isolation the first requirement, ahead of anything the model does: one organization must never be able to retrieve another organization’s volunteers, even by accident.
The diagram is wider than the screen: swipe sideways to see all of it →
Each customer has its own index and its own Foundry agent, and the agent’s search tool also carries a customer filter as defence in depth. Both services use managed identities, so no secret appears in configuration or in the repository.
One index and one Foundry agent per customer. A filter bug cannot leak data, because the other customers’ data is not in that index. Trade-off: one more provisioning step per customer, so the worker does it once and records the agent id.
The customer is derived from the authenticated Entra tenant on the turn. A tenant that is present but unknown is refused with no email-domain fallback, and a customer whose data is not ready is refused too. Trade-off: anonymous test chats always get the refusal, by design.
Ingestion and conversation deploy, scale and fail separately. Their only coupling is the index schema and the registry. Trade-off: the registry has to be kept in sync between them, which is the next thing to move to a durable store.
gpt-4o classifies intent, with a deterministic keyword fallback so the bot keeps working during an outage. “Refine” rewrites the cached dossier with zero new retrieval. All prompts forbid invented names, hours or history.
The first end-to-end deployment surfaced most of the real work. A selection:
The first resolver fell back to email-domain matching whenever the tenant id didn’t match. A caller from an unrelated tenant with an address at a customer’s domain would have been served that customer’s data.
Tenant present but unknown now means refuse, with no fallback. Rows with no index or agent are refused as well. Eight xUnit tests lock the rules in.
Agent provisioning needs Foundry data-plane rights, which took 5–15 minutes to propagate. A list-or-create call on every pass made each sync fail until then.
The worker provisions once, writes the agent id to the registry and skips the call afterwards. Steady-state syncs no longer depend on Foundry RBAC, and one failing customer no longer stops the others.
Some registrations arrived without one of the four identifiers needed for a stable document key.
Mapping is null-safe. Incomplete records are skipped and counted in the sync log, so they don’t crash the pass or produce duplicates.
The new SDK ignored the old Bot Framework settings, and the host failed at startup until the MSAL provider package was referenced. Once it started, outbound tokens were issued but every reply failed with 401.
Moved to the SDK’s connection configuration, named the vault secret so it lands on the right config key, and pointed the authority at the bot’s home tenant, as single-tenant bots require.
The agent’s App Service had been created on the wrong runtime stack, so the .NET app never started.
Fixed the stack and added a health-endpoint check to the runbook, so this is caught before the bot is wired up.
Custom-engine agents need manifest schema 1.20 or later, and the Copilot scope needs 1.21. A zip built with the wrong tool also produced paths the validator refused.
Moved to schema 1.21 and wrote a build script that puts the three files at the zip root. This and 12 other findings are in the runbook.
An illustrative replay with synthetic organizations and volunteers. It contains no client data or code. Each scenario shows the checks the agent runs before it answers.
| Measure | Before | After | Source |
|---|---|---|---|
| Prep per event | Manual record lookups, ≈30 min | Dossier in under 1 min | estimate |
| Data freshness | Ad-hoc exports | Automatic sync every 6 h | config |
| Onboarding a customer | New product, no process | Config only: 1 row + 2 secrets | derived |
| Provisioning calls per sync | 1 per customer | 0 after the first pass | derived |
| Cross-tenant fallback paths | 1 (email domain) | 0, covered by 8 tests | code |
| Refine a dossier | Re-query and re-draft | Rewrite cached text, 0 retrievals | code |
Source: code = counted in the repository; config = default setting; derived = follows from the code change; estimate = to confirm with pilot data.
Free scoping call · 30 minutes · No commitment
Write a few lines: the process or system, what hurts today (time, errors, cost or an AI pilot that stalled) and who uses it. After a short call you get a written plan and a fixed quote within ≈3 days.
Opens a pre-filled email · reply within one business day
The model reads. The code counts.
All projects