Housing Anomaly Platform · AfifAI
Case study · Data & AI platform · 2026

One review platform for a national housing portfolio's anomalies

Engineering, the service center, finance and the spokesperson's office at an Israeli national public-housing company now work from one Hebrew web app. A daily pipeline turns ERP data into ranked anomalies, AI-written asset reports and review queues, and every human decision is saved safely across replicas behind corporate sign-in.

Python · FastAPINext.js · TypeScriptAzure Container AppsMicrosoft FabricAzure OpenAIEntra IDTerraformAzure DevOps
Client
Israeli national public-housing company (confidential)
My role
Team lead & architect (hands-on)
Stage
Dev + prod on Azure, daily production pipeline
Status
In production
At a glance
2–10backend replicas writing shared review state with no lost updates (was last-write-wins)
33 to 0LLM report calls on a daily run whose evidence didn't change
≈1 MBread per asset card instead of a ≈300 MB table (1 of 300 buckets)
1,846Python tests plus a TypeScript check gating every merge (no gate before)
0public backend endpoints and stored deploy secrets after the Entra cut-over
15review use cases across 4 departments, plus a 10-screen electricity module
42pipeline steps in 8 phases, from ERP lakehouse to reviewed results
9,472electricity bills scored automatically from 9,429 source PDFs
The problem

Tens of thousands of assets, signals scattered across the ERP

The company maintains a very large portfolio of apartments and buildings. Maintenance calls, contractor projects, inspector visits, price catalogs, resident service history and utility bills all sit in different ERP tables. Spotting a contractor who is always late, a building with recurring failures, a project whose quantities don't add up or an empty apartment that still uses electricity meant analysts pulling exports and reading them by hand. Each department had its own spreadsheet habits and no shared record of what had already been reviewed.

What I owned

Architecture, delivery and the production hardening

  • Led the team and set the architecture: an offline pipeline that does the heavy ERP work, and a thin API that only serves prepared results.
  • Built across the stack: FastAPI routers, the Next.js Hebrew UI, the pipeline DAG, detectors and rankers, and the LLM report and assistant flows.
  • Moved delivery into the client's Azure DevOps: PR validation, auto-deploy to dev, approval-gated and master-only prod, and a Terraform plan/apply pipeline.
  • Put production behind Entra ID. The backend became internal-only, and every API call now goes through a same-origin proxy.
  • Found and closed the data-loss paths in shared review state across replicas.
Architecture

Heavy work offline, a thin and private API online

The diagram is wider than the screen: swipe sideways to see all of it →

The backend is not allowed to import raw-data loaders or read ERP snapshots, and a test enforces this. That keeps every request inside a 2 GiB container and makes response time independent of ERP size. Storage is split by owner: the pipeline owns what it computes, and the app owns what people decide. A pipeline rerun can never erase a review.

Design decisions

Keep the request path small, keep humans in charge

Prepared artifacts, not live ERP queries

All joins, features and scoring run in the nightly job (4 CPU / 28 GiB). The API reads partitioned parquet: 300 hash buckets, a 5-minute cache capped at 32 buckets. Trade-off: data is as fresh as the last run, and the run records a manifest so staleness is visible.

Blob storage with optimistic concurrency, not a new database

Review state is a small set of JSON documents. Instead of adding a database, every shared write goes through one helper: read with ETag, re-apply the change, write only if nothing changed in between, retry up to 3 times. Trade-off: heavy contention returns a clean 503 instead of queueing.

Identity at the edge

The app has no login code. Entra ID (Easy Auth) guards the frontend, the browser talks only to a same-origin proxy, and the backend trusts forwarded identity headers only because it has no public ingress. Trade-off: per-user roles live in Entra groups, not in the app.

AI proposes, a person decides

LLM calls return validated, versioned structured output. Rankings and AI assessments are labelled as review priority, not proof of fault. Responses to media inquiries go through draft → approved roles. Trade-off: slower than auto-publishing, but every outcome has an owner.

Releases you can trace and undo

Images are tagged with the commit SHA. Prod runs only from master after an approval, and each deploy waits until the new revision reports healthy (up to 40 checks, 15 s apart). Trade-off: a few minutes more per deploy, but one-step rollback.

Engineering log

Problems that only show up in production

Data safety

A review toggle could wipe every review

Toggling one row's status read the use case's review file, changed one key and saved. If that read failed for any reason, the code treated it as “no reviews yet” and saved a nearly empty file over all of them.

Only a true 404 starts empty. Any other read error propagates and nothing is written. The write is conditional on the version that was read, and the UI shows the reviewer the server actually saved.

Concurrency

Two replicas, one file, one lost edit

Prod runs at least 2 backend replicas. Renovation-priority edits, alert subscriptions and price-catalog approvals were read-modify-write with no version check, so two near-simultaneous saves kept only the later one.

One shared helper does ETag-conditional writes and re-applies the change on newer data. A boundary test fails the build if new code writes shared JSON any other way.

Access

Sign-in without touching app code

The browser called the backend's public URL directly, so a frontend-only sign-in would have left the API open.

API calls now go to a same-origin /backend-api rewrite baked into the build. The backend ingress became internal, and identity headers are trusted only behind that boundary. The header user now comes from the backend, not from a browser call that failed behind the proxy.

Delivery

Deploys that looked green but weren't

The first Azure DevOps builds failed on a wrong Dockerfile path and a floating build agent. A healthy container could also hide a frontend built with the wrong API wiring.

Pinned the agent to Ubuntu 24.04, shared one build-and-deploy script across components, gave each revision a unique suffix, added a wait for health, and added a check that inspects the packaged build's API settings.

Infrastructure

Terraform that could surprise prod

Existing environments were managed from a developer machine, with no reviewed plan.

Plan-only runs publish a summary artifact, and the apply needs an environment approval and runs only from master. A Key Vault admin grant was pinned so plans stop showing false drift.

LLM cost

Regenerating reports nobody's data had changed

The daily run selects the top 20 assets and 13 buildings for AI reports, at one request a minute to respect quota.

Each report carries a fingerprint of its complete evidence: calls, findings, notes and money. Unchanged evidence reuses the stored report. Large regenerations stop unless someone explicitly approves them, and all workers share the Retry-After cooldowns.

Try it

Two reviewers save at the same moment

Replica A and replica B both handle a save to the same review file.

    Illustration of the write protocol. No client data.

    Outcome

    Where it landed

    Area
    Before
    After
    Shared review state
    Last write wins. A failed read could save an empty file.
    ETag-conditional writes with 3 retries. Read errors fail safe (503).
    Access
    Public backend, IP allowlists
    Entra ID on the frontend, backend internal-only
    Release
    Manually triggered workflows, no merge gate
    PR gate (1,846 tests + tsc), auto-deploy dev, approved prod from master
    Release effort
    ≈30–45 min of manual steps per componentestimate
    Merge, then one approval. Health-checked, SHA-tagged rollback.
    AI reports on unchanged data
    33 calls, ≈33 min at quota pacing
    0 calls. The stored report is reused.
    Asset card data read
    Full table, ≈300 MB
    One bucket, ≈1 MB, cached 5 min
    Periodic asset report
    ≈2 h compiling ERP exports by handestimate
    Generated from prepared evidence in minutes
    Electricity bill checks
    ≈300 analyst hours per full pass of ≈9.5k billsestimate
    Scored automatically. Only flagged cases reach a person.

    The electricity module ships switched off until its operator worker and production data are in place. Its warm reload measured 41.8 s at a 696 MiB peak inside the 2 GiB backend limit.

    Stack

    Technology stack

    Backend
    Python 3.12, FastAPI, pandas, PyArrow, Pydantic
    Front end
    Next.js, TypeScript, Hebrew RTL, charts and exports
    Data
    Microsoft Fabric (OneLake Delta / SQL endpoint), partitioned parquet on Azure Blob
    AI
    Azure OpenAI (GPT-5.x, GPT-4o), tool-calling chat, versioned structured prompts
    Cloud
    Azure Container Apps and Jobs, ACR, Key Vault, App Insights, Entra ID
    Delivery
    Terraform, Azure DevOps pipelines, workload identity, pytest

    Free scoping call · 30 minutes · No commitment

    Bring one process. Leave with a plan and a number to beat.

    Write a few lines: the process or system, what hurts today (time, errors, cost or an AI pilot that stalled) and who uses it. After a short call you get a written plan and a fixed quote within ≈3 days.

    Get a free scoping callafif.mansour@gmail.com

    Opens a pre-filled email · reply within one business day

    The model reads. The code counts.

    All projects