Overview
Akordi is an AI-powered construction intelligence platform.
The work centred on one question: how can project teams hand slow, manual, expert-bound constructability work to a multi-agent AI system, and still stay confidently in control of decisions that carry real consequences?
What I owned
I owned the product from strategy and design, through hands-on prototyping, to delivery in production.
- Vision and roadmap. The long-term product vision, strategy and roadmap for an AI-native experience, in a fast-moving, ambiguous space.
- How people work with the agents. How users delegate to specialist agents, then review and steer what comes back.
- Quality. Evaluation methods, quality gates and feedback loops with construction experts.
- The data underneath. Turning deep domain knowledge into structured, machine-readable data (objectives, requirements, constraints, characteristics and solutions) that grounds the agents and makes their output explainable.
- Hands-on prototyping. Building with LLMs and full-stack code, tested against real project documents.
The problem
Constructability work had two problems.
Expertise is locked in people’s heads
Constructability knowledge sits with a handful of experts. The work is slow, manual and hard to scale.
There is no traceable “why”
It is hard to see why a solution was proposed, and how it addresses the client’s objectives and requirements. Without that link, nobody can check it, and an AI that proposes one is even harder to trust.
The solution
Three specialist agents In production
Each agent does one job and runs on its own: the objectives agent identifies a project’s objectives, requirements and constraints; the risk agent identifies its risks and opportunities; the constructability agent identifies suitable construction solutions.
Grounded, not guessed
The constructability agent answers from a rather than from the model’s memory. I rebuilt that library into about 1,600 fully traceable, licence-clean solutions, so every recommendation can be checked and defended.
Choosing the agent’s design by testing it
There were several ways to design the constructability agent: where its answers come from (the model’s own knowledge, the library, or both), and whether extra context, such as the project’s objectives and risks, makes them better. Rather than pick one on instinct, I built four versions and tested them against each other. The design that won answers from the library, using the context the other agents extract, as long as the library covers enough of what the brief needs.
Connected end to end: a knowledge graph Prototype
In the prototype, each agent’s output feeds the next, and together they build one connected model of the project: its objectives, requirements, constraints and characteristics, the risks and opportunities they create, and the construction solutions that address them. Every link is stored, so the result is a project knowledge graph. Start from a solution and trace it back to the objective it serves, or start from an objective and see every risk and solution connected to it.
The three agents are in production individually; connecting them end to end into this knowledge graph is still a prototype. The connection grew out of my Digital RFP concept, which set what each agent takes in and passes on.
How success is measured
Two kinds of measure, kept separate. Agent quality is whether an agent’s output is good enough, and it is not a one-off check before launch. Each agent was evaluated on synthetic project briefs before deployment, and every new version is run on the same briefs and compared with the previous one before it goes live. Anything that can be checked by code is; an AI judge, checked against experts, handles only the judgement calls. Product success is whether teams use the agents, come back to them and get a better result: it was defined, but not yet measured.
↑ higher is better · ↓ lower is better · KPI a metric with a target · ✓ Met tested and met its target · Below target tested, target missed · Tested measured, no target result to show · Planned defined, not yet measured. Testing used synthetic project briefs: realistic briefs written for the test, with a known answer where there is one, run before deployment.
Evaluating agent quality
Is the output good enough? Checked before deployment, then again with every new version in production: run on the same briefs and compared with the previous version before it replaces it.
Objectives agent
Extraction: measured against a known answer
- Recall ↑KPI ≥85%Below target
- % of the objectives, requirements and constraints in synthetic project briefs with a known answer key that it picks up.
- Precision ↑KPI ≥70%Below target
- % of what it extracts that is right, against the same briefs.
- Grade ↑KPI ≥80%Below target
- % of records graded B or better on synthetic project briefs. Record grades A to D measure output qualityAAccepted as isShownBNeeds a quick glanceShownCMust be correctedHidden unless requestedDDiscardedNever shownHow a record is graded: a separate QA process, with no shared memory with the extraction, re-checks every record on five things: a verbatim evidence match, a full citation, the right record type, the best tag, and a self-contained statement. The score sets the grade, and each record carries notes on any check it failed.
- Groundedness ↑KPI 100%✓ Met
- % of extracted items that can actually be found in the source document.
Risk agent
Extraction: measured against a known answer
- Recall ↑KPI ≥85%Below target
- % of the known risks in synthetic project briefs that it finds.
- Precision ↑KPI ≥70%Below target
- % of the risks it finds that are in the answer key.
- Grade ↑KPI ≥80%Below target
- % of risks graded B or better on the same briefs. Record grades A to D measure output qualityAAccepted as isShownBNeeds a quick glanceShownCMust be correctedHidden unless requestedDDiscardedNever shownHow a record is graded: a separate QA process, with no shared memory with the extraction, re-checks every record on five things: a verbatim evidence match, a full citation, the right record type, the best tag, and a self-contained statement. The score sets the grade, and each record carries notes on any check it failed.
- Groundedness ↑KPI 100%✓ Met
- % of risks that can actually be found in the source document.
Constructability agent
Generation: no single right answer
- Relevance ↑KPI ≥80%✓ Met
- % of solutions scored 3 or higher for fit with the brief, on a 1 to 4 scale by experts and an AI judge; 4 is a non-obvious “lightbulb”.
- Novelty ↑KPI ≥1 per briefBelow target
- Non-obvious “lightbulb” solutions per brief, the ones an expert would not reach for by default. Met on 3 of 4 test briefs; the miss was the brief the library covered least.
- Traceable why ↑KPI ≥90%Below target
- % of solutions linked to the requirement, constraint or risk they address.
- Groundedness ↑KPI 100%✓ Met
- % of solutions that can be found in , never invented.
Connected workflow
Prototype
- Traceability ↑Planned
- % of solutions that trace back through a risk to the client objective it serves.
- Expert acceptance ↑Planned
- % of A and B records experts accept without changes.
On the platform, everything the agents found was really in the document, but they missed too much and some items were not right. Groundedness met its target; recall, precision and grade fell short. The biggest gap, recall, came from how the platform processes documents into the knowledge base every agent reads from, not from the model.
Measuring product success Planned
Do teams use the agents, come back to them, and get a better result? Adoption, retention and outcomes, defined for measuring as use grows.
- North star metric ↑
- % of critical constructability risks and requirements addressed in the bid.
- Leading indicator ↑
- % of eligible bids where the agents’ outputs were used in bid decisions.
- Lagging indicator ↑
- Bid win rate and client feedback on the bid, against comparable projects that did not use the agents.
Objectives agent
- Adoption · Use ↑Leading
- % of eligible projects that run it.
- Retention · Re-use ↑Leading
- % of teams that use it again on their next eligible project.
- Outcome · Speed ↓Leading
- Expert hours to a reviewed, usable objectives, requirements and constraints register, against doing it by hand.
Risk agent
- Adoption · Use ↑Leading
- % of eligible projects that run it.
- Retention · Re-use ↑Leading
- % of teams that use it again on their next eligible project.
- Outcome · Speed ↓Leading
- Expert hours to a reviewed, usable risk register, against doing it by hand.
Constructability agent
- Adoption · Use ↑Leading
- % of eligible projects that run it.
- Retention · Re-use ↑Leading
- % of teams that use it again on their next eligible project.
- Outcome · Speed ↓Leading
- Expert hours to a reviewed shortlist of construction solutions, against working it out by hand in workshops.
- Outcome · Shortlisted ↑Leading
- % of suggested solutions selected for further assessment.
Connected workflow
Prototype
- Adoption · Use ↑Leading
- % of eligible projects run through the connected workflow.
- Retention · Re-use ↑Leading
- % of teams that run the connected workflow again on their next eligible project.
- Outcome · Speed ↓Leading
- Expert hours from tender documents to reviewed objectives, risks and shortlisted solutions, against doing it by hand.
Expert hours include the time spent reviewing and correcting the agents’ output. Retention counts only teams that have since started another eligible project. Leading: measurable while the bid is prepared. Lagging: known once the bid result is in.
Guardrails what must not get worse as use grows
- False alarms ↓Leading % of flagged risks that experts dismiss as not real. Too many, and experts stop trusting the output.
- Missed critical risks ↓Lagging Critical risks found later, in review or on site, that the agents did not flag. In construction these cost the most, in safety and money.
Outcome
Evaluation showed where the agents delivered and where their input held them back.
Objectives and risks
Below targetEvery item the agents found was really in the source document, but recall, precision and grade stayed below target. I investigated the document-processing pipeline with engineering and proposed changes; the work had not started when I left.
Source traceability
WorkingEvery output links back to its source, so teams can check the evidence. Linking each construction solution to the objectives and risks it addresses is the knowledge graph, still a prototype.
Construction solutions
Shown in testingGrounded in the library, the agent beat AI given the same context on weighted quality in three of four synthetic briefs, with a near tie on the fourth. Coverage gaps remained, and letting the AI add its own solutions showed no consistent improvement.
What I learned
Traceable is not the same as useful.
Every solution in the library could be traced to its source, but the library did not cover enough of each brief’s needs to recommend solutions consistently. An agent’s output is only as good as its input, so I would now validate data coverage, and how the data will be produced and maintained, before building the agent.
An AI judge can assist evaluation, but it does not replace experts.
In testing, the AI judge and an expert agreed within one band on 74 to 92% of solutions, depending on which expert it was compared with and whether the judge was calibrated, against 81% between two experts. That supports using it to assist evaluation; experts are still needed to calibrate it and to make the judgement calls.
What is live and what is not: the objectives, risk and constructability agents are each in production on their own; connecting them end to end into a knowledge graph is still a prototype.