Policy Coherence Analyzer · How the analysis works
From policy documents to the coherence brief, one step at a time.
The method works with any set of national policy documents that set targets. The user uploads two or more, as PDF, Word or text files, and names each document’s type, for example an NDC, an NBSAP, a NAP, LDN targets or a sectoral strategy.
Technical details
Documents enter through POST /api/extract, with the document type recorded per file. The example on this page is Mongolia’s analysis: 8 documents and 178 targets.
Each document is read page by page, tables included, and every passage keeps its page number. Forewords, annexes and boilerplate are set aside, so the extraction concentrates on the passages that set targets.
Technical details
Page-aware parsers (pymupdf, python-docx) keep the page spans. Text is split into chunks of about 30,000 characters with a 2,000-character overlap, and a light relevance filter keeps or drops each chunk, keeping it whenever unsure.
The prompt
KEEP or DROP.{chunk}From the relevant passages, each policy target is recorded with the exact sentence it came from. The wording may be lightly tidied, for example a stray label dropped, but it is not reworded, summarised or extended, and each target is labelled to show how it was recorded.
Technical details
A strict verbatim contract: sourceText is copied character for character; the displayed text allows only deterministic clean-up (dropping a numeric label, rejoining broken lines). Every target keeps its sources and page numbers.
The prompt
{chunk}Overlapping sections can pick up the same target twice. Duplicates are merged into one entry that keeps all their source quotes, and every figure, year and named framework is checked against the original document.
Technical details
Consolidation merges near-duplicates and keeps every source; targets with different numbers stay separate. A local check (validate_claim_grounding) then looks for every figure in the source quotes, and a miss marks the target for review (_provenanceFlag).
The check
%, year, ha, tCO2e… in the text:Before any comparison, the extracted targets go back to the user. Each is checked against its source and kept, corrected or removed, and targets the extraction missed can be added. The analysis starts only from a confirmed set.
Technical details
This is the review step of the upload wizard: targets can be kept, dropped, edited (text and page numbers) or added by hand. Rejected targets do not enter the pairing or rating steps.
For each target, the tool notes whether it contains a measurable outcome (a figure, a percentage, an area) and whether it is time-bound (a year or a deadline), and keeps the exact phrases. Both are early indicators of how ready a commitment is for implementation.
The example pair
NBSAP · 1 Spatial planning
A date: “By 2030”
NDC · Water resources 2
Neither a figure nor a date
Technical details
Targets are checked in batches of 12 for isQuantitative and isTimeBound, and the phrases are returned. Administrative references (“Article 2”) are excluded. Works in any language, with no extra language-processing libraries.
The prompt
Article 2 / Target 3 references; return the exact phrasesEvery target is scored against published reference frameworks, one per lens: Biodiversity (the GLOBE taxonomy), Climate mitigation (the IPCC greenhouse gas inventory sectors), Climate adaptation (the UAE Framework for Global Climate Resilience) and Human rights (UNDP guidance, draft under expert review). Each target keeps one primary area and any others it clearly relates to, so the brief can switch lenses without a new analysis. A country can add lenses of its own.
The example pair, under the Biodiversity lens
NBSAP · 1 Spatial planning
Biodiversity planning and finance
NDC · Water resources 2
Sustainable use
Technical details
A relevance score (0–1) for each (target × category); the top-scoring category becomes primary, and any above the relevance threshold are tagged relevant. Lenses: the GLOBE taxonomy with its subcategories, the IPCC greenhouse gas inventory sectors, the seven climate-resilience targets of the Global Goal on Adaptation (UAE Framework for Global Climate Resilience, decision 2/CMA.5) and nine human rights themes, plus a country’s own adaptation goals where it provides them. Nature-based solutions are paused. A country office can set its own lenses, as Sri Lanka did with its nine NDC 3.0 adaptation sectors in place of the Global Goal on Adaptation and a loss and damage lens. A lens with a single category has nothing to rank, so a target counts in it only above the relevance threshold. This is the most call-intensive step (n × c scores), so results are cached.
The prompt
{GLOBE · IPCC sectors · GGA · human rights, or the country's own set}{name} – {description}{target text}primary, others above the threshold are tagged relevantCoherence is about how different documents relate, so each target is paired with every target of the other documents, and not with the targets of its own document. Every pair is rated, with no filter in advance, so links between distant sectors are read too.
Technical details
The cross-product of every pairing of different documents, each pair counted once: for Mongolia’s 8 documents and 178 targets, 13,404 target pairs (NBSAP × NDC alone is 20 × 36 = 720). No filter is applied before the rating.
The logic
Before two targets are compared, a first AI agent, the Target Analyst, breaks each one into five parts: its goal, the action proposed, the ecosystem or area, who is responsible, and the expected outcome. Comparing these parts grounds the rating in what each target sets out to do.
Technical details
Agent 1 extracts five fields, factual only, with no inference beyond the target text. Keeping the breakdown (Agent 1) apart from the rating (Agent 2) keeps ratings grounded and consistent across countries and frameworks.
The prompt
{target}, extract:A second AI agent, the Alignment Advisor, receives only the two breakdowns and rates the pair: strong, moderate or partial alignment, or no clear relationship. It rates a pair a potential misalignment only when it can name how the two interact: a shared resource, the same place, or one making the other harder to reach. A loose link stays partial. Every rating comes with a short explanation.
The example pair
Potential misalignment·Competing for resources: land·Manageable
National biodiversity targets for 2030
1 Spatial planning
Nationally Determined Contribution
Water resources 2
AI explanation · Medium confidence
Both targets support ecosystem protection and planning, but the NDC also names water accumulation facilities, irrigation ponds, canals and pipelines, pasture irrigation, household water delivery, and conditional flow retention and transfer in specific river basins, which can compete with the NBSAP target’s aim to maintain natural ecosystems and prevent further decline across all territory through land-use planning.
Technical details
Runs on gpt-5.4 (Azure OpenAI, temperature 0 for repeatable results). The pipeline stores none, low, medium, high or flagged; the brief shows them as no clear relationship, partial, moderate and strong alignment, and potential misalignment. A potential misalignment also records its type, manageability and confidence, so a reviewer sees why it was given.
The prompt
none / low / medium / high, or flagged.{decomposition A}{decomposition B}With no further AI, the ratings are counted: the share of target pairs that are aligned (strong or moderate alignment), partially aligned, or show potential misalignment, for the whole set, for each pair of documents and for each policy area. These shares are what the coherence brief shows.
The example pair
Between the NDC and the NBSAP, 92% of the 720 target pairs are aligned and 3% show potential misalignment. This pair is 1 of those 19.
Technical details
Counts are computed locally, with no model calls. Aligned counts strong and moderate alignment. For Mongolia: 66% of 13,404 target pairs are aligned, 29% partially aligned and 5% show potential misalignment.
Across all pairs, an AI synthesis groups the aligned and the potentially misaligned pairs into recurring themes, and each potential misalignment is counted by type. Both show where to look first; the final interpretation rests with national experts.
Target pairs with potential misalignment
5% of Mongolia’s target pairs (671 of 13,404)
Types of potential misalignment
The example pair
Of the 19 potential misalignments between the NDC and the NBSAP, 12 concern delivery and coordination and 7 compete for resources. This pair is 1 of the 7; the resource involved is land.
Technical details
A synthesis step reads the type mix and the pairs behind it and groups recurring themes across document pairs; a theme’s panel can add an AI-suggested starting point, anchored to the evidence shown. In Mongolia’s set, 5% of target pairs show potential misalignment, and 52% of those involve just 7 targets.
Every AI explanation in the brief can be rated with a thumbs up or down and an optional note. Ratings are anonymous and kept with the exact text they refer to, so reviewers can see where a reading helped and where it fell short, and the team can use them to improve later runs.
Technical details
Ratings are captured on target-pair explanations, document-pair summaries and theme summaries (POST /api/feedback) in an append-only ledger keyed to the exact AI text, so a rating stays interpretable after the pipeline is re-run.
The same steps compare the budget lines of a Biodiversity Expenditure Review (BER) with the targets, showing which targets have a matching budget line in the reviewed spending.
Technical details
Level 2, finance. Each review is first rebuilt from its own workbook, every published total reproduced. Its lines are described in the review’s and the budget’s own words, without amounts; Claude Opus places each in a GLOBE area, with a second AI model’s reading beside it, for a person to check. A line with no clear area stays in the totals as general biodiversity spending. Each line with reviewed spending is then broken down and rated against every policy target; the explanation says what the spending goes to, never whether the amount is enough. A line counts for every target it matches, so per-target figures overlap. A BER covers only the spending it reviews; every link to a target comes from the AI comparison. Available for Mongolia, Panama and Sri Lanka.
The reuse
The same steps compare actions reported in Biennial Transparency Reports (BTRs) and national reports with the targets: which targets the reported actions address, and where an action is potentially misaligned with a target.
Technical details
Level 3, implementation. Reported measures (BTR, and national reports such as the seventh national report to the Convention on Biological Diversity, NR7) become pseudo-targets through the same two-agent steps; an adapted prompt asks whether a measure directly implements, partially supports, or has no relationship to each target. Reported actions are the country’s own reporting, one view of implementation. Available for Mongolia and Panama; NR7 for Mongolia.
The question