1. Recall Ceiling & Unreachable Reference Targets
Visualises the 0.442 recall ceiling: 53 of 95 reference pairs share zero terms with UN SDG targets, rendering them unreachable by lexical methods regardless of threshold tuning.
Total Projects
23,451
All projects in the set
Target-Level Assigned
9,394
40.1% • High-precision target links
Goal-Only Assigned
6,882
29.3% • Broad goal relevance
Unassigned
7,175
30.6% • Below calibrated threshold
Quick Audits:
Loading index...
Click a row for its SDG evidence, or the title for the CORDIS record.
| Project ID | Acronym | Project Title | Level | Assigned SDGs | Confidence |
|---|---|---|---|---|---|
| Loading project search index (23,451 projects)... | |||||
UN Sustainable Development Goals Distribution
Breakdown of 23,451 Horizon Europe projects mapped across the 17 Sustainable Development Goals. Click a goal to see its targets and how many projects each holds; from there, a click on a target lists those projects on the Home page.
Pipeline Evaluation & Benchmark Metrics
Ground-truth validation results on the held-out 100 evaluation sample and development agreement metrics from the frozen pipeline run (output 0938fe34…, pipeline commit 0246412b, metrics artefact babd3741…).
Target-Level Precision
0.447
Target-level precision on held-out evaluation 100 sample (calibrated threshold).
Target-Level Recall
0.221
Target-level recall on held-out 100 sample (reflects vocabulary ceiling constraints).
Recall Ceiling
0.442
53 of 95 reference pairs share zero vocabulary with official target descriptions at any threshold.
Development Cohen's Kappa
0.677
Agreement between two model seats on development 50 sample (substantial agreement baseline).
Methodological Findings & Constraints
- Vocabulary Divergence & The Recall Ceiling (0.442): Keyword and text-matching pipelines cannot reach 53 of 95 gold-standard target associations because CORDIS project descriptions and UN SDG target texts use non-overlapping terminologies. This ceiling is an inherent property of lexical overlap, not a pipeline bug.
-
Unassigned Rate (30.6%):
7,175 of 23,451 projects receive no SDG assignment (
assignment_level: none). Calibrated thresholding intentionally rejects low-confidence matches to preserve high precision (0.447) rather than forcing arbitrary labels. - Single-Reference Annotation Limit: The reference labels for the 100 evaluation projects come from a single annotator. A planned second annotation of 20 projects by the author was not made, so no adjudication took place and this limitation covers all 100.
- Deterministic Verification & Explanation Pass: To ensure 100% deterministic reproducibility without external API key dependencies, model-generated explanations were parked under the methodology specification during the frozen run.
Self-Proof Reproduction Protocol
Every reported metric is verified by self-proof inside scripts/metrics.py:
python scripts/metrics.py --results data/pipeline/evaluation-100-v2.json --results-sha256 0938fe34e41cd5d8ddccfba69238481fdf6470e54046104ad7a3535da6743709 --dev-results data/pipeline/development-50-v1.json --dev-results-sha256 032a4881774ce9adbbc84bb8a949b7d06843ad77a23aae1c4b77d6de522ea460 --handoff data/sample/handoff-evaluation-100.json --out data/pipeline/evaluation-100-metrics.json
The script validates development-50 agreement (0.677) and held-out 100 target metrics (P=0.447, R=0.221, Ceiling=0.442) against the frozen run output 0938fe34….
Notebook Visualisations (Embedded SVGs)
The six figures committed under figures/ are embedded directly without redrawing or modifications.
2. Disagreement Shape Across Reference Standards
Distribution and topology of classification agreements and disagreements between automated heuristic scoring and manual reference annotations.
3. Goals by Project Count
Frequency of projects mapped to 0, 1, 2, or 3+ SDG goals. 30.6% (7,175) receive 0 assignments, while single-goal and dual-goal projects form the majority of assignments.
4. Most Linked SDG Targets
The top SDG targets linked across Horizon Europe funding, highlighting prominent concentrations in health (Goal 3), climate action (Goal 13), and energy transition (Goal 7).
5. Relevance by CORDIS Field
Signal strength and scoring contribution across project Title, Objective, and Call Title fields, illustrating the critical importance of multi-field provenance.
6. Assignment Confidence Bands
How the assignments split across the three confidence bands, from high-certainty matches to exploratory ones.
Methodology & Provenance
Deterministic classification methodology, taxonomy integration, reproducibility commands, and cryptographic verification digests for the Horizon Europe CORDIS SDG mapping analysis.
What the data is
The analysis operates on official European Union open datasets and established international taxonomies:
- The Horizon Europe projects: A complete snapshot of 23,451 research projects funded under the European Union's Horizon Europe framework programme (2021–2027), sourced from the official CORDIS open data repository. Each project record contains project identifiers, title, acronym, teaser summary, comprehensive objective text, free keywords, and classified scientific domains.
- UN Sustainable Development Goals Taxonomy: The 17 Goals and 169 Targets defined by the United Nations 2030 Agenda for Sustainable Development, sourced from the Publications Office of the European Union (EU Vocabularies). The taxonomy establishes standard descriptions, indicators, and official target statements.
- European Science Vocabulary (EuroSciVoc): The multilingual taxonomy of scientific fields maintained by the Publications Office of the European Union. A curated, peer-reviewed domain crosswalk containing 235 mappings connects EuroSciVoc disciplines directly to relevant SDG targets and goals.
- Five Analyzed Text Fields: Project evidence is extracted across five distinct fields: Project Title, Teaser, Objective, Free Keywords, and EuroSciVoc Fields. The pipeline explicitly tracks which fields are populated in each CORDIS record to distinguish between absent text fields and non-matching text.
What the pipeline does
The system performs deterministic multi-label classification to assign UN SDGs to Horizon Europe research projects:
-
Dual Evidence Architecture:
The classifier synthesises two complementary channels of evidence:
• Lexical Target Matching: High-precision phrase and keyword matching using strict word-boundary regular expressions against SDG target definitions, guarded by negative filters and domain stop-lists to eliminate polysemous false positives.
• Taxonomic Crosswalk: Domain-level assignments inferred from official EuroSciVoc classifications attached to each project. -
Assignment Tiers & Calibrated Precision:
Projects are stratified into three mutually distinct assignment tiers:
• Target-Level Assigned (40.1%, 9,394 projects): Substantive evidence meeting calibrated scoring thresholds for one or more specific SDG targets.
• Goal-Only Assigned (29.3%, 6,882 projects): High-level relevance to broader SDG themes without meeting granular target-level evidence criteria.
• Unassigned (30.6%, 7,175 projects): Projects lacking sufficient lexical or taxonomic overlap are left unassigned, preserving high precision (0.447) rather than introducing speculative labels. - Full Attribution & Auditability: Every assigned target or goal retains complete evidence provenance, recording the exact matched phrases and the specific project fields in which they occurred.
- Source Text Fidelity: All project titles and acronyms are preserved verbatim from raw CORDIS source bytes without normalization or escaping (including leading hyphens in projects 101073045 and 101275778).
What was frozen and when
Methodological specifications, logic, and evaluation artifacts were formally frozen on 12 September 2026:
-
Pipeline Freeze (12 September 2026):
The classification algorithm, matching rules, scoring weights, vocabulary stop-lists, and assignment thresholds were frozen at commit
0246412b44f722567feaa2e6c1b6e695de86c923. - Single Blind Evaluation Run: Evaluation was performed once against the frozen pipeline using a held-out benchmark of 100 projects. The run yielded target-level precision of 0.447 and recall of 0.221, reflecting an inherent vocabulary ceiling of 0.442 where 53 of 95 reference pairs share zero lexical overlap with official UN target texts.
- Fast run checked against the slow run: A performance optimization (substring pre-check skip) was verified against a slow control run across all 23,451 projects (28,833 assigned rows). Two independent instruments confirmed byte-identical results: zero missing projects, zero row differences, and zero score or band discrepancies.
-
Frozen Run Output:
The full evaluation output hashes cryptographically to
0938fe34e41cd5d8ddccfba69238481fdf6470e54046104ad7a3535da6743709.
How to reproduce it
The entire pipeline executes using only the Python standard library (version 3.10+) with zero external package dependencies. Run the reproduction suite in sequence:
python scripts/make_manifest.py python scripts/verify_licences.py python scripts/extract_targets.py python scripts/extract_key_terms.py python scripts/verify_crosswalk.py python scripts/draw_sample.py python scripts/build_handoff.py python scripts/build_handoff.py --set evaluation --out data/sample/handoff-evaluation-100.json python scripts/pipeline.py --handoff data/sample/handoff-150.json --set development --out data/pipeline/development-50-v2.json python scripts/pipeline.py --handoff data/sample/handoff-evaluation-100.json --set evaluation --freeze-seq 207 --out data/pipeline/evaluation-100-v2.json python scripts/explain.py --results data/pipeline/development-50-v2.json --out data/pipeline/development-50-explanations-v2.json python scripts/metrics.py --results data/pipeline/evaluation-100-v2.json --results-sha256 0938fe34e41cd5d8ddccfba69238481fdf6470e54046104ad7a3535da6743709 --dev-results data/pipeline/development-50-v1.json --dev-results-sha256 032a4881774ce9adbbc84bb8a949b7d06843ad77a23aae1c4b77d6de522ea460 --handoff data/sample/handoff-evaluation-100.json --out data/pipeline/evaluation-100-metrics.json python scripts/compare_control_table.py --control <path-to-control-table> python scripts/test_reported_figures.py
Verify these files
Cryptographic digests for all pipeline artefacts and specification documents, verified on disk:
| Artefact | Repository Path | SHA-256 / Commit Digest |
|---|---|---|
| Mapping Table | data/corpus/mapping-table-0246412.csv |
181d378a3078014193f3ae4f684b00b375ed62f3d24b2b34c060f6457b4a00d3 |
| Pipeline Commit | scripts/pipeline.py (frozen commit) |
0246412b44f722567feaa2e6c1b6e695de86c923 |
| Frozen Output | Evaluation run output | 0938fe34e41cd5d8ddccfba69238481fdf6470e54046104ad7a3535da6743709 |
| Specification | docs/cordis-sdg-registration-v10.md |
c5f2a26ce6049e801a7e757efc2cbe4253689987fad761f1d97cfb5b0ad68fb2 |
| Figures Document | data/corpus/figures.json |
03d79580081ca40bf0d1d25b5279a57880ddf09d750490553593387126d92f6f |
| Metrics Artefact | data/pipeline/evaluation-100-metrics.json |
babd37415581ce82ecb09c4501052ce811e834e57dad7c6735389ce321dd8a2e |
| Notebook Figures | figures/*.svg (6 embedded figures) |
Byte-identical SVG vector assets |