{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "introduction",
   "metadata": {},
   "source": [
    "# Prediction Boundary\n",
    "### Reproduce the numbers behind a delayed optical-control comparison\n",
    "\n",
    "**Noether Labs · 18 November 2026**\n",
    "\n",
    "A telescope's correction reaches its mirror after the atmosphere has moved. Predicting that motion can help. Whether it helps enough depends on the operating condition and the evidence used to compare controllers.\n",
    "\n",
    "This notebook reconstructs the primary statistics from **600 held-out simulated trials**. It checks all **120 predeclared comparisons**, plots the twelve predictor-versus-integrator conditions, and works through one result: about **49.10 → 42.75 nm** of residual wavefront error in the clean, low-flux, faster-wind condition.\n",
    "\n",
    "Choose **Runtime → Run all** in Colab. A CPU runtime is sufficient. The only download is a **1.35 MB JSON report** at a fixed Git commit; its SHA-256 fingerprint must match before any calculation. No API keys, GPU, package installation or raw-dataset download is required in a standard Colab Python environment.\n",
    "\n",
    "This is a reconstruction of statistics from published per-trial metrics. It does **not** rerun the optical simulation or reconstruct those metrics from raw wavefronts. The separate physical audit and complete raw release are linked at the end."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "download-notes",
   "metadata": {},
   "source": [
    "## 1. Obtain the exact report\n",
    "\n",
    "The revision and fingerprint below identify a specific published result. If either the source changes or a transfer corrupts the bytes, the notebook stops. The filename alone is not the evidence identity."
   ]
  },
  {
   "cell_type": "code",
   "id": "download",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import hashlib\n",
    "import itertools\n",
    "import json\n",
    "import math\n",
    "import statistics\n",
    "import urllib.request\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.stats import t as student_t\n",
    "\n",
    "REVISION = \"3be53dfc1f1340d0c120c7bd5d600ea6b80f48f0\"\n",
    "REPORT_URL = (\n",
    "    f\"https://raw.githubusercontent.com/noetherlabs/noether-ao/{REVISION}/\"\n",
    "    \"results/controller_study_v1/report.json\"\n",
    ")\n",
    "REPORT_SHA256 = \"d0f0ee69bcc19cd5aa0408197343cb2155508237f28692b857a5fa728ddf06be\"\n",
    "MAX_REPORT_BYTES = 2 * 1024**2\n",
    "\n",
    "\n",
    "def decode_report(payload):\n",
    "    if not isinstance(payload, bytes) or len(payload) > MAX_REPORT_BYTES:\n",
    "        raise ValueError(\"The report is absent or exceeds the 2 MiB download limit.\")\n",
    "    if hashlib.sha256(payload).hexdigest() != REPORT_SHA256:\n",
    "        raise ValueError(\"Report fingerprint differs. Stop; do not substitute another revision.\")\n",
    "    return json.loads(payload)\n",
    "\n",
    "\n",
    "with urllib.request.urlopen(REPORT_URL, timeout=30) as response:\n",
    "    report_bytes = response.read(MAX_REPORT_BYTES + 1)\n",
    "report = decode_report(report_bytes)\n",
    "print(f\"Verified {len(report_bytes):,} bytes at commit {REVISION[:12]}.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "method",
   "metadata": {},
   "source": [
    "## 2. What is being compared?\n",
    "\n",
    "Five selected controllers were evaluated in twelve conditions: two photon levels × two wind multipliers × three fault settings. Each condition has the same ten paired atmosphere/detector seed identities. This gives 5 × 12 × 10 = 600 trials, but only **ten independent seed units within a condition**. Frames and controller variants are not extra independent replicates.\n",
    "\n",
    "The episode contains 512 frames. Its primary score uses frames 128–511. Faults begin at frame 256; the primary score therefore includes the earlier part of the episode by design. We do not move that window after looking at the result.\n",
    "\n",
    "For run $s$, the reported wavefront error is\n",
    "\n",
    "$$\n",
    "W_s = \\frac{750\\,\\mathrm{nm}}{2\\pi}\n",
    "\\sqrt{\\frac{1}{384}\\sum_{t=128}^{511}\\operatorname{Var}_{\\mathrm{pupil}}[\\phi_{s,t}]}.\n",
    "$$\n",
    "\n",
    "Here $\\phi$ is residual phase in radians. This is the root of mean phase variance, rather than the mean of framewise RMS errors. This notebook starts from the retained $W_s$ values.\n",
    "\n",
    "For a pair of controllers, define $d_s=W_{s,\\mathrm{left}}-W_{s,\\mathrm{right}}$. The fixed comparison reports\n",
    "\n",
    "$$\n",
    "\\bar d \\;\\pm\\; t_{9,\\,1-0.05/(2\\times120)}\\;\\frac{s_d}{\\sqrt{10}}.\n",
    "$$\n",
    "\n",
    "There are $\\binom{5}{2}\\times12=120$ comparisons. Keeping that family size matters even when we display only twelve. Bonferroni does not require independence between comparisons. The Student-t interval is exact for independent normal seed differences; with ten seeds and a nonnormal distribution, its coverage is approximate.\n",
    "\n",
    "An interval excluding zero is only part of the decision. Every trial of the proposed winner must also have finite outputs, improve on its zero-mirror reference, and stay within the 1% saturation bound. We reconstruct those checks rather than treating a positive mean as sufficient evidence.\n",
    "\n",
    "The code below validates complete trial and comparison identities. It then recomputes each mean, interval and winner from per-trial metrics, checking the stored paired differences as well. A small floating-point tolerance applies to arithmetic; the report itself still requires an exact byte fingerprint."
   ]
  },
  {
   "cell_type": "code",
   "id": "contract",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "CONTROLLERS = (\"integrator\", \"predictive\", \"forward\", \"fixed\", \"learned\")\n",
    "PHOTONS = (1000.0, 1000000.0)\n",
    "WINDS = (0.5, 2.0)\n",
    "FAULTS = (\"none\", \"optical_gain\", \"vibration\")\n",
    "SEEDS = tuple(range(23001, 23011))\n",
    "FAMILY_SIZE = 120\n",
    "FAMILY_ALPHA = 0.05\n",
    "SATURATION_LIMIT = 0.01\n",
    "CONDITIONS = tuple(itertools.product(PHOTONS, WINDS, FAULTS))\n",
    "PAIRS = tuple(itertools.combinations(CONTROLLERS, 2))\n",
    "EXPECTED_TRIALS = set(itertools.product(CONTROLLERS, PHOTONS, WINDS, FAULTS, SEEDS))\n",
    "EXPECTED_COMPARISONS = {\n",
    "    (left, right, photons, wind, fault)\n",
    "    for left, right in PAIRS\n",
    "    for photons, wind, fault in CONDITIONS\n",
    "}\n",
    "\n",
    "# These tolerances apply only to the independently recomputed floating-point\n",
    "# statistics. Authentication above always requires the exact original bytes.\n",
    "STAT_ATOL_NM = 1e-9\n",
    "STAT_RTOL = 1e-11\n",
    "\n",
    "\n",
    "def finite_number(value):\n",
    "    return type(value) in (int, float) and math.isfinite(value)\n",
    "\n",
    "\n",
    "def index_trials(document):\n",
    "    if (document.get(\"status\") != \"complete_descriptive_report\"\n",
    "            or document.get(\"study_id\") != \"controller_study_v1\"\n",
    "            or document.get(\"post_burn_frames\") != [128, 512]\n",
    "            or document.get(\"primary_pairwise_family\") != FAMILY_SIZE\n",
    "            or document.get(\"familywise_alpha\") != FAMILY_ALPHA\n",
    "            or document.get(\"evaluation_records\") != len(EXPECTED_TRIALS)):\n",
    "        raise ValueError(\"The report does not describe the frozen primary study.\")\n",
    "    trials = document.get(\"evaluation_trials\")\n",
    "    if type(trials) is not list or len(trials) != len(EXPECTED_TRIALS):\n",
    "        raise ValueError(\"Exactly 600 held-out trial records are required.\")\n",
    "    indexed = {}\n",
    "    for trial in trials:\n",
    "        case = trial[\"case\"]\n",
    "        controller = case[\"candidate\"][\"controller\"]\n",
    "        photons, wind, fault, seed = (case[k] for k in (\"photons\", \"wind_scale\", \"fault\", \"seed\"))\n",
    "        if (type(controller) is not str or type(fault) is not str\n",
    "                or type(seed) is not int or not finite_number(photons)\n",
    "                or not finite_number(wind) or case.get(\"stage\") != \"evaluation\"\n",
    "                or type(case.get(\"detector_seed\")) is not int\n",
    "                or case[\"detector_seed\"] != seed + 100000):\n",
    "            raise ValueError(\"Malformed evaluation identity or detector pairing.\")\n",
    "        key = controller, photons, wind, fault, seed\n",
    "        if key not in EXPECTED_TRIALS or key in indexed:\n",
    "            raise ValueError(\"Unexpected or duplicate held-out trial identity.\")\n",
    "        expected_id = \"predictive-01\" if controller == \"predictive\" else controller + \"-04\"\n",
    "        if case[\"candidate\"][\"id\"] != expected_id or trial.get(\"status\") != \"completed\":\n",
    "            raise ValueError(\"Wrong selected candidate or incomplete trial.\")\n",
    "        metric = trial[\"post_burn\"]\n",
    "        if (type(metric.get(\"all_outputs_finite\")) is not bool\n",
    "                or not all(finite_number(metric.get(k))\n",
    "                           for k in (\"wfe_nm\", \"open_wfe_nm\", \"saturation_fraction\"))\n",
    "                or metric[\"wfe_nm\"] < 0 or metric[\"open_wfe_nm\"] <= 0\n",
    "                or not 0 <= metric[\"saturation_fraction\"] <= 1):\n",
    "            raise ValueError(\"Missing or invalid primary trial metrics.\")\n",
    "        indexed[key] = metric\n",
    "    if set(indexed) != EXPECTED_TRIALS:\n",
    "        raise ValueError(\"The declared held-out cohort is incomplete.\")\n",
    "    return indexed\n",
    "\n",
    "\n",
    "def candidate_eligible(indexed, controller, condition):\n",
    "    rows = [indexed[(controller, *condition, seed)] for seed in SEEDS]\n",
    "    return all(row[\"all_outputs_finite\"] is True\n",
    "               and row[\"wfe_nm\"] < row[\"open_wfe_nm\"]\n",
    "               and row[\"saturation_fraction\"] <= SATURATION_LIMIT for row in rows)\n",
    "\n",
    "\n",
    "def paired_summary(indexed, left, right, condition):\n",
    "    differences = [indexed[(left, *condition, seed)][\"wfe_nm\"]\n",
    "                   - indexed[(right, *condition, seed)][\"wfe_nm\"] for seed in SEEDS]\n",
    "    mean = statistics.mean(differences)\n",
    "    sd = statistics.stdev(differences)\n",
    "    critical = float(student_t.ppf(1 - FAMILY_ALPHA / (2 * FAMILY_SIZE), len(SEEDS) - 1))\n",
    "    half_width = critical * sd / math.sqrt(len(SEEDS))\n",
    "    low, high = mean - half_width, mean + half_width\n",
    "    winner = None\n",
    "    if low > 0 and candidate_eligible(indexed, right, condition):\n",
    "        winner = right\n",
    "    elif high < 0 and candidate_eligible(indexed, left, condition):\n",
    "        winner = left\n",
    "    return dict(mean_nm=mean, low_nm=low, high_nm=high, n=len(SEEDS),\n",
    "                paired_differences_nm=differences, winner=winner)\n",
    "\n",
    "\n",
    "def close(actual, expected, label):\n",
    "    if not (finite_number(actual) and finite_number(expected)\n",
    "            and math.isclose(actual, expected, abs_tol=STAT_ATOL_NM, rel_tol=STAT_RTOL)):\n",
    "        raise ValueError(f\"Reconstruction differs: {label}.\")\n",
    "\n",
    "\n",
    "def reproduce(document):\n",
    "    indexed = index_trials(document)\n",
    "    rows = document.get(\"comparisons\")\n",
    "    if type(rows) is not list or len(rows) != FAMILY_SIZE:\n",
    "        raise ValueError(\"All 120 predeclared comparisons are required.\")\n",
    "    reconstructed = {}\n",
    "    for saved in rows:\n",
    "        key = saved[\"left\"], saved[\"right\"], saved[\"photons\"], saved[\"wind_scale\"], saved[\"fault\"]\n",
    "        if key not in EXPECTED_COMPARISONS or key in reconstructed:\n",
    "            raise ValueError(\"Unexpected or duplicate comparison identity.\")\n",
    "        if saved[\"difference\"] != \"left_minus_right\":\n",
    "            raise ValueError(\"Comparison sign convention changed.\")\n",
    "        primary = saved[\"post_burn\"]\n",
    "        if (primary[\"status\"] != \"complete\" or primary[\"family_size\"] != FAMILY_SIZE\n",
    "                or primary[\"inference\"] != \"bonferroni_simultaneous\"\n",
    "                or primary[\"interval\"][\"n\"] != len(SEEDS)):\n",
    "            raise ValueError(\"Primary comparison rule changed.\")\n",
    "        computed = paired_summary(indexed, key[0], key[1], key[2:])\n",
    "        paired = primary[\"paired_differences_nm\"]\n",
    "        if type(paired) is not list or len(paired) != len(SEEDS):\n",
    "            raise ValueError(\"All ten paired differences are required.\")\n",
    "        for actual, expected in zip(computed[\"paired_differences_nm\"], paired):\n",
    "            close(actual, expected, \"paired WFE difference\")\n",
    "        for field in (\"mean_nm\", \"low_nm\", \"high_nm\"):\n",
    "            close(computed[field], primary[\"interval\"][field], field)\n",
    "        close(primary[\"interval\"][\"confidence\"], 1 - FAMILY_ALPHA / FAMILY_SIZE,\n",
    "              \"marginal confidence\")\n",
    "        if computed[\"winner\"] != primary[\"winner\"]:\n",
    "            raise ValueError(\"Interval and eligibility gates do not reproduce the retained winner.\")\n",
    "        reconstructed[key] = computed\n",
    "    if set(reconstructed) != EXPECTED_COMPARISONS:\n",
    "        raise ValueError(\"Comparison family is incomplete.\")\n",
    "    return indexed, reconstructed\n",
    "\n",
    "\n",
    "trials, comparisons = reproduce(report)\n",
    "print(f\"Reconstructed {len(trials)} held-out trials and all {len(comparisons)} primary contrasts.\")\n",
    "print(f\"Statistics matched using max({STAT_ATOL_NM:g} nm, {STAT_RTOL:g} × magnitude) tolerance.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "results-notes",
   "metadata": {},
   "source": [
    "## 3. Keep the whole operating envelope visible\n",
    "\n",
    "Positive differences below favor the predictor. “Uncertain” means the frozen procedure did not establish a winner; it does not mean the controllers are equivalent. The same shared seed set is reused across conditions."
   ]
  },
  {
   "cell_type": "code",
   "id": "table",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "FAULT_LABELS = {\"none\": \"Clean\", \"optical_gain\": \"Optical gain\", \"vibration\": \"Vibration\"}\n",
    "prediction = [comparisons[(\"integrator\", \"predictive\", *condition)] for condition in CONDITIONS]\n",
    "print(\"Photons    Wind  Condition       Integrator - predictor [adjusted interval], nm\")\n",
    "for condition, row in zip(CONDITIONS, prediction):\n",
    "    photons, wind, fault = condition\n",
    "    verdict = \"predictive advantage\" if row[\"winner\"] == \"predictive\" else \"uncertain\"\n",
    "    print(f\"{photons:9,.0f}  {wind:3.1f}  {FAULT_LABELS[fault]:14s}\"\n",
    "          f\" {row['mean_nm']:7.2f} [{row['low_nm']:7.2f}, {row['high_nm']:7.2f}]  {verdict}\")\n",
    "supported_prediction = sum(row[\"winner\"] == \"predictive\" for row in prediction)\n",
    "learned_beats_both = sum(\n",
    "    comparisons[(\"integrator\", \"learned\", *condition)][\"winner\"] == \"learned\"\n",
    "    and comparisons[(\"forward\", \"learned\", *condition)][\"winner\"] == \"learned\"\n",
    "    for condition in CONDITIONS\n",
    ")\n",
    "print(f\"\\nPredictor supported against integrator: {supported_prediction}/12 conditions.\")\n",
    "print(f\"Learned mixture supported against both integrator and forward: {learned_beats_both}/12.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "id": "figure",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "plt.rcParams.update({\"font.family\": \"STIXGeneral\", \"mathtext.fontset\": \"stix\",\n",
    "                     \"font.size\": 12, \"axes.spines.top\": False,\n",
    "                     \"axes.spines.right\": False, \"svg.fonttype\": \"path\"})\n",
    "figure, axes = plt.subplots(1, 2, figsize=(12, 22 / 3), sharex=True, sharey=True)\n",
    "condition_labels = [f\"Wind {wind:g} / {FAULT_LABELS[fault]}\" for wind in WINDS for fault in FAULTS]\n",
    "for ax, photons in zip(axes, PHOTONS):\n",
    "    ax.axvline(0, color=\"0.55\", linestyle=\"--\", linewidth=0.8)\n",
    "    for y, (wind, fault) in enumerate(itertools.product(WINDS, FAULTS)):\n",
    "        row = comparisons[(\"integrator\", \"predictive\", photons, wind, fault)]\n",
    "        color = \"#266a83\" if row[\"winner\"] == \"predictive\" else \"#636363\"\n",
    "        ax.plot([row[\"low_nm\"], row[\"high_nm\"]], [y, y], color=color, linewidth=1.5)\n",
    "        ax.plot(row[\"mean_nm\"], y, \"o\", color=color, markersize=4)\n",
    "    ax.set_title(f\"{photons:,.0f} photons / frame\", loc=\"left\")\n",
    "    ax.set_xlabel(\"Integrator − predictor WFE (nm)\")\n",
    "    ax.set_yticks(range(6), condition_labels)\n",
    "    ax.tick_params(axis=\"y\", length=0)\n",
    "axes[0].invert_yaxis()\n",
    "figure.suptitle(\"Where prediction helps\", fontsize=17, x=0.02, ha=\"left\", y=0.98)\n",
    "figure.text(0.02, 0.92, \"All twelve conditions from the completed simulated comparison\", fontsize=12)\n",
    "figure.text(0.02, 0.04,\n",
    "            \"Positive values favor prediction. Ten paired seeds per condition; intervals retain the full 120-comparison correction.\\n\"\n",
    "            \"Blue: interval and trial eligibility support prediction. Gray: uncertainty remains; this does not establish equivalence.\",\n",
    "            fontsize=10, color=\"0.3\")\n",
    "figure.subplots_adjust(left=0.19, right=0.98, bottom=0.20, top=0.85, wspace=0.08)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "seed-spread-notes",
   "metadata": {},
   "source": [
    "### Nearly the same mean is not the same evidence\n",
    "\n",
    "In clean, faster-wind operation, the mean integrator-minus-predictor difference is about 6.35 nm at low flux and 6.58 nm at high flux. The conclusions differ because the ten seed differences have very different spreads.\n",
    "\n",
    "At high flux, seed 23005 contributes a 46.40 nm difference; the other nine lie between about 1.98 and 2.48 nm. The predeclared Student-t interval reflects that spread. We retain all ten seeds and the original interval. This descriptive inspection neither diagnoses the cause of that run nor justifies removing it. With only two tested photon levels, the study also cannot locate a flux threshold at which prediction starts or stops helping."
   ]
  },
  {
   "cell_type": "code",
   "id": "seed_spread",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "seed_figure, seed_axes = plt.subplots(1, 2, figsize=(12, 22 / 3), sharex=True, sharey=True)\n",
    "seed_examples = []\n",
    "for ax, photons in zip(seed_axes, PHOTONS):\n",
    "    row = comparisons[(\"integrator\", \"predictive\", photons, 2.0, \"none\")]\n",
    "    differences = row[\"paired_differences_nm\"]\n",
    "    seed_examples.append(row)\n",
    "    ax.axvline(0, color=\"0.55\", linestyle=\"--\", linewidth=0.8)\n",
    "    ax.plot(differences, range(10), \"o\", color=\"#266a83\", markersize=4)\n",
    "    ax.plot([row[\"low_nm\"], row[\"high_nm\"]], [11, 11], color=\"#222222\", linewidth=1.5)\n",
    "    ax.plot(row[\"mean_nm\"], 11, \"D\", color=\"#222222\", markersize=5)\n",
    "    ax.axhline(10, color=\"0.85\", linewidth=0.7)\n",
    "    ax.set_title(f\"{photons:,.0f} photons / frame\", loc=\"left\", pad=12)\n",
    "    ax.set_xlabel(\"Integrator − predictor WFE (nm)\")\n",
    "    ax.set_yticks([*range(10), 11], [*map(str, SEEDS), \"Mean + interval\"])\n",
    "    ax.tick_params(axis=\"y\", length=0)\n",
    "seed_axes[0].invert_yaxis()\n",
    "seed_axes[0].set_ylabel(\"Paired seed\", labelpad=10)\n",
    "seed_figure.suptitle(\"Similar means, different uncertainty\", fontsize=17,\n",
    "                    x=0.02, ha=\"left\", y=0.98)\n",
    "seed_figure.text(0.02, 0.92, \"Clean operation, wind multiplier 2; every evaluation seed retained\", fontsize=12)\n",
    "seed_figure.text(0.02, 0.04,\n",
    "                 \"Circles: individual paired differences. Diamonds: means, with the original 120-comparison-adjusted intervals.\\n\"\n",
    "                 \"Both panels use the same horizontal scale. No seed is removed or downweighted.\",\n",
    "                 fontsize=10, color=\"0.3\")\n",
    "seed_figure.subplots_adjust(left=0.16, right=0.98, bottom=0.20, top=0.85, wspace=0.08)\n",
    "plt.show()\n",
    "for photons, row in zip(PHOTONS, seed_examples):\n",
    "    print(f\"{photons:,.0f} photons: mean {row['mean_nm']:.6f} nm; \"\n",
    "          f\"interval [{row['low_nm']:.6f}, {row['high_nm']:.6f}] nm\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "example-notes",
   "metadata": {},
   "source": [
    "## 4. Work through one condition\n",
    "\n",
    "Take 1,000 photons per frame, wind multiplier 2, and no injected fault. We calculate the two means and their paired difference directly. The percentage below divides the **difference of controller means** by the integrator mean. It is neither the mean of ten percentage improvements nor a pooled effect across the instrument's operating envelope."
   ]
  },
  {
   "cell_type": "code",
   "id": "example",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "condition = (1000.0, 2.0, \"none\")\n",
    "integrator_mean = statistics.mean(trials[(\"integrator\", *condition, seed)][\"wfe_nm\"] for seed in SEEDS)\n",
    "predictive_mean = statistics.mean(trials[(\"predictive\", *condition, seed)][\"wfe_nm\"] for seed in SEEDS)\n",
    "example = comparisons[(\"integrator\", \"predictive\", *condition)]\n",
    "percent_of_integrator_mean = 100 * (integrator_mean - predictive_mean) / integrator_mean\n",
    "print(f\"Integrator mean: {integrator_mean:.6f} nm\")\n",
    "print(f\"Predictor mean:  {predictive_mean:.6f} nm\")\n",
    "print(f\"Paired difference: {example['mean_nm']:.6f} nm\")\n",
    "print(f\"Adjusted interval: [{example['low_nm']:.6f}, {example['high_nm']:.6f}] nm\")\n",
    "print(f\"Difference / integrator mean: {percent_of_integrator_mean:.3f}%\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "interpretation",
   "metadata": {},
   "source": [
    "The complete comparison supports prediction over the integrator in five of twelve conditions, all at the lower photon level. The remaining seven comparisons are uncertain. The learned mixture does not beat both the integrator and the nonlinear forward controller in any condition under the frozen rule.\n",
    "\n",
    "That pattern is a reason to investigate a particular operating region. It does not isolate why the region exists. Temporal prediction, measurement noise, controller tuning and finite-sample uncertainty all enter this experiment. A causal explanation would require a separate ablation.\n",
    "\n",
    "The learned result is also specific to the implementation and training budget: one initialization and a finite optimization run. It does not rule out a better-trained mixture or a different learned controller. Some statistically separated controller differences in the full family are only hundredths of a nanometre; the protocol did not declare a minimum improvement worth adopting on an instrument."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "integrity-notes",
   "metadata": {},
   "source": [
    "## 5. Try to break the reconstruction\n",
    "\n",
    "These small checks make the notebook refuse missing or duplicate trials, a new seed, altered metrics, duplicate comparisons, a reduced comparison family, or a changed report. The last check leaves a favorable numerical interval intact but makes one proposed-winner trial exceed the saturation bound. It must withhold that winner.\n",
    "\n",
    "The copies exist only in memory. Nothing rewrites the downloaded evidence."
   ]
  },
  {
   "cell_type": "code",
   "id": "checks",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from copy import deepcopy\n",
    "\n",
    "\n",
    "def must_refuse(label, operation):\n",
    "    try:\n",
    "        operation()\n",
    "    except ValueError:\n",
    "        print(f\"Refused: {label}\")\n",
    "    else:\n",
    "        raise AssertionError(f\"Accepted corrupted evidence: {label}\")\n",
    "\n",
    "\n",
    "must_refuse(\"altered report bytes\", lambda: decode_report(report_bytes + b\" \"))\n",
    "changed = deepcopy(report)\n",
    "changed[\"evaluation_trials\"].pop()\n",
    "must_refuse(\"missing trial\", lambda: reproduce(changed))\n",
    "changed = deepcopy(report)\n",
    "changed[\"evaluation_trials\"][0] = deepcopy(changed[\"evaluation_trials\"][1])\n",
    "must_refuse(\"duplicate trial\", lambda: reproduce(changed))\n",
    "changed = deepcopy(report)\n",
    "changed[\"evaluation_trials\"][0][\"case\"][\"seed\"] = 999999\n",
    "changed[\"evaluation_trials\"][0][\"case\"][\"detector_seed\"] = 1099999\n",
    "must_refuse(\"undeclared seed\", lambda: reproduce(changed))\n",
    "changed = deepcopy(report)\n",
    "changed[\"evaluation_trials\"][0][\"post_burn\"][\"wfe_nm\"] += 0.1\n",
    "must_refuse(\"changed per-trial metric\", lambda: reproduce(changed))\n",
    "changed = deepcopy(report)\n",
    "changed[\"comparisons\"][0] = deepcopy(changed[\"comparisons\"][1])\n",
    "must_refuse(\"duplicate comparison\", lambda: reproduce(changed))\n",
    "changed = deepcopy(report)\n",
    "changed[\"primary_pairwise_family\"] = 12\n",
    "must_refuse(\"smaller correction selected after seeing results\", lambda: reproduce(changed))\n",
    "\n",
    "# A favorable interval alone is insufficient. The intervention must also pass\n",
    "# the frozen per-trial usefulness and actuator-saturation checks.\n",
    "changed = deepcopy(report)\n",
    "for trial in changed[\"evaluation_trials\"]:\n",
    "    case = trial[\"case\"]\n",
    "    if (case[\"candidate\"][\"controller\"] == \"predictive\"\n",
    "            and (case[\"photons\"], case[\"wind_scale\"], case[\"fault\"]) == condition\n",
    "            and case[\"seed\"] == SEEDS[0]):\n",
    "        trial[\"post_burn\"][\"saturation_fraction\"] = 0.02\n",
    "        break\n",
    "ineligible_trials = index_trials(changed)\n",
    "ineligible = paired_summary(ineligible_trials, \"integrator\", \"predictive\", condition)\n",
    "if ineligible[\"low_nm\"] <= 0 or ineligible[\"winner\"] is not None:\n",
    "    raise AssertionError(\"The favorable interval must not override failed eligibility.\")\n",
    "must_refuse(\"retained winner with newly failed saturation eligibility\", lambda: reproduce(changed))\n",
    "print(\"All eight evidence-corruption checks passed.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "scope-and-sources",
   "metadata": {},
   "source": [
    "## What this establishes\n",
    "\n",
    "The notebook independently reduces the **published per-trial WFE metrics** and checks their statistical consistency with the published comparisons. It does not establish physical fidelity of the simulated wavefront sensor, independently generate the atmosphere, or measure hardware performance. A simulated 1 ms time step is not a measured 1 ms control deadline.\n",
    "\n",
    "The full raw release contains sensor, phase, command and training arrays. Reconstructing WFE from those arrays is a different and larger exercise. The retained physical audit checks phase variance, coherent pupil Strehl, command delay and modal bounds using a separate NumPy reduction, while sharing the saved basis and atmospheric streams. These checks should not be described as on-sky validation.\n",
    "\n",
    "- [Article: Prediction Boundary](https://www.noether.one/research/prediction-boundary)\n",
    "- [Published report and all comparisons, fixed revision](https://github.com/noetherlabs/noether-ao/tree/3be53dfc1f1340d0c120c7bd5d600ea6b80f48f0/results/controller_study_v1)\n",
    "- [Frozen study protocol](https://github.com/noetherlabs/noether-ao/blob/3be53dfc1f1340d0c120c7bd5d600ea6b80f48f0/docs/controller_study_v1.md)\n",
    "- [Physical reconstruction audit](https://github.com/noetherlabs/noether-ao/blob/3be53dfc1f1340d0c120c7bd5d600ea6b80f48f0/results/controller_study_v1/physical-audit.json)\n",
    "- [Raw dataset, pinned revision](https://huggingface.co/datasets/noetherlabs/prediction-boundary-v1/tree/73234e5a6a1374568179562aab76f9f6caa9354e)\n",
    "\n",
    "**Citation:** Noether Labs. *Prediction Boundary: Where prediction helps in a delayed optical loop*. 2026. https://www.noether.one/research/prediction-boundary\n",
    "\n",
    "The two figures are available as notebook variables, <code>figure</code> and <code>seed_figure</code>. To save either in a chosen directory, use Matplotlib's <code>savefig</code> with .svg, .png or .pdf; <code>dpi=150</code> preserves the intended 1800 × 1100 PNG dimensions. Exporting is optional and does not change the calculation.\n",
    "\n",
    "The raw dataset is distributed under CC-BY-4.0. Project-owned Noether AO code is Apache-2.0. The completed evaluation cohort is public and exposed; it should not be reused as an untouched test set for a subsequently tuned method."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "name": "prediction-boundary.ipynb",
   "provenance": []
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
