What has changed, and why
Verdicts change for stated reasons, never quietly. This is the working record, newest first: every engine and rules version, the distribution before and after, the anchors that moved. It is written for the people who build the engine and left unedited here, jargon included, because a tidied summary would be a second document a reader could not check against the first.
40 entries. Live now: engine 5.2.0 · rules 0.5.1 · evidence table ev-2026.10-r231 · computed October 13, 2026.
The public, append-only record of decisions and versioned artifacts for the v4.3
assessment engine cutover. Grounded in scoring-cutover-plan.md,
scoring-methodology-v4.3.md, and assessment-schema.md. Newest entries first.
2026-10-09 — Engine 5.2.0: three dots need a trial; one scale, one sentence, one receipt
Trigger: the scoring & presentation audit of 9 October
(documents/reports/Scoring-Audit-2026-10.pdf), written for the critical reader. Eight findings,
all in the data: the dots did not discriminate, "trial-level" was printed over rows with no trial,
layer 2 contradicted layer 1 on mechanism-based standings, and the proof trail was three clicks deep
per concern. Engine 5.1.1 → 5.2.0; rules unchanged (0.5.1); table unchanged (ev-2026.09-r228).
strength()— "Moderate" requires a trial. The second door (best_source_tier <= 2with no RCT) is gone; a systematic review or meta-analysis counts as trial evidence, a tier-2 source alone does not. A verifiedestablishedstanding on aguideline-namedbasis now reads "Strong" (it is the same factguideline_endorsement: first-linerecords). Distribution of dots, 45,446 verdicts: moderate 25,304 → 11,559; limited 1,178 → 14,922; strong 245 → 246; none 18,686 unchanged. R9p (Promising) needs moderate dots, so 3,198 verdicts moved Promising → Unproven — exactly the rows whose sentence had said "trial-level evidence" with rct_count 0 or null. Verdict counts: promising 7,916 → 4,718; unproven 1,112 → 4,310; every other word unchanged. Sentences containing "trial" over a row with no trial: 3,198 → 0.- Carrier basis on the record (
q1_basis):caseForIt()is basis-aware — "a reference standard by mechanism, not a trial result" where that is the standing's only basis (7,611 of 9,182 R5 Good choices), "approved or monograph active", "named first-line in a current guideline", or "trial-supported". Layer 2 no longer contradicts the chip sentence. - Dots label beside a verdict that never rested on a row: strength
noneon a non-R11 verdict reads "Not from a trial row" (8,654 rows: Supporting role, Gentle choice, Full protection, Avoid) instead of "Not yet assessed". - Promising → the early-evidence tone (
verdict-tone.js): Promising and Unproven share one step of the published scale; the word and the dots carry the difference. - The scale, published (
VERDICT_SCALE,VerdictScale.astro): six steps — Recommended · Good · Supporting · Early evidence · Not for this · Avoid — one tone each, every verdict word on exactly one step. Legend on the verdict block, on concern listings, in full on /methodology. - The verdict block: a closed row is chip · concern · sentence · receipt line
(
receiptParts(): the standing document with the date it was opened, the declared dose against the studied range, the date the row was checked, the rule and how many products share the reading). Dots, tolerability and reader observations moved into the open state. The inline "Show the work" drawer is gone; every row links to the receipt page. "250 reviews collected" (a collector cap printed as a count on 711 rows) reads "250+ reviews read for this product (not concern-specific)". - The receipt page
/products/<slug>/receipt/(noindex, not in the sitemap): for every concern the rule in words and as the predicate, the three answers with their documents, the carrier, dose and studied range, every study with its opened date, the reader sample, the date the verdict last changed, and whether the pair is a published anchor and passes. - Concern listings: the shelf on the scale — a distribution bar and counts at the top of
every /conditions/<slug>/ page, a line stating the order inside a step (dots, dose, tolerability,
alphabet; no number), and "what the words mean here". "No.N" removed on concern listings
(
batchedGrid({ noRank })): the facets tie in groups of hundreds (dehydration: 890 products on one tuple) and a number claimed a difference the evidence does not hold. - /methodology: rules in words (
src/lib/rule-words.js—whenInWords()from the same predicate the engine runs) with the predicate beneath and the number of verdicts each rule decided; the scale in full; the confidence-band section removed (no v5 page rendered one); the hard gates stated honestly (two fire from the label, four have a public, empty register); the panel sentence replaced by the truth (editorial decisions with rationale, no panel yet). - Four public files:
/methodology/evidence/(all 228 rows, standing document, counted trials, studied dose, every source with its opened date; nobody named),/methodology/queue/(11,150 ingredient–concern pairs with no row, by products decided; per concern),/methodology/anchors/(57 anchors checked live against the build),/methodology/changelog/(this file, rendered). - Not done, deliberately:
guideline_endorsementon the 38 first-line-capable rows that lack it is research (read the guideline on the row, record it) and goes through the evidence pipeline, never typed; the merge of Promising and Unproven into one word waits until the dots have carried the difference for a release; job-style verdicts for cosmetic concerns (dullness, dehydration, winter skin, dark circles, rough texture, sensitivity) are a rules change with their own anchors.
2026-09-08 — Community Stage 1: "Did it work for you?"
Trigger: the audit's community section (§5), item 16 of its list; the first user-generated
signal on the site. Design note: help/community-stage1.md. Contract: CONTRACTS.md §7b.
- The record: one structured answer per (product, concern, voter) — improved / no change / worse / too early — plus an optional duration band and the profile's skin type; nothing else leaves the browser. No free text, so nothing to moderate. Re-votes overwrite.
- Server:
da-private/community.db(separate file, self-initialising),POST /api/community/outcome.php(same-origin, 60/h per IP) andGET /api/community/outcomes.php. The aggregate returns counts only at n ≥ 25, enforced server-side; below it, the count so far. Delete purges account rows; export ships them (communityfield). - Client:
src/scripts/outcome-vote.js(mountVote,fetchAggregate,aggregateText);da_outcomesholds the reader's own answers (identityid, registered in da-syncUNION_ID); the device tokendac_voteris not ada_*key by design. - Rendered: under each concern verdict on the reader's own concerns, the three-tap widget
(VerdictBlock
.vc__vote); beside every concern verdict, for everyone, "27 readers with acne: 78 % improved · 22 % no change · 0 % worse · 0 % too early to say" once the threshold is met (.vc__ugc). The line sits beside the verdict and the dots and is blended into neither. This pass also fixed the "In your profile" chip on verdict rows, which had never been toggled. - Verified end to end against the LOCAL API only (
mobile-audit/community-vote-check.mjs): widget mounts on the two profile concerns, vote saves, duration follow-up saves,da_outcomeswritten, aggregate updates in place and survives a reload. Local test rows deleted afterwards. - Not yet: "Helped me" on answers, the unanswered-question box, corrections with a field — the same shape (structured record, device or account key, threshold before display).
2026-09-08 — Best-of membership and stored order follow the verdicts
Trigger: the open decision from 6 Sep, taken by the user.
rerank-best-of.pynow readsverdicts.json: condition lists order by the concern's verdict facets (verdict → evidence → form → tolerability → observations) with the stored order as the tie-break — the same comparator the best-of page and the condition pages already use, so the stored order no longer disagrees with the rendered one. General lists order by the category-job verdict, then tolerability, then the product's primary verdict rank.ranking.byisconcern-verdict/job-verdict, stamped with the verdict engine and rules versions.- Membership floor in the verdict vocabulary. A member leaves a condition list only on a RESEARCHED negative verdict for the concern (Not for this, Avoid, Falls short) — 70 members across 210 lists (comedogenic products on the large-pores lists, rinse-off "Not for this" on the acne-scarring lists). "Not yet assessed" members stay, at the bottom: a first pass that dropped everything below Unproven removed 478 members for being unresearched, which is the queue punishing products — reverted from the backup before it was committed.
- The 4.8.0 grade path stays as the fallback when no verdict build exists. Two lists are under five members (oils for anti-aging: 3; snail mucin for acne scars: 4 — three members read Not for this), and both already carry the honest title.
2026-09-06 — Best-of: the honest-title rule (32 lists reframed)
Trigger: audit finding 9 and the user's decision (5 Sep) on the best-of question. The audit's "cap the lists" lead was false — every list is already capped at 40 members; the 1,088 figure was the candidate count. The real finding: condition lists whose members are all Supporting role, and lists in categories with no job verdict, carried a "Best" claim the verdicts do not support.
- Rule (in
rerank-best-of.py, so it re-applies on every rerank): a "Best X for Y" list with no member at Strong / Good / Full for Y becomes "X That Suit Y" (category lists) or "X for Y: What the Verdicts Say" (ingredient and other lists); any list in an unmeasurable category (masks, body care, lip care, oils) becomes "X: What the Verdicts Say". The authored title is kept intitle_authoredand restored automatically when the rule stops applying. Averdict_noterenders under the H1 saying why (.bf-vnoteinbest/[slug].astro). - Applied: 32 of 210 lists — 13 category-condition lists (Cleansers That Suit Rosacea, Serums That Suit Dullness, Eye Creams That Suit Dark Circles …), 13 in unmeasurable categories (every mask, body-moisturizer, lip-care and oil list), 6 ingredient lists (Vitamin C for Dullness, Snail Mucin for Acne Scars …). Slugs and URLs unchanged; the list order is unchanged.
- Open, for a decision: the rerank still orders condition lists by the 4.8.0 condition grade
(
ranking.by: condition-grade, floor B−), a hidden number the site no longer shows. Ordering by the verdict facets — the same comparator the condition pages use — would make a list agree with the concern page it links to. It reorders 210 pages, so it is a decision, not a fix.
2026-09-05 — Find: federated search and the Home · Find · My Skin · Learn · Tools tab bar
Trigger: audit finding 8 (Ask searched answers only; Learn had no home on the phone).
/search-index.json— a slim federated index (1.7 MB raw, fetched on first search intent): every published product with its card verdict (v/vl/vc), the 18 approved therapy guides, the 41 tools, and the live blog titles. Answers keep/questions-index.json.searchAll()inanswer-search.js— the same parseQuery and synonym expansion, AND across tokens, grouped results: Products (ranked by verdict, then match), Answers, Therapy guides, Tools, Articles. A product's category and verdict concern are searchable, so "rosacea" lists the Strong and Good choices for rosacea first, then the answers, the guide and the articles.- The overlay (
AnswerSearch.astro) renders the groups with a header and an "all …" link per group, and its empty state carries the seven browse destinations the old Browse sheet held. - Tab bar: Home · Find (opens the overlay; lit on catalogue pages) · My Skin · Learn (new sheet: therapy guides, questions, blog, ingredients, how verdicts are decided, site guide; lit on those routes) · Tools. The Browse sheet stays for the hamburger's handoff row. The header search trigger's label is "Search DermApproved".
- Headless check:
mobile-audit/find-tab-check.mjs(390 px, touch) — five tabs, Learn sheet opens, "cerave moisturizing" → products with verdict chips, "accutane" → isotretinoin answers, "retinoid ramp" → the tool, "rosacea" → products, answers, guide, articles. No page errors. - Tools registry hint "Answer three questions, get your file" → "Set up your profile, get your file".
2026-09-05 — Product pages link into the questions library; listings speak in verdicts
Trigger: audit Next items 11 and 13.
- Related answers on every product page (
/products/<slug>/, Related section): up to three featured answers from the questions library for the page's concerns — two from the primary concern, one from the next library (a second concern, or the category's topic library: sunscreen, retinoids, exfoliation) — rotated by a hash of the product slug so pages do not share the same three. Red-flag answers are excluded. No therapy-guide link was added: the Aug 2026 clinical-review rule that a medical-escalation prompt never sits on a purchase page stands; the audit's "How {concern} is treated →" suggestion is therefore not implemented on product pages (the condition page carries it). - Listing headers: "showing top 431 by score" → "592 are a Strong or Good choice; showing the 431 with the strongest verdicts" on concern pages (count computed per concern at build), "showing the N with the strongest verdicts" on category, skin-type and brand pages; "recommended products" → "products"; the /products sort option "Top Rated" → "Strongest verdicts first"; the homepage search hint "sorted by score" → "strongest verdicts first". Best-of lists are unchanged (the rename/cap decision is editorial).
- The "Looks compatible" pregnancy / isotretinoin / shelf-active lines the audit asked for already
existed in
matchProduct(); nothing to add.
2026-09-05 — Engine 5.1.1: verdict history, the brand-rep line, guides read the profile, glossary sweep
Trigger: the last Now item and the first two Next items of the 4 Sep audit.
formula/verdict-history.json(committed, ~2 MB): per (product, concern) the verdict key and the date it last changed, maintained bybuild-verdicts.py. Every concern verdict now carriessince. Seeded 2026-09-05 with the build date; from the next change onward a row reads "Unchanged since …".- Brand-rep line under every verdict (
.vc__same, layer 2): "“Promising” is the verdict on 6,741 of 38,141 concern verdicts across the catalog; rule R9p decided 6,741 of them. As of 5 September 2026, engine 5.1.1." plus a link "Brand or formulator? Disclose a concentration or correct a fact →" to/feedback/?type=disclosure&page=…&concern=…, which prefills the feedback type, the related page and a message skeleton (what the page says · what is wrong · a source we can check). The free-text form is the corrections channel until Community Stage 1 gives it a field. - Therapy guides read the My Skin profile (
/treatments/<condition>/, audit finding 7 — the layer read zero fields): (1) intake prefill — pregnancy, under-18, reactive skin, isotretinoin, the actives in the profile, and "no cycle" for a male profile — only for options the pathway's own questions offer, only when nothing is stored yet, named on screen with an edit link, and never auto-advanced; (2) special situations that apply (pregnancy, age, deeper skin tones) open first with an "Applies to you" chip; (3) pathway routine lines whose active is in the profile carry "you use this" (data-pick-activeonOtcPathway.astro). Verified headless with a pregnant, reactive, adapalene + BPO profile on the acne guide (mobile-audit/guide-profile-check.mjs). - Glossary sweep — "graded / grade / well-graded / Graded for you / four questions" replaced by
verdict vocabulary on the homepage, /my-skin, feedback, account, settings and disclaimer copy; the
gate (
check-glossary.py) now carries the retired-vocabulary rules. PASS.
2026-09-05 — Rules 0.5.1: Excellent needs four measured features; the reconciling sentence; reading rows ordered once
Trigger: the open item from 5.1.0 and items 2–3 of the audit's Now list.
- J1
job_measured_min: 4. Three binary checks (UVA filter, photostability, tolerability) read "Excellent at the job" on 95 % of sunscreens. Categories with fewer than four measurable features now cap at Does the job. Job verdicts: Excellent 346 · Does the job 4,121 · Falls short 518 (was 1,695 / 2,772 / 518). Excellent is now a moisturizer or a cleanser with a disclosed pH. Anchors 57/57 PASS. - Reconciling sentence (
VerdictBlock.astro,.vb__recon): rendered when the review's verdict or pros assert efficacy (clinical / proven / studies / evidence / results …), no concern reaches Strong choice or Full protection, and at least one row is Promising, Unproven or Not for this. 1,647 products. Copy: the review was written on the formulation and the published trials; the verdict applies the stricter standing test; where they differ the verdict is the site's position. orderReading()insrc/lib/audience.js— one ordering for the homepage, blog hub and /my-skin reading rows: age and gender filters (fail-open as before), gender-neutral pieces first when the profile has not stated a gender, band affinity, then one category per row. The three surfaces' hand-rolled filter-and-sort blocks are replaced by the call. The/for-youbuild pool keeps recency but caps any category at two of the eight base rows and gender-marked posts at two. Verified headless with a genderless acne + hyperpigmentation profile: no men's-grooming card on any of the three rows; four distinct categories in the homepage row.rerank-best-of.py --writeandcompare-engines.pyrerun (35.5 % ≥ 2-step disagreement, unchanged).
2026-09-04 — Verdicts 5.1.0: the page stops arguing with itself (engine 5.1.0, rules 0.5.0, anchors 0.2.1)
Trigger: the 4 Sep flow, scoring-credibility, personalization and community audit
(documents/reports/Flow-Scoring-Community-Audit-2026-09.md). Its first six findings were one
problem: the verdict vocabulary was stricter than the site's own evidence dots, its own job line and
its own retired letters, and the product page showed all of them at once.
Rules (0.5.0).
- R9p
Promising. Q1 plausible + Q2 adequate/plausible + no supportive job + carrier dots moderate or strong. Before: "UNPROVEN · ●●●○ Moderate evidence" on 7,018 rows. After the build: 6,741 Promising, 940 Unproven. Same rank (2); the dots order them. Anchors: every anchor that acceptedunprovennow acceptspromisingtoo (8 entries) — no anchor's judgement moved. - R9 closing clause reads the other heroes: "nothing else here addresses it" only when every
other hero has a researched
nonerow; "the other actives here are not yet assessed for it" when that is the truth. C E Ferulic no longer claims nothing else addresses sun damage. - J1 needs ≥ 85 over at least three measured features and no longer says "unusually well"; J3 names the criterion it missed ("comes up short on: seals moisture in") instead of "its basics".
Engine (5.1.0).
- Carrier tie-break: standing → form → INCI position → declared % → slug. 5.0.x fell to the slug (alphabetical), so ferulic acid 0.5 % carried C E Ferulic; vitamin C does now.
- Job verdict over measured features only. Unmeasured features entered at a prior of 60, which
parked whole categories in the 60–65 band: toners fell short 100 %, body care 96 %, oils 78 %. Now
unmeasured features are excluded and named; under 50 % measurable weight ⇒ no job verdict
(
job: null,job_unmeasuredbeside it): body care 493, masks 623, cleansers with no surfactant detected 337, lip care 170, oils 131. Tonerpurpose_substancewas also looking up an evidence row againstcond=Noneand could never succeed — it now reads the product's own listed concerns. The oilfatty_acid_profileproxy (the comedogenic flag counted twice) is disclosed as unmeasured. Job verdicts now: Excellent 1,695 · Does the job 2,772 · Falls short 518 (was 1,060 / 3,300 / 2,379). Open: sunscreens still read Excellent on 95 % (three binary checks all pass) — the label overstates a checklist; candidates are a fourth measured feature or a "Does the job" cap for ≤ 3 features. vdpayloads (/my-skin-index.json, the product-page match record) carry[verdict, rank, strength_rank]; readers of[0]/[1]are unaffected.
Surfaces.
- Verdict block: the job line renders below the concern verdicts;
Not yet assessedrows collapse to one sentence under the grid ("Also listed for … — in our research queue; a gap in our table, not a mark against the product"); a product with only queued rows shows that line alone. - Cards / matcher:
not-yet-assessednever becomes a personal chip;promisingis a gap-kind reason (no weight), likeunproven. - Product hero stamp: no "A+ · Top grade" fallback on a verdict build.
- My Skin shortlist and shelf cards rank and label by verdict (
vd), never by letter. - Methodology: Verdicts section first; the two scores described as the engine underneath.
- Product page "Community — What People Say" → "Reviews — What Reviewers Tend to Say", editor summaries no longer in quotation marks.
Gates: check-verdict-anchors.py --strict 57/57 PASS · compare-engines.py 35.5 % ≥ 2-step
disagreement with 4.8.0 (R9p now the second largest source, as intended) · rerank-best-of.py --write
run · check-glossary.py PASS.
2026-09-02 — Verdicts 5.0.1: cutover audit, spec drift closed (engine 5.0.1, rules 0.4.0, table ev-2026.09-r228)
Trigger: an audit of the cutover against VERDICTS.md and MASTER-PLAN §3 the same day. The gates passed; the claim "no 0–100 score and no letter renders anywhere" did not hold, and three engine derivations differed from the published spec.
Engine (5.0.1).
- Evidence dots now follow §3.6 exactly. 5.0.0 granted four dots to any row at tier
well-established/established-mechanism; 7,752 of its 7,952 four-dot marks came from that shortcut and 200 from the published rule. Now:strongonly onguideline_endorsementstrong / first-line or a high-certainty, independently replicated meta-analysis. Result: 200 verdicts at four dots from 11 rows (adapalene, BPO and tretinoin for acne; tretinoin for aging; azelaic acid for rosacea; tranexamic acid for melasma; coal tar for psoriasis; ketoconazole, selenium sulfide, zinc pyrithione and salicylic acid for dandruff); 21,194 at three. Strong choice with three dots is now the common case (1,128 of 1,194) — the sulfur rule, applied. Follow-up for the research pipeline, not the engine: 27 first-line rows carryguideline_endorsement: nonewhere a guideline plainly exists (hydroquinone|melasma, petrolatum|eczema, the M020 filters for sun damage) — seeformula/dose-research-queue.txt. - Q1 source floor (§3.9). A verified
established/supportedstanding needs a source at tier 3 or better, the no-standing tier path needs tier ≤ 2 (the duplicate unguarded branch that made the check dead is gone). Zero verdicts moved on the current table. The trial ladder that lifts a verifiedemergingstanding tosupportedon ≥ 2 consistent tier-≤2 RCTs is kept and now documented in VERDICTS.md §4 — it is what the niacinamide|hyperpigmentation and retinol|aging anchors assert. - SPF is parsed from the whole record (name, size, keywords, alt, badge, editorial), and an
unstated SPF on the
sunscreencategory is read as 30 with the assumption stated in the sentence (VERDICTS §7, which 5.0.0 did not implement). Basic protection 297 → 147; Full protection 1,824 → 1,974; the 77 sunscreens that read "Basic protection … at SPF not stated" (COSRX Airy-Light, Krave Beet Shield, Anua) now read their real SPF. P4 now fires on 94 pairs: 90 SPF moisturizers with no figure anywhere and four SPF 15–29 sunscreens. - Filter names. The
{filters}slot is built from the INCI facts (UVA list + minerals) with reader-facing names; the "with its filters" fallback that shipped on 791 mineral-sunscreen verdicts is gone (0). Remaining fallback text: "a minor flag" on 29 sensitivity verdicts. meta.shadowis false and the version is 5.0.1; the sidecar is the live one.
Surfaces. Client payloads now carry the verdict beside the letter so no client re-render paints
a letter next to a server-rendered verdict: cardRecord (Show more, the profile-hidden grid), the
product-page match record (vd), the browsing-concern strip (verdict chip, verdict order), and the
routine index (v/vd). Routine pages show the verdict pill instead of NN/100 and the letter;
brand pages show Strong/Good choice counts instead of "Avg score NN/100"; best-of lists are
ordered at build time from verdicts.json (condition lists by the concern's facets, general lists
by the category-job verdict, pins on top, stored order as the tie-break) and say so; the editors'
pick chip and the for-you launches carry the verdict; the grade-ceiling strip renders nothing on a
verdict build; check-highlights.py checks verdict agreement. <body data-da-verdicts> marks a
verdict build for client code.
Rule and vocabulary decisions now on the record (MASTER-PLAN §3.3 / §11 amended): R3 reads
Strong choice for a first-line active at a plausible, undeclared dose (479 pairs) where draft 2 had
Good choice; Unproven, Gentle choice and Basic protection are verdict words; R2r caps
rinse-off actives at Good choice, replacing draft 2's "no ceiling". All editorial, pending the panel.
Gates: anchors v2 57/57 strict PASS · highlights PASS · glossary PASS · routine-engine test PASS · compare-engines 35.5% ≥ 2 steps (unchanged).
Still open after this pass (audit findings not yet closed): the compare page's "Overall" 0–100
row; /100 and letter chips in dupe-finder, sunscreen-finder and pregnancy-safe; letters in the
/my-skin picks, shelf cards and the homepage strip; ungated 4.8.0 copy on /methodology
("Overall Score (0–100)", the Confidence section) and /system ("letter grade per skin concern");
category / skin-type / brand / ingredient grids still ordered by the hidden 4.8.0 overall; the value
tier as sixth facet and the "facet in play" display on listings; a verdict-sentence gate; layer-3
traces; the six 4.8.0 anchor waivers that expired at cutover and still pass check-anchors.py.
2026-09-02 — Verdicts v5 CUTOVER (engine 5.0.0-shadow.1, rules 0.4.0, table ev-2026.09-r228)
What changed for a reader. Every product page, card, listing, the My Skin matcher and compare
now render a verdict per concern — Strong choice · Good choice · Supporting role · Unproven ·
Not for this · Not yet assessed · Avoid (sunscreens: Full / Basic protection / Falls short;
sensitivity: Good choice / Supporting role / Avoid from tolerability alone) — read off the published
rule tree in formula/verdict-rules.json, with the rule named on the page, the three questions
answered in "The case for it", and the studies under "Show the work". Evidence strength (four dots)
and collected reviews sit beside the verdict and are never blended in. No 0–100 score and no letter
renders anywhere. /methodology#verdicts lists every rule with its rationale; /system and
/disclaimer describe the verdict as an assessment of ingredient fit, never a treatment claim.
Distribution at cutover (38,141 product-concern verdicts): not-yet-assessed 8,348 · good-choice 7,957 · unproven 7,681 · supporting-role 7,003 · full-protection 1,824 · avoid 1,692 · gentle-choice 1,681 · strong-choice 1,194 · not-for-this 386 · basic-protection 297 · falls-short 78. Against 4.8.0 letters, 35.4% of pairs differ by two steps or more, dominated by B-tier products whose actives have no evidence row ("Not yet assessed") or early-evidence rows ("Unproven") — the prior inversion made visible rather than a scoring disagreement.
Decisions taken by editorial judgement (Option 1), pending an advisory panel: the 16 open anchor
questions are resolved in formula/anchor-verdicts.json with a written rationale each — rinse-off
actives cap at Good choice except medicated shampoos for scalp concerns (rule R2r); 10% azelaic
acid reads Good choice for rosacea because the approved presentation is 15% (dose authored from the
row's FINACEA source); KP keratolytics read Supporting role with the active named (R8a) because KP
has no guideline and the rows rest on single trials; vitamin C stays Unproven for hyperpigmentation
until its row is re-researched. Ketoconazole 1% OTC dose authored from a DailyMed label opened
2 Sep (Nizoral-class shampoo → Strong choice). Anchors v2: 57 pass, 0 waived, 0 fail (strict).
Rollback. VERDICT_ENGINE=0 restores the 4.8.0 render exactly; assessments.json keeps
building for one release. Gates after any engine or table change: npm run verdicts,
npm run verdicts:anchors (strict), npm run verdicts:compare.
2026-09-02 — Verdicts v5 shadow.1, and a drying-alcohol flag fix in 4.8.0 (MASTER-PLAN Phase 1)
Table v2 (additive). migrate-evidence-table-v2.py added dose (min/max/unit,
requires_leave_on, rinse_off_ok, onset_weeks, source, needs_research) and verification
(verified_against_sources, verified_on, source_tier_min) to all 228 rows; 187 doses parsed
from the existing prose, 38 clinical rows queued in formula/dose-research-queue.txt. No
existing field renamed or removed. Spec: VERDICTS.md §3.
Rule register 0.3.0 (formula/verdict-rules.json): the three-question tree (F1–F3 for
fit-only claims, R1–R12 for concerns, P1–P5 for photoprotection, J1–J3 for category jobs),
a supportive-roles table, an explicit trigger table per concern (Q3 is "harsh" only when
a listed trigger leaf fires at ≥ min_points, top-half position for alcohol; Tier 1–2 seals
veto harsh), caution-only leaves (photosensitisation, aggressive actives, pregnancy flags,
pH, preservation, allergen count — they add a caution line, never an avoid), and a sixth
verdict word "Unproven" (rank 2) for plausible-tier actives without a supportive job —
the first shadow run had put a 10% niacinamide serum at "Not for this" for
hyperpigmentation. SPF 15–29 broad-spectrum reads "Basic protection" (the FDA
early-skin-aging bar), not "falls short".
build-verdicts.py writes src/data/verdicts.json (shadow; nothing renders): 9,398
products, 38,141 (product, concern) verdicts. Reuses the 4.8.0 engine for INCI facts,
deductions, dose proxy, craft and the category job so Q3 is identical; adds Q1/Q2, the tree,
evidence dots and observations. Distribution: good choice 7,585 · unproven 7,681 · not yet
assessed 8,350 · supporting role 7,001 · avoid 1,692 · full protection 1,824 · gentle
choice (sensitivity) 1,681 · strong choice 1,566 · not for this 386 · basic protection 297 ·
falls short 78. compare-engines.py → formula/verdicts-vs-4.8.0.md: 35.5% of pairs
differ from the 4.8.0 letter tier by ≥ 2 steps, dominated by B-tier products whose heroes
have no row (now "not yet assessed", 6,270) or plausible-tier rows ("unproven", 3,922) —
the prior inversion made visible, not a scoring disagreement. Panel review is the next gate.
4.8.0 fix found by the shadow run: inci-flag-rules.json drying_alcohol matched
"benzyl alcohol" (a preservative) — the exclusion list held fatty alcohols only. Twenty-one
exclusions added (benzyl, phenethyl, isocetyl, isostearyl, batyl, caprylyl, lanolin,
C-number fatty alcohol blends, the aromatic fragrance alcohols the fragrance leaf already
counts). Rebuilt: 1,422 grades rose, 1,006 changed letter, none fell (Aveeno Eczema
Therapy Rescue Balm eczema A → A+; its balm had been losing fit for its preservative).
Anchors strict PASS · codes:check PASS · best-of reranked (71 lists).
2026-09-02 — Every reason code speaks English; a gate keeps it so (MASTER-PLAN Phase 0, item 5)
Trigger: the assessment (p. 10): the reasons that decide most A grades — standing, the clinical outcomes floor, first-line tempering, certifications, stratified outcomes — had been added after the humaniser was written and fell to its de-hyphenation fallback on the product page: "Aging — outcomes prior clinical floor — tretinoin.", "Eczema — certification verified premium — +2." Measured before the change: 108 of 146 live code shapes, 25,398 occurrences across the grade cards.
Change: humanizeReasonCode() gains a sentence family for each live shape
(evidence-best-supported, standing-<value>:<basis>-for-<cond> with the
basis spelled out, outcomes-prior-clinical-floor, first-line-fit-tempering,
rinse-off-job, certainty-<level>, the four certification* codes with seal
names, stratified-outcomes:NNpct-improve-nN, the two INCI-guard codes from
item 2, and the dormant category-job* pair). humanizeReasonCodeStrict()
returns null on the fallback, and scripts/check-reason-codes.mjs
(npm run codes:check) fails when any code in assessments.json would hit
it. Result on the current sidecar: 93,493 codes, 0 unmatched. Run it after
every build-assessments.py.
2026-09-02 — Best-of lists reranked on engine 4.8.0 (MASTER-PLAN Phase 0, item 4)
Trigger: every one of the 210 lists carried an order baked from engine 4.3.1 on 2026-06-16 while the engine moved four versions; 7.9% of member pairs were inverted against the current grades and 36 members sat below the B− floor the page copy promised. The page also described the ranking in the four retired editorial pillars ("by ingredient quality, value, and safety").
Change: rerank-best-of.py --write run against the 4.8.0 sidecar (after
the sunscreen-filter and derivative-slug fixes of 2026-09-01): 173 lists
reordered, 37 unchanged, 38 members dropped below B− on condition lists, none
gutted (best-oil-for-aging was already a three-member list). The script now
stamps ranking.engine from the sidecar instead of a literal, and moves
last_updated only when the order or membership actually changed — 173 lists
now read "Updated September 2026", 37 keep June. /best/[slug] states its real
basis: "ranked by each product's acne grade, nothing below B−" on condition
lists, "ranked by our overall assessment of formula, safety and value" on
general ones; the meta description follows.
Not changed: list membership is still selected by gen-best-of-candidates.py
on the legacy overall — a v5 item. Rerank is not yet part of the build; run it
after every engine or table change until build-verdicts.py orders lists at
build time (MASTER-PLAN §3.11).
2026-09-01 — Derivative slugs and the INCI standing guard (MASTER-PLAN Phase 0, item 2)
Trigger: the September assessment (p. 12): hero names were substring-mapped
onto approved actives and inherited their regulatory standing. Naturium Azelaic
Acid Derivative Complex 10% (potassium azeloyl diglycinate) was #1 for rosacea
site-wide at 96.0, above The Ordinary's 10% azelaic acid; Trilogy Rosehip Oil
("Trans-Retinoic Acid Precursors" → tretinoin) was A and #2 for aging above
every retinol; Sunday Riley's oleyl adapalenate read as adapalene.
Change (map + engine + gate; table unchanged):
- One slug per molecule.
scripts/ingredient-map.jsongains derivative entries ahead of their parents (first match wins):potassium-azeloyl- diglycinate,oleyl-adapalenate,kojic-dipalmitate,capryloyl- salicylic-acid,silanediol-salicylate,hydroxybutyl-trimethylammonium- salicylate, and an ordering entry sending "retinoic acid precursor" torosehip-oilbefore the tretinoin pattern can claim it.potassium azeloyl diglycinateremoved from the azelaic-acid entry. 33 heroes on 32 products re-slugged inproducts.json(backup insrc/data/backups/); the 13 PAD heroes, 13 salicylate derivatives and willow-bark heroes that older map order had left onsalicylic-acid, 3 kojic dipalmitate, 2 oleyl adapalenate, the rosehip, and one cleansing-balm hero that was never mineral oil. formula/standing-inci-names.json— the exact INCI names each of the 38 standing-carrying slugs must literally carry. Engine guard (hero_supported_by_inci, both hero loops): a hero whose slug is listed and whose product INCI (≥ 5 entries) carries none of the names is withheld from the claim with reason code<slug>:hero-not-in-inci:standing-withheld-for- <cond>. Exception, also coded: the hero is the molecule with a declared concentration ("Zinc Oxide 12.9%", "Salicylic Acid (2%)") — the US OTC label lists drug actives apart from the inactive list and eight catalogue INCIs carry only the inactive half; standing stands and the product is loggedactive-declared-not-in-inci:inci-incomplete.- Authoring gate:
check-batch.pyerrors when a hero carries a listed slug and the INCI lacks the molecule.
Effect (rebuilt 2026-09-01): 72 grades moved. Naturium derivative complex rosacea A+ 96.0 → B 71.2 (rosacea top five is now five azelaic-acid products); Trilogy rosehip aging A 86.1 → B+ 73.6; Sunday Riley adaptive retinoid acne A+ 91.9 → B+ 74.4; Geek & Gorgeous aPAD rosacea A+ → B+; iS Clinical Active Serum acne A+ → B (willow bark, not salicylic acid). One collateral drop that is a data gap, not a scoring call: SKIN1004 Hyalu-Cica Water-Fit Sun Serum sun-damage A+ → D+ because its stored INCI lists no UV filter at all. Anchors unchanged (19 pass / 6 fail); golden PASS; sanity 29/29; tier share unchanged at A 26.7%.
Follow-up: formula/inci-gaps-2026-09-01.txt — 8 "declared" products need
the Active Ingredient line appended to inci_list; 14 "withheld" products need
their real ingredient list verified (SKIN1004 sun serum first). Split tinosorb
into S and M when table v2 lands.
2026-09-01 — Table ev-2026.09-r228: sunscreen-filter parity (MASTER-PLAN Phase 0, item 1)
Trigger: the September site assessment (documents/reports/Site-Assessment-2026-09.pdf,
p. 12): no organic UV filter had an evidence row, so a chemical sunscreen's
sun-damage grade was carried by its niacinamide or vitamin E at limited = 30,
or by the neutral prior. Measured before the change, sunscreen-category products
graded for sun-damage: mineral (zinc/TiO2 hero) A-tier 455 · B 12; organic
A 7 · B 254 · C 131 · D 121 · F 3. La Roche-Posay Anthelios UV Pro Sport
SPF 50 read C+ with the reason "Niacinamide has limited evidence for sun damage."
The same product class was split by whether its filter had a row.
Change (table only; engine 4.8.0 unchanged): thirteen (filter, sun-damage)
rows added with the same shape and basis as the existing titanium-dioxide row —
standing: established, basis regulatory-active + trial, trial anchor the
Nambour/Hughes 2013 RCT (PMID 23732711, opened this session).
- US monograph actives, source 21 CFR § 352.10 via Cornell LII (opened
2026-09-01; part heading marked stayed) + § 201.327 for the early-skin-aging
claim linkage: avobenzone, octinoxate, octocrylene, homosalate, octisalate,
oxybenzone, ensulizole, and the placeholder slug
uv-filters(47 products; standing granted only because the G4 gate independently requires a UVA filter in the INCI). - EU Annex VI filters, source Regulation (EC) 1223/2009 OJ L 342 PDF via
EUR-Lex (opened 2026-09-01): tinosorb (S and M), uvinul-a-plus, ethylhexyl-
triazone, mexoryl-xl, mexoryl-sx.
mexoryl-400(3 products) left un-rowed — no source opened. - Tiers derived by the rule, not typed: avobenzone and octinoxate emerging (the Nambour formulation's own filters per the zinc-oxide row's Methods reading; no intervention-indirectness downgrade), the rest limited.
- Not opened this session and recorded as
research_todo: the M020 PDF (accessdata 404), the 2026-06-10 Federal Register bemotrizinol order (federalregister.gov unreachable from the fetcher), Drugs@FDA for ecamsule.
Effect (rebuilt 2026-09-01): organic sunscreens now A 400 · B 98 · C 7 ·
D 9 · F 2; mineral A 458 · B 9. Anthelios UV Pro Sport C+ → A+ (89.6),
carrier avobenzone. Three products fell by ≤ 4.3 points (Miguhara Daily Care
Sun Cream A− → B+) because §5 max-aggregation picks the carrier on evidence
points alone — octinoxate at emerging (88) now outranks titanium dioxide
(84) and brings its own undeclared-dose proxy (50) with it. Logged, not
patched: the v5 verdict tree (MASTER-PLAN §3) replaces max-aggregation.
Catalogue-wide A-tier share 25.7% → 26.8% — parity, not a lift; the frozen
gb-2026.07 boundaries were not touched. Anchors: unchanged at 19 pass /
6 fail (none sunscreen-related). Golden exemplar: PASS. Sanity gate: 29/29.
Follow-ups: split tinosorb into tinosorb-s / tinosorb-m and replace
the uv-filters hero on 47 products (one slug per molecule, Phase 0 item 2);
rerank best-of lists (Phase 0 item 4) — the sunscreen lists are stale against
this table until then.
2026-08-24 — Engine 4.8.0: treatment-class calibration (the acne inversion)
Trigger: dermatologist feedback + the new grade-ranked /products view made the inversion undeniable: for acne, hydrocolloid/SA spot patches graded 97.5 and cosmetic BHA serums 95–97 while Epiduo sat at 91 and adapalene 0.1% at 92 — every anchor FLOOR passed while the ordering was clinically upside-down. Three engine causes, three coupled fixes:
- Trial-tier modulation of standing evidence (
STANDING_TIER_MODULATION): the 4.4.0 standing override made every monograph active saturate at 100 — salicylic acid (tier emerging, 52 on the trial ladder) scored identical evidence to adapalene (well-established). Standing still drives the score; the row's trial tier now modulates it (well-established ×1.0 → promising ×0.95 → emerging ×0.88 → limited ×0.84 → traditional ×0.80). Exempt: rows whose basis includesmechanism-established(monograph reference standards — petrolatum/eczema, zinc-oxide/sun-damage). The exemption is not cosmetic: modulating petrolatum let a standing-less mechanism row steal §5 carrier selection and silently drop the clinical outcomes floor (Aquaphor/eczema −13.8 before the exemption; exactly 0.0 after). - Small-n shrinkage of stratified outcomes (
OUTCOMES_SHRINKAGE_K = 30): pts′ = prior + (pts − prior)·n/(n+K). A 46-review scrape at 100%-improved measured a full 100.0 and outweighed the AAD guideline by 4 grade points; credibility is now earned at n/(n+30) — the N_strat minimum buys half. - First-line-strength carrier (verified established standing on the
regulatory/guideline channel AND well-established trial tier): outcomes
prior 85 (
FIRST_LINE_OUTCOMES_PRIOR) instead of 80, and bounded fit tempering (FIRST_LINE_FIT_FORGIVENESS= +8 fit points, never past 95, treatment-classroutine_step == "treatment"only, leave-on only). The class gate and the bound are load-bearing: the first cut used an unbounded fit FLOOR and moved a fragranced azelaic moisturizer +18.2 grade points for rosacea — fragrance is not the mechanism's cost and is never forgiven.
Gate upgrades: anchor-products.json 1.2.0 adds ORDERING assertions
(above: peers an anchor must out-rank, rank-ceiling-capped) — floors catch
absolute demotion, these catch inversions. Epiduo added as an acne anchor;
adapalene + Epiduo assert above the leading patch and cosmetic BHA.
Result: acne top-20 is now BPO/azelaic/retinoid leave-on treatments;
adapalene #14, Epiduo #21 (was #121), Peace Out patches #44 (was #1). Golden
test updated to 4.8.0 semantics and PASS; anchors 19 pass / 6 fail — all six
failures are pre-existing at identical scores in the 4.7.0 baseline
(limited/emerging-tier table rows awaiting the standing hand-off:
lactic-acid/KP, retinol/aging, vitamin-C/hyperpigmentation — see
standing-handoff-4.4.md), and the four acne ordering failures the baseline
would have shown are fixed. Site-wide: 38,121 grades compared, 623 letters
changed (601 down / 22 up — deflation of over-saturated small-n grades; no
product gained more than ~5 points).
2026-08-21 — Engine 4.7.0: clinical outcomes-prior floor
Trigger: Differin Epiduo Acne Gel — adapalene 0.1% + BPO 2.5%, the
AAD-guideline first-line fixed combination — ranked 117th of 1,322 for acne.
Component trace: formula-for-job 95, fit 91.5 (both at or above the catalog's
top acne products), and outcomes 60 — the neutral prior, because the
Rx-heritage/pharmacy channel produces no scrapeable retailer reviews. The
2026-07 comment on OUTCOMES_PRIOR ("identical priors are ranking-neutral")
stopped being true the day stratified outcomes landed for part of the catalog:
products whose sales channel can't produce reviews were systematically buried
below review-rich cosmetics. This is the prior-inversion problem
(evidence-coverage memo) in its worst form — the penalty fell precisely on
the most clinically validated class.
Change: when a grade's outcomes leaf is still a prior AND the carrying
active holds a verified established standing for that condition
(regulatory-active / guideline channel — the same test row_evidence already
enforces), the outcomes prior floors at CLINICAL_OUTCOMES_PRIOR = 80
instead of 60. Rationale: for these actives, trial evidence is knowledge
about expected user outcomes — the trials measured exactly that. Calibration
(n=30 grades where established-carrier outcomes WERE measured): stratified
mean 97.3, p10 94.4. The floor sits ~14 points below the poorest
measured member of the class, so collected data always dominates and
collection still matters. Stratified grades are untouched; early and the
confidence band are untouched (the floor changes the point estimate, not the
claim of knowledge); disclosed per-grade via
source: prior:clinical-evidence-floor(<slug>) and reason code
<cond>:outcomes-prior-clinical-floor:<slug>.
Impact: 2,023 grades lifted (+4.0 grade points each: 0.2 × 20); 291 newly
≥ A−, 451 newly A+ (letter shares move ~+1.2pp against frozen gb-2026.07
boundaries). Epiduo acne 86.95 → 90.95 (A+). Displayed rank moves little
(117 → 116) by design — the floor lifts the whole established class
together, so it corrects the channel bias without reshuffling within-class
order. Monotonicity: the change can only raise grades, so no floor-type anchor
can regress; anchor gate 18 pass / 6 FAIL — all six pre-existing
(keratosis-pilaris, retinol-aging, vitamin-C-hyperpigmentation calibration
debt, unrelated to outcomes priors). Golden exemplar PASS,
check-personalization.py PASS.
Companion (display layer, same day): the Editors' choice lane on condition
pages (EditorsChoice.astro, fed by highlights.json) — where dermatologist
consensus and the engine still disagree, the editors say so above the ranked
list with their name on it, never via a silent pin inside it. The remaining
open defect is treatment-class vs support-class collision at the head of
condition lists (patches/exfoliants with immaculate INCIs outranking
guideline treatments); logged, not addressed here.
2026-08-10 — Engine 4.6.2: INCI tokenizer fixes + BPO self-preservation
Trigger: Differin Daily Deep Cleanser (5% BPO, purpose-built acne wash) read
B+ for acne while every peer medicated cleanser (PanOxyl, Effaclar, CeraVe
Acne Foaming) read A+. Tracing the 8-point gap surfaced two tokenizer defects
and one mis-specified rule — all of the "shifts hundreds of products a little"
family that audit-signals.py was built for.
1. Comma-split shredded IUPAC digit,digit names. normalize_inci split on
every comma, so 1,2-Hexanediol became the entries 1 + 2-hexanediol and no
flag pattern could ever match it. 212 products were penalized −10 for "no
preservative" while listing 1,2-hexanediol as their preservative (the entire
COSRX hexanediol-preserved line, most K-beauty). Same defect hid
2-bromo-2-nitropropane-1,3-diol from the sensitizing-preservatives rule and
inflated INCI position counts (phantom 1 entries) feeding the dose proxy.
Fix: entry separator is now a comma not followed by a digit.
2. Trailing period defeated the hydroxide exemption. A list ending
"…, Sodium Hydroxide." produced the entry sodium hydroxide., which token-matched
the harsh list but failed the exact-equality pH-adjuster exemption — the
product ate −4 as if it were true soap. Entries now strip trailing periods.
3. Benzoyl peroxide is self-preserving. 12 water-based OTC BPO drug products
(both Differin BPO washes, PanOxyl bar, Epiduo, ZAPZYT…) fired
water_based_inadequate because their drug-facts labels list no cosmetic
preservative — but a 2.5–10% oxidizing biocide preserves its own vehicle.
New self_preserving_actives list in preservation_adequacy (currently just
benzoyl peroxide) counts as adequate preservation.
Outcomes fingerprint migration. Changing normalization changes INCI
fingerprints, and the engine reads a fingerprint mismatch as "reformulated" —
a naive rebuild silently orphaned collected outcomes for products with
digit,digit INCIs. One-time migration re-stamped fingerprint_at_collection
on 280 of 541 outcomes files, only where the stored value exactly equaled
the old normalizer's fingerprint of the current INCI (provably the same
formula; 0 genuine reformulations found). Backup of the pre-migration store
kept with the session scratchpad.
Net effect (same inputs, engine-only diff): 975 grade letters up / 79 down of 38,046. Upgrades are precisely the falsely-penalized hexanediol/BPO/hydroxide sets; downgrades are small dose-proxy shifts from corrected entry counts landing on letter boundaries. Golden test PASS; anchor gate unchanged (18 pass, same 6 pre-existing failures); §3 invariants hold. Differin Daily Deep Cleanser: acne B+ 78.1 → A+ 86.5 (tolerability 82→96), overall 80→82 — the rank ceiling (79.5) still keeps it from headlining mixed acne lists above leave-on treatments, which is the designed behavior.
The 4.3.2 contact-time cap no longer touches the stored grade. A cleanser's condition grade is now its honest job-relative score — CeraVe Hydrating Cream-to-Foam reads A+ 88.4 for dryness instead of a capped B+ 77.4. This completes the demotion the 4.5.0 entry recorded as a consequence ("the cap is a legacy workaround — retire or demote it to a display safeguard"): with the category-job axis scoring cleansers on gentle-function rather than treatment, capping the grade itself punished exactly the framing the model adopted.
Why not full retirement — the no-cap rebuild test (2026-07-28). Rebuilt the full catalog with the cap neutralized: the anchor gate held identical (18/24, same 6 pre-existing KP/aging/CEF failures — the job axis alone keeps every eczema anchor cream above every cleanser), but rinse-off products took #1 on 7 of 24 mixed condition lists (a brightening cleanser topping dark-spots above tranexamic/azelaic leave-ons; cleansers #1 for hyperpigmentation, dullness, excess-oiliness, fungal-acne, KP, large-pores). Contact time still cannot headline a treatment ranking — but that is a list-ordering fact, not a grade fact.
Mechanics: every rinse-off grade now carries rank_ceiling: 79.5 (just
under the frozen gb-2026.07 A- boundary); score_pre_cap/capped_by and the
4.6.0 soft knee are gone. Ordering enforcement lives in
src/lib/assessment.js#byConditionScoreDesc: mixed lists rank by
min(score, rank_ceiling) with the raw score as first tie-break, so the
ceiling binds cross-category while strong and weak cleansers stay fully
distinguishable among themselves (the property the 4.4.2 knee existed to
protect). Product pages, card chips, and within-category cleanser rankings all
show the true grade. Invariants updated: rank_ceiling must equal the
constant, no score may exceed 100, components reproduce the stored score
directly (no pre-cap special case).
100-clamp (bug found by the test): the certification premium was added after the weighted sum with no clamp — CeraVe Baby Wash & Shampoo and CeraVe Baby Moisturizing Cream sat at a stored 103.00 for sensitivity in the live 4.6.0 sidecar. Scores now clamp to 100 before rounding; the invariant allows components to reproduce a >100 raw value only when the stored score is 100.
Results: golden PASS; anchor gate 18/24 (unchanged set); zero ordering
violations — no rinse-off product ranks above any leave-on scoring over 79.5.
Two accepted edge cases: fungal-acne and keratosis-pilaris get a rinse-off #1
because no leave-on in those lists clears 79.5 — defensible (washing is
first-line malassezia care; KP's weak leave-ons are the known open anchor
failure, tracked separately). Sidecar remains flag-gated (ASSESSMENT_ENGINE
off), so nothing user-facing changes until cutover.
2026-07-28 — Engine 4.6.0: signal audit + 7 systemic rule fixes; brand trust v2; recognition display
New instrument: scripts/audit-signals.py — read-only audit of the
engine's INPUTS (leaf saturation, per-rule ingredient hit-frequency, unknown
rates, factor-score spread). The anchor gate checks 24 outputs; this checks
the signals behind 38,000 grades. Run it after every engine change.
Rule fixes, all one family — a rule firing on the wrong ingredient, found by
owner spot-checks + the audit: (1) sodium/potassium hydroxide are trace pH
adjusters, not surfactants/irritants — they were the TOP triggers of both
cleanser_harshness and irritant_load (2,315 products); counted as soap only
with a saponifiable fatty acid now. (2) citric acid removed from
photosensitization (top trigger, 2,548 products; a buffer, not an exfoliating
AHA). (3) surfactant_mildness rescaled to the real 0–12 harsh_pts range (976
products were pinned at 15/100). (4) usability_feel finish bonus fired on
9,312/9,313 products (positive-only tag vocabulary) — folded into base.
(5) medicated-rinse-off escape excludes skin-PROTECTANT monograph actives —
cold creams with petrolatum were riding the efficacy axis while syndets rode
the job axis, one list scored on two questions. (6) rinse-off cap is a
monotonic soft knee ([knee,100]→[knee,cap]) — the old compression mapped a
true 88.4 BELOW an uncapped 79.3. CeraVe Hydrating Cream-to-Foam: B− → B+,
rank #85 → #20 of 498 cleansers for dryness.
Brand trust v2: base 58 / cap +37 (max 95 — the old 75 ceiling read as
"60-ish" for spotless legacy brands); reachable track-record curve (6·log10);
clinical_investment implemented (was defined, never coded);
heritage_adoption; medical seals 10 vs generic 4; transparency scaled by
disclosed-concentration share, REWARD-only; opacity penalty for branded secret
heroes, halved when real outcomes exist. CeraVe 60.4 → 90.1, Cetaphil 95,
Augustinus Bader 64.7. All credential leaves always emitted (0 when unearned,
with the reasoning stated) so the breakdown shows what was assessed.
Recognition display: assessments carry a recognition block; the audit
trail renders named seals with the tier ladder, counted/not-counted role, and
directory links; no-seal products get the not-a-mark-against note.
Gate 18/24 with CATEGORY_JOBS on; golden PASS; both flags remain OFF. Lesson recorded: three of these defects were invisible to the anchor gate (none touched an anchor) — input audits and owner spot-checks catch what output gates structurally cannot.
2026-07-27 — Engine 4.4.1 (Phase 5b): two dose defects fixed; the D1 inversion found and NOT patched
The anchor gate's remaining failures pointed at the dose channel, not evidence.
Fix 1 — sub-1% actives stop being punished for potency. dose_proxy fell
back to INCI position whenever a concentration was undisclosed. A retinoid or
peptide studied at 0.075–0.5% always sits low in an INCI list precisely
because it is potent, so the position rule scored potency as pixie-dusting.
The rubric already said studied ranges come from the evidence row's
dose_range — nothing read them. Now: undisclosed + studied max < 1% reads as
the rubric's "cannot tell" bucket (not-locatable, confidence lowered) instead
of a position penalty.
Fix 2 — condition grades use the CARRYING active's dose. Dose was the mean across every hero with a row — the same hero-mean bug §5 removed from evidence, still live in the sibling channel. A condition grade asks whether this product does this job, so the dose that matters belongs to the active that carries the claim. (The overall score keeps the mean: it measures whole-formula quality, where every active counts.)
Result: golden PASS, invariants hold. RoC Retinol aging rank #2,579 → #412 of 3,362 — its relative position is now defensible (top 12%), though its absolute letter still misses the B+ floor, so the anchor still fails. Catalog movement vs 4.3.2 is now two-sided (5,141 up / 3,059 down, worst −6.5): products whose carrying active is under-dosed but that carried a well-dosed supporting ingredient were previously flattered, and no longer are.
The D1 inversion — characterized, deliberately NOT patched
The KP cluster has a different and more serious cause, and it is general, not KP-specific:
A researched-but-thin active scores 30. A product with no relevant active at all scores 50 — the D1 no-admissible-evidence prior. Honest research that finds weak support therefore ranks below silence.
Measured: Nécessaire's plain fragrance-free body lotion (no keratolytic; ev 50,
prior:no-admissible-evidence(D1)) outranks AmLactin 12% ammonium lactate
(ev 30, researched) for keratosis pilaris. AmLactin stays #19 of 169.
This is not patched because the obvious fix is wrong. D1's prior protects products whose ingredients simply have no row yet — the table holds 196 rows against a far larger catalog — so lowering the prior globally would punish the research backlog, exactly what D1 forbade. The engine cannot presently distinguish "has a claim-relevant active we haven't researched" from "has no claim-relevant active at all," and that distinction is the whole fix. It changes an owner-approved decision and needs an explicit call.
Flag remains off; nothing user-facing changed.
2026-07-27 — Engine 4.5.2: measurable job features + not-applicable handling
keratolytic_capability and targeted_substance are now measured rather
than punted to the evidence axis. This matters most for keratosis pilaris,
where NO guideline exists anywhere, so no ingredient can earn standing and the
evidence axis is flat across the entire condition — whether a product actually
carries a keratolytic at a real dose is the only discriminating fact available.
AmLactin now scores 95 on "Handles rough, bumpy skin" (12% lactic acid,
disclosed).
Not-applicable handling (the bug this exposed). The first cut scored
keratolytic_capability for every condition, so eczema barrier creams were
marked DOWN for lacking an acid they should not contain — Aveeno Eczema
Therapy and Eucerin Eczema Relief both fell out of the gate (18 → 16). A job
feature irrelevant to the graded condition is now dropped from the weighting
and renormalised away (n/a: sentinel), never scored as a failure. Gate back
to 18/24, Aveeno A− 81.3.
The residual KP failure is a ranking-SCOPE artifact, not a scoring one. The mixed KP list is topped by body washes; AmLactin sits #28 of 169. Filtering to leave-on products only — what within-category ranking will show — moves it to #21 of 142 and puts keratolytic products at the top. The remaining gap is its "Recognised active" term (30, the D1/thin-standing issue) plus two still-unmeasured body-care features. Conclusion: stop adding scoring levers for this cluster — build category segmentation.
2026-07-27 — Engine 4.5.1: emollient/occlusive split + medicated escape narrowed
Measurement gap closed. inci-flag-rules.json lumped emollients and
occlusives into one emollients_occlusives set, so "softens" and "seals"
shared a signal — fatal for a model where occlusivity is a virtue for eczema
and a liability for acne. Split by MECHANISM into occlusives (film-forming,
measurable TEWL reduction), emollients (fill and soften) and
barrier_lipids; the original union is retained so the category floor's
behaviour is unchanged. Petrolatum/lanolin/mineral oil now read as the
reference occlusives (100) vs other occlusives (80), and barrier lipids score
higher when ceramide AND cholesterol are both present.
Medicated escape narrowed to rinse-off. 4.5.0's rule sent any product
carrying a standing-bearing active back to the efficacy axis. For a leave-on
moisturizer, holding petrolatum IS the job (occlusion) — so the broad rule
silently disabled the job model on nearly every good eczema moisturizer
(Aquaphor and CeraVe Moisturizing Cream both bypassed it, seal=None). Now
only rinse-off medicated formats escape: PanOxyl stays on the efficacy axis and
passes its B+ anchor, while leave-on formats keep the job axis and are credited
for standing through the 0.75/0.25 blend.
Results: golden PASS, invariants hold, anchor gate 18/24 with the job axis on, 24,089 grades scored on it. Aquaphor eczema A 83.4 (seal 100 / soften 60 — correctly read as a pure occlusive), CeraVe Moisturizing Cream A+ 91.1 (seal 100 / soften 85). Flag still opt-in and off by default.
2026-07-27 — Engine 4.5.0: category job specs wired (opt-in CATEGORY_JOBS)
The reframe, implemented: we do not rank OTC products on whether they treat a condition — we rank how favourably they impact that skin while performing their category's intended function.
New artifact formula/category-job-specs.json (cjs-2026.07): 14 categories,
each with a stated function, an efficacy_led flag and weighted job features
(weights sum to 1.0), plus 20 cross terms for pairs where the job changes
sign. Canonical example: occlusivity is ×1.5 for moisturizer|eczema
(petrolatum is the reference emollient) and ×0.45 for moisturizer|acne —
identical property, opposite sign, underivable from either layer alone. Every
feature carries label + user_explanation because this rationale is meant to
be RENDERED; signal names the artifact each claim is traced to.
Engine composition — efficacy-led categories (treatment, retinoid, serum, exfoliant) keep the evidence/dose/craft path. Function-led categories score on the job axis, blended 0.75 job / 0.25 evidence so standing is never discarded. Two rules the anchor gate forced out, both correct:
- Medicated formats stay on the efficacy axis. A 10% benzoyl-peroxide WASH is delivering treatment, not merely cleansing. Scoring PanOxyl as a cleanser dropped it to D. Category alone cannot decide the axis — carrying an active with verified standing for this condition does.
- Job-led must blend, not replace. Pure job scoring stripped Aveeno Eczema Therapy of the standing its monograph oatmeal earns (A → B+).
Unmeasured features are disclosed, never guessed — JOB_UNKNOWN_PRIOR (60)
plus reason code <cond>:category-job-unmeasured:<n>, and ≥3 unknowns marks the
grade Early. Current coverage: 24,506 grades score on the job axis; most carry
1–3 unmeasured features (usability/reapplication signals and the emollient vs
occlusive split are the outstanding measurement work — the flag rules lump
emollients and occlusives into one set).
Results: golden PASS, all §3 invariants hold, anchor gate 18/24 with the
job axis ON (same as off, but now on the honest model), and RoC Retinol/aging
improves D+ 61.8 → B 69.1. Flag is opt-in and OFF by default until
measurement coverage improves; ASSESSMENT_ENGINE remains off, so nothing is
user-facing either way.
Consequence recorded: within-category ranking makes the rinse-off contact-time cap a legacy workaround — it existed only to keep cleansers out of mixed lists. Retire or demote it to a display safeguard once condition pages segment by category.
2026-07-27 — Engine 4.4.0: standing-vs-certainty LIVE in the sidecar (Phase 5 core)
The redesign is implemented. Three changes, all inside the evidence channel:
- Standing drives the score. A row whose verified standing rests on a
channel beyond the trial ladder (
regulatory-active,guideline-named,guideline-class-exemplar) is scored from its standing value. A row that merely confirms its seed keeps the existing ladder (§4.2) — this also preserves the physicochemical ladder'splausible-mechanism= 45, a value the 4-step standing vocabulary cannot express. Bug caught and fixed mid-phase: the first cut scored every verified row from the standing map, silently repricing plausible-mechanism rows 45 → 30 and producing 2,502 spurious downgrades; the channel rule removed them (now 0). - Certainty relocated to confidence. The carrying row's GRADE certainty
sets
confidence.clinical_evidence(high→strong, low/very-low→thin) and is disclosed on the evidence part ascertainty. It never enters the number. - Certification basis wired (§11.1): Tier 1 → 100, Tier 2 → 75 on the
matching condition grade, saturating (max, not additive), evidence-part
only — so the irritation and fit channels always still apply. Sidecar
records
certifications_version.
Results (9,313 products, 38,046 grades, vs the 4.3.2 baseline): golden PASS, all §3 invariants hold. 3,098 grades up, 0 down; 1,850 overall scores up, 0 down. Band shape A 13% → 16% (simulation predicted 15%; the delta is the certification channel, which the simulation did not model). Anchor gate 11/24 → 17/24, and every eczema anchor now passes — Aveeno Eczema Therapy A (from B− pre-audit), Vanicream A+, Lipikar A+, CeraVe A, Eucerin A, Aquaphor A. The inversion that started this work is gone.
The 7 remaining anchor failures — NOT waived
Two are the designed disagreement §4.4 already argued in advance (vitamin C is adjunctive for pigment): SkinCeuticals C E Ferulic B, Discoloration Defense B. Those are waiver candidates on reasoning already recorded.
The other five are not defensible as-is, and the gate has surfaced a defect the redesign never addressed — the evidence channel stops discriminating, and non-efficacy factors decide the ranking:
- Keratosis pilaris (AmLactin B, CeraVe SA C+, FAB scrub B−). Research
verified that no KP ingredient has any regulatory or guideline standing:
no KP guideline exists anywhere, ammonium lactate's Rx approval (Lac-Hydrin)
is for xerosis/ichthyosis, and the salicylic-acid monograph does not name KP.
So every KP ingredient sits at
thin= 30, evidence is constant across the whole condition, and the ranking is decided by craft/fit alone. AmLactin — the 12% ammonium-lactate standard of care — ranks #19 of 169, below a plain fragrance-free body lotion. - Aging (RoC Retinol D+, CeraVe Retinol B). Retinol correctly stayed
emerging(tretinoin ≠ retinol; no photoaging monograph). But RoC ranks #2,579 of 3,362 for aging, because its undisclosed retinol concentration drivesdoseto 43 via the INCI-position proxy — the proxy treats "concentration not printed on the box" as if it were pixie-dusting, on a product with published RCTs on its own formulation.
Waiving these would be precisely the self-serving move §6.4/§9.2 warns against. They are recorded as open defects for a follow-up pass: the dose proxy's treatment of undisclosed concentrations, and the no-standing-condition case where a constant evidence channel hands the ranking to other factors.
Flag remains off; nothing user-facing changed.
2026-07-27 — Phase 2c (Tier 2/4 pass): NRS wired in, DAAB walked (zero overlap), registry rs-2026.07.1
NRS Seal of Acceptance promoted candidate → verified-live and added to the certification builder. Its searchable directory is JS/antibot-gated, so the accepted-product list is assembled from the Society's own press announcements (Jan 2024 launch cohort of 10 + Apr 2024 additions of 6 — both opened and parsed in-session): 16 products, 4 catalog matches (Avène Tolerance Control, Prequel Redness Reform + Sun Barrier, Skinfix barrier+ cleanser), condition rosacea, Tier 2. A general same-source dedup also trimmed size/variant duplicates (CDA, SCF). Registry now 236 certifications across 215 products.
DAAB (Tier 4) walked — legitimate zero: Logo_Liste_2025_9.pdf (12pp) parsed; products are German drugstore/pharmacy lines (Isana/Rossmann, Physiogel shampoo) with no catalog overlap. Recorded in the source registry; re-walk only if German pharmacy brands enter the catalog. Remaining Tier 4 bodies (ECARF, Allergy UK, Skin Health Alliance) deferred on the same prior. Still open in 2c: Tier 3 regulatory designations (MFDS/quasi-drug per-product labeling) and Tier 5 placements (Consumer Reports owner-provided; editorial list collection).
2026-07-27 — Phase 2b: Tier 1–2 certification walk complete (product-certifications.json 1.0.0)
scripts/build-product-certifications.py (rerunnable; live-fetch + cache,
--offline for cached HTML) walked all six verified Tier 1–2 directories
against the catalog: CDA skincare (188 entries) + CDA sunscreen (64), ESC seal
(30), NPF seal (23), SCF seal (467 sunscreen rows), NEA seal (827 product pages
enumerated via their product sitemap). Result: 238 certifications across 213
catalog products (NEA 154, CDA 42, SCF 34, ESC 4, NPF 4), written to
formula/product-certifications.json; 25 borderline cases quarantined in
product-certifications-review.txt — never in the registry.
Matching is strict and variant-safe: full brand+name token containment, numeric
/SPF tokens must agree exactly, variant qualifiers (baby/kids/tinted/mineral/…)
disqualify rather than fuzz, ambiguous = review-file. NEA needed a two-stage
match (their directory renames products — "Vanicream Moisturizing Cream" for
our Moisturizing Skin Cream, "Lipikar Soothing Eczema Relief Cream"): candidate
generation by token overlap, then the product's own NEA detail page fetched and
its brand-name + <h2> hero fields decide; size/variant duplicates deduped.
All five NEA-Seal eczema anchors matched (Aveeno ET, CeraVe MC, Vanicream,
Eucerin ER, Lipikar), which the credibility-symmetry rule requires.
Not yet consumed by the engine (4.4.0 wires the certification basis in
Phase 5). Still open in 2c: Tier 3 regulatory designations (MFDS/quasi-drug,
per-product labeling research), Tier 4 sensitivity registries, Tier 5
placements (Consumer Reports data owner-provided).
2026-07-27 — Phases 2 + 3 shipped; Phase 4 standing-research pipeline staged
Phase 2 — anchor-set corrections per proposal §9.2 (anchor-products.json
1.1.0, discipline test recorded in _meta.revisions): struck Olay Regenerist/
aging; floors A− → B+ on RoC Retinol/aging, SkinCeuticals Discoloration
Defense/hyperpigmentation, CeraVe SA Smoothing Cream/KP. Gate now 11/24;
all 13 remaining failures are the standing gap, as designed.
Phase 3 — standing schema migration (ingredient-evidence-table.json
0.5.0-standing, backed up pre-migration): every one of the 196 rows carries a
standing block {value, basis[], source, confidence} seeded from the existing
trial ladder (physicochemical rows seed as mechanism-established) at
confidence: needs-verification — the tripwire; no standing may drive a score
until verified. tier untouched; sanity-check-evidence PASS; golden PASS
(engine 4.3.x unaffected).
Phase 4 infrastructure (mirrors the evidence pipeline): 195 pairs queued
in formula/standing-queue/standing-batch-001..033.txt (6/batch,
priority-ordered — the 123 limited/emerging clinical rows first), skill
/process-next-standing-batch (atomic claim, §4 basis tests incl. the §9.1
two-part class-exemplar test, §4.4 anti-inflation calibration cases, verified-
URL rule), merge gate scripts/merge-standing-rows.py (enum + source
validation, trial-only rows must match the seed, stronger basis can only raise;
refuses while claims held; reports remaining needs-verification count —
Phase 4 exits at zero). Validator smoke-tested both directions.
2026-07-27 — Phase 2a: comprehensive recognition-source registry (recognition-sources.json rs-2026.07)
Systematic international sweep completed before the recognition research pass;
canonical machine-readable registry at formula/recognition-sources.json,
superseding the proposal's inline tables for enumeration. 24 sources across
the tier ladder + 7 recorded exclusions. Additions found by the sweep:
National Rosacea Society Seal of Acceptance (2024 launch → Tier 2, rosacea),
DAAB (Germany → Tier 4), CHOICE Australia / Stiftung Warentest / ÖKO-TEST /
Which? (international Tier 5A lab-testers; CHOICE's 2025 SPF test triggered a
national recall). @cosme/Glowpick/Hwahae verified consumer-voted → display-only
by rule; their review corpora flagged as potential future outcomes sources.
Verified exclusions: AAD seal (phased out by the AAD), UK National Eczema
Society (does not endorse), EAA logo, Cancer Council. Governance: registry
entries carry verification.status; a candidate source contributes nothing
until promoted to verified-live (same tripwire as needs-verification
standings). Paywalled sources (Consumer Reports, Which?, Stiftung Warentest)
= owner-provided data with recorded provenance.
2026-07-27 — Engine 4.3.3: max-aggregation (evidence redesign Phase 1) SHIPPED to the sidecar
Phase 1 of the approved redesign (proposal §5, §10): the evidence component of
every condition grade and of the overall score is now carried by the
best-supported claim-relevant active (max) instead of the mean across named
heroes — a product is no longer punished for containing a second proven active.
Pure formula change, no new data. The carrying active is recorded per grade as
reason code <cond>:evidence-best-supported:<slug> (ties break alphabetically
for determinism); dose adequacy is deliberately unchanged (its own channel).
All three Phase 1 gates met on the rebuild (9,313 products, 541 outcomes files consumed):
- Golden test PASS — exemplar values unchanged (single-active per claim, so max == mean there by construction).
- Anchor gate 6/25 → 11/25, exactly as the §7 simulation predicted. Both §3.2(b) smoking guns resolve: Effaclar Duo now passes its acne anchor at A− (BPO carries; SA no longer dilutes), and Paula's Choice 10% Azelaic passes rosacea at A (the soothing agents no longer cost 35 points).
- Zero downward movement: 1,787 of 38,046 condition grades moved, all up (sim said 1,517 — the delta is corpus growth since the sim run); 836 overall scores up, 0 down. Largest single lift +14.0 (Dove DermaSeries eczema lotion, eczema).
The 14 remaining anchor failures are the standing gap (evidence availability ≠ efficacy) — Phases 2–5. Flag remains off; nothing user-facing changed.
2026-07-27 — Evidence redesign APPROVED, with amendment: certifications in, editorial lists calibration-only
The owner approved the standing-vs-certainty redesign
(formula/evidence-redesign-proposal.md) — Standing/Certainty split plus
max-aggregation — and added one balancing requirement, recorded as proposal
§11. Two kinds of external signal, treated oppositely:
- Criteria-based medical certifications become a score input — a sixth,
product-level standing basis
certification, with an international, four-tier roster (all programs verified live this session; §11.1): Tier 1 professional derm associations (CDA Recognized Products); Tier 2 condition-focused medical orgs (NEA Seal → eczema, Eczema Society of Canada Seal → eczema, NPF Seal → psoriasis (weaker claim, maps lower), Skin Cancer Foundation Seal → photoprotection); Tier 3 regulatory functional designations — the K/J-beauty balance channel (Korea MFDS functional cosmetic, Japan MHLW quasi-drug; high-volume/low-selectivity → maps lower, neverestablishedalone); Tier 4 allergy/sensitivity bodies (ECARF, Allergy UK, Skin Health Alliance) scoped to the irritation channel only. Verified exclusions: EAA logo (corporate-partner mark, EAA states it is not an endorsement), Cancer Council (sells own products). Inclusion bar: published criteria + independent expert panel + live verifiable directory; every assignment cited against the directory in-session (PMID rule applies). Exact point mapping is set during implementation with the anchor gate as acceptance test (§11.3); a seal can never override the irritation/fit channels. Credibility symmetry rule: in-catalog products on Tier 1–2 lists auto-enter the calibration corpus with floors — grading a CDA/NEA-listed product poorly presumes our model wrong. - Expert editorial lists enter the score as Tier 5 (owner revision later the same day, superseding the initial calibration-only decision; §6.3 narrowed, not discarded). Guardrails (§11.2): hard cap of a few points, structurally unable to cross more than one grade boundary; ≥2 independent 5A/5B sources to reach the cap; condition-scoped only (unscoped "best of year" = display-only) — the anti-popularity-leak rule; 24-month recency; verified-URL provenance; reader-voted awards and Tier 5C content sites (Healthline/Verywell) never weigh. Sub-tiers: 5A instrumented lab (GH Beauty Lab, Consumer Reports — CR data owner-provided via subscriber account, provenance recorded), 5B expert panel (Allure BoB, Wirecutter), 5C display/corroboration only. Tier ladder formalized (§11.3): T1 > T2 > T3 > T4 (irritation only) > T5 (capped) > display-only(0); stacked recognitions saturate. Calibration-corpus role unchanged. New §11.2.1 "External recognition & research" panel: every recognition found, linked + dated + tier-labeled with its score role, unequal weights stated plainly; purposes = demonstrated research depth, reader convenience (outbound links), SEO (unique dated content, schema.org Product.award). Phases gain 2c: recognition research pass (T1–4 directories vs catalog, then T5 placements, best-of members first, claimable batches).
Implementation phases gain 2b (product-certifications registry + basis wiring) and 7 (calibration corpus expansion). Sequencing unchanged: Phase 1 (max-aggregation, engine 4.3.3) lands first — no new research required. Flag remains off; nothing user-facing changed.
2026-07-26 — Engine 4.3.2: calibration audit, two structural fixes, anchor gate
A credibility audit (triggered by "Petrolatum has limited evidence for eczema" rendering on product pages) found the pre-fix eczema ranking near-inverted at the top: bar soap, shampoos and body washes held A+ while every NEA-Seal eczema cream sat in the B range (Aveeno Eczema Therapy B−). Three compounding causes; the two that are wrong under ANY methodology are fixed in 4.3.2, the third is a designed follow-up.
Fix 1 — condition grades no longer inherit general popularity. When a
product had collected reviews but too few mentioned the graded condition, the
Outcomes(c) component fell back to the general de-biased sentiment signal
(fallback mean 93.7 vs 60.0 prior — ~+7 grade points awarded for merely having
been sampled; 46 of 411 eczema grades). The general signal already feeds the
overall score via F3; per-condition it is now withheld and the neutral prior
holds, with reason code <cond>:outcomes-below-n-strat:neutral-prior.
Stratified (condition-matched) outcomes are unchanged.
Fix 2 — contact-time gate. Rinse-off products (routine_step: cleanser or
category cleanser) cap at 79.5 (= just under the frozen gb-2026.07 A− bound,
so B+ ceiling) on every condition grade, reason code
<cond>:rinse-off-contact-time:grade-capped, uncapped value preserved as
score_pre_cap so components still audit. Seconds of contact can support
"gentle, doesn't aggravate" — never a top treatment ranking. (Generalizes the
Peace Out rinse-off lesson into the engine.)
New permanent artifact — anchor calibration gate. formula/ anchor-products.json (25 consensus anchors: NEA Seal holders, FDA-monograph/
guideline actives, standard-of-care staples across 8 conditions) +
scripts/check-anchors.py (report mode; --strict for CI). Governance: a
failing anchor means the MODEL is presumed wrong; the only way to ship one is
an explicit waiver recording the specific defended disagreement.
Results: golden test PASS; all §3 invariants hold. Rinse-off products are
out of the treatment rankings (Dove bar 89.1→79.5 capped; PanOxyl wash lands
exactly at its designed B+ ceiling and passes its B+ anchor floor). Anchor gate:
6/25 pass, 19 fail — the remaining failures are the third cause, evidence
availability standing in for efficacy (limited=30 for petrolatum/colloidal
oatmeal/ammonium lactate/retinol while any glycerin-named-hero product scores
100; 67% of clinical evidence rows sit at limited; RoC Retinol grades D for
aging). That requires the standing-vs-certainty evidence redesign (regulatory
monograph + guideline-class channel, best-supported-active instead of
hero-mean) — deliberately NOT patched here; it changes methodology and awaits
sign-off. The anchor gate is the acceptance test for that redesign.
Flag remains off; nothing user-facing changed. ASSESSMENT_ENGINE=1 surfaces
now read the 4.3.2 sidecar.
Follow-up drafted (2026-07-26): formula/evidence-redesign-proposal.md — the
standing-vs-certainty redesign for the third cause, prepared for Dr. Thomas +
owner review. Simulated read-only over all 38,046 condition grades via
scripts/simulate-standing.py against a 16-pair prototype overlay
(formula/standing-overlay-prototype.json, NOT engine-consumed): max-aggregation
alone takes the anchor gate 6/25 → 11/25, standing alone → 15/25, both → 21/25,
while moving only 2,466 of 38,046 grades (6.5%, all upward; A band 13% → 15%) —
a targeted correction, not inflation. Four residual anchor disagreements are
itemized in the proposal §8 as explicit clinical judgment calls. Not implemented;
awaiting sign-off.
2026-07-22 — Three more free review sources wired: Yotpo, Okendo, Stamped
Amazon and Sephora were evaluated and rejected as collection sources — both are Akamai/login-walled (Sephora product pages 403 to raw fetch and to headless Chromium; Amazon is login-walled + ToS-hostile). The scalable win is instead the rest of the Shopify review-widget family — the same $0 public-API route as Judge.me, reaching the K-beauty / DTC tail. All three wired and proven live:
| App | Key discovery (server-side) | Data endpoint | Proven |
|---|---|---|---|
| Yotpo v3 | staticw2\.yotpo\.com\/{KEY} in product-page HTML (escaped slashes) |
api-cdn.yotpo.com/v3/storefront/store/{key}/product/{pid}/reviews + bottomline.starDistribution |
COSRX snail essence 3,180 reviews, full distribution |
| Okendo | subscriberId":"{uuid}" in HTML |
api.okendo.io/v1/stores/{sub}/products/shopify-{pid}/reviews + nextUrl paging |
Anua Heartleaf toner 200, syndication flagged |
| Stamped | myshopify domain in HTML → stamped.io/api/getappkey → apiKey |
stamped.io/api/widget/reviews?productId&apiKey&storeUrl |
SKIN1004 sun serum 303 |
- Collector (
collect-outcomes.py): three normalizers + fetchers writing the exact row shape the extractor already consumes (verified/incentivized/ syndicated flags mapped per app; Yotpo uses its realbottomlinepopulation distribution, Okendo/Stamped build the distribution from the sampled ratings so shrinkage honestly treats them as thin).fetch_slugdispatches Target → BV → Judge.me → Yotpo → Okendo → Stamped. - Resolver (
resolve-outcome-sources.py):resolve_judgeme→ unifiedresolve_shopify— matches the product in the store catalog, detects the store's review app across a few sampled product pages (keys are store-wide; one page with the widget suffices), probes for a live count, and returns the richest source (Yotpo → Okendo → Stamped → Judge.me). Hardened with Shopify 429 (local_rate_limited) backoff so a transient throttle is never cached as a permanent "no-catalog" miss. New--retry-shopifyre-examines exactly the misses these adapters can now convert (the ~615shopify-match-no-judgeme/shopify-no-title-match/ catalog-error members previously checked only against Judge.me). - Coverage note: the earlier changelog's "K-beauty US Shopify stores carry thin, near-uniform-5★ syndicated sets" finding still holds for some brands (Torriden US = 0 reviews, Isntree thin) — but the big K-beauty houses on Yotpo (COSRX 3,180) and Okendo (Anua 200) carry real volume.
Result of the source expansion (2026-07-24 rebuild)
--retry-shopifypass: +290 resolutions (137 Okendo, 80 Yotpo, 73 Stamped), best-of members resolved 193 → 507. Match precision spot-checked: 0.7% mismatch (2 removed, 1 over-drop restored).- Collected corpus 140 → 443 outcomes files (3.2×), all on ext-1.1 (449 extractions, uniform version; cumulative extractor spend ≈ $160). Two transient batch timeouts (krave lip jelly, medik8 retinal) self-recovered on retry.
- Assessments rebuilt: 9,313 assessed, 100% coverage, all §3 invariants hold, golden PASS. Confidence bands 318 High / 8,995 Moderate / 0 Early at the overall level (was 8 High at pilot start, 100 at the first-batch rebuild). 68 condition grades off "Early", 75 on a stratified basis.
- Grade movement vs legacy (443 collected): mean −3.1, median −3, range −22..+19 — the honest, two-sided recalibration D3 predicted. Lifts where real reviews beat the inflated LLM prior (Advanced Clinicals vit-C +18, Anua azelaic +14); drops where they undercut it (EltaMD UV Clear −22, Clinique Take The Day Off −19).
- Best-of rerank dry-run: 0 gutted lists ✅ (the §7 cutover gate holds),
809 condition-list members fall below the B− floor (the accepted deflation).
Still DRY-RUN — nothing written;
--writeremains gated on explicit P5 go.
2026-07-18 — ext-1.1: efficacy definition tightened; κ 0.375 → 0.851 — extractor cleared for scaling
The one field flagged by the 2026-07-18 reliability audit (efficacy, κ=0.375 vs
the 0.7 target) is fixed. New frozen extractor:
ext-1.1/claude-haiku-4-5-20251001/64654286.
What changed in the prompt (collect-outcomes.py)
effis now an ordered decision procedure (harm → improvement → no-change → null, stop at first match) instead of a one-line definition. The boundary rules the raters were splitting on are explicit: efficacy is about the reviewer's skin, not the product — product feel/finish/scent/packaging and generic praise ("love it", "perfect for sensitive skin") are NOT efficacy; absence of harm ("didn't break me out") is tolerability → null; "too soon to tell" → null; mixed harm+improvement resolves by the reviewer's overall verdict, else "-".- Context rule tightened (the audit's other systematic issue):
ctxis pre-existing states the reviewer has — a product-caused state ("gave me dry patches") is asp/eff, never ctx. - Two boundary mapping examples added; irritation bucket now names caused dryness explicitly.
Verification — pair audit (new audit-extractor.py --pair mode, $1.86)
After a prompt change the stored ext-1.0 labels can't serve as reference (definition shift ≠ error), so the pair mode rates all 100 sample quotes fresh with both Haiku (bulk extractor) and Sonnet-5 (independent rater) under the identical ext-1.1 contract and scores their agreement:
| Field | ext-1.0 (Haiku~Sonnet) | ext-1.1 (Haiku~Sonnet) |
|---|---|---|
| efficacy | 61% exact / κ=0.375 | 91% exact / κ=0.851 ✅ clears 0.7 |
| aspects | F1=0.713 | F1=0.846 |
| context | F1=0.787 | F1=0.872 (recall 1.0) |
Label shift vs stored ext-1.0: 31/100 changed, 24 of them + → null — the
generic-praise/product-feel inflation being removed, exactly the intended
semantic change. Report: formula/outcomes-extractor-audit-ext-1.1-pair.md.
Consequence for collected data
The 13 pilot outcomes files were extracted with ext-1.0 and carried that +
inflation (24% of $8, no re-fetching)** — the whole corpus now sits on
ext-1.1. Outcomes rebuilt, assessments regenerated (all §3 invariants hold,
8 products at High confidence), golden test PASS.+ rows flip to null under ext-1.1). **Re-extracted same
day from cached raw (
2026-07-17 — Outcomes pilot: RUN. Real review data in the engine; first grades off "Early"
The 50-product outcomes pilot (outcomes-pilot-plan.md) was executed end-to-end.
Engine bumped 4.3.0 → 4.3.1 (consumes src/data/outcomes/<slug>.json). Golden
test PASS throughout; flag-off site behavior unchanged; all §3 invariants hold on
all 9,313 records.
What was built
scripts/pick-outcomes-pilot.py— deterministic stride-sampled selector (no RNG): 50 slugs, 20/20/10 volume tiers, all 8 required categories, 3 gated, golden exemplar included →formula/outcomes-pilot-slugs.json.scripts/collect-outcomes.py— fetch → extract → build. Fetch pulls real rating distributions + stratified text (recent/helpful/verified/seeded-spread, cap 250) from Target's r2d2 review API (the one major retailer API not bot-gated; every raw response cached in gitignoredsrc/data/outcomes-raw/). Extract runs the frozen extractorext-1.0/claude-haiku-4-5-20251001/368e059c(transcription contract; batches of 25; version = model + prompt-sha). Build applies the locked §5.3 de-bias (verified ×1.0/×0.5 as a delta on the pooled mean, incentivized −0.38★, shape checks, per-category shrinkage, 24-mo half-life age weights) and writes schema-§7 files withrecords[]quotes andfingerprint_at_collection(fingerprint fn imported from the engine so they can't drift).- Engine outcomes path (
build-assessments.py): F3 = 65% aspect sentiment / 35% de-biased stars (+ repurchase ±10 / tolerability −15 bands); gradeOutcomes(c)stratified fromrecords[]atN_strat=30(condition ∪ adjacent skin type; condition-relevant failure aspect at 0.8 weight), general-fallback one band lower, prior+Early otherwise; confidence lift with freshness / reformulation guards;sourcesin components now cite the collection.
Pilot results (decision-gate numbers)
| Metric | Result |
|---|---|
| Source coverage | 17/50 resolvable at a fetchable retailer (Target); 13/50 yielded data; 9/50 substantive (≥40 texts). K-beauty/J-beauty/EU-pharmacy/DTC tail is unreachable via open APIs (Sephora/Ulta/iHerb/Amazon all bot-gated) — the scale-up blocker to solve. |
| Collection volume | 1,830 text reviews extracted; ~20k pooled ratings across 13 products |
| Extraction cost | $8.16 total ≈ $1.10 per 250-review product (Haiku, batches of 25) → full-catalog projection ≈ $6–10k at same density, or ~$2k for the ~2k best-of members |
| Stratified realism | 6 stratified grades fired (golden: rosacea n=57 + acne n=58; Gold Bond: KP n=47 + rough-texture n=43; Aveeno sensitivity n=70; Naturium dryness n=33); 62 grades on general-fallback. Stratification is reachable for hero products but most grades ride the general signal at 250-text sampling. |
| Score movement | Substantive products +5 to +8 overall (real F3 87–99 vs prior 60); thin collections −2/−3 (shrinkage holds them at the category prior). 8 products lifted to High confidence; golden exemplar 77→83-High, rosacea A+ stratified early:false. |
| Extractor accuracy (operator read, 18-pair sample) | ~92% rows fully faithful. Systematic issues: product-caused states occasionally transcribed as pre-existing context ("gave me dry patches" → ctx dryness); "thick texture" polarity judgment calls. Formal κ audit sample at formula/outcomes-extractor-audit-sample.json (100 records, deterministic). |
Pilot findings that changed the design (fixed in this pass)
- Tiny-n modifier noise — 1 negative in 3 records = "33% irritation" → −15.
Aspect-band modifiers now require
n_sampled ≥ 20(MIN_ASPECT_N). - Confidence over-lift — a 2-rating collection reached band High. High now
requires a real sample (
user_evidence == "moderate", n ≥ 30). - Hidden incentivized reviews — Target's
is_incentivizedflag misses promo reviews whose disclosure lives in the text ([This review was collected as part of a promotion.]). Text-marker detection added: measured incentivized shares of 36–50% on four products (!), moving adjusted means −0.15 to −0.29★. The de-bias layer is not optional.
Second free source wired — Bazaarvoice (2026-07-17, addendum)
Constraint from the user: no budget for paid API access or scraping tools — collect only via built-in capabilities. Added a second free source to the collector alongside Target r2d2:
-
Bazaarvoice Conversations API (
api.bazaarvoice.com/data/reviews.json). The passkey is a public client-side constant every brand ships in its own page bundle (apps.bazaarvoice.com/deployments/<client>/…/ratings-config.js), not a secret — extract it + thedata-bv-product-idfrom the brand's product page. Proven live: CeraVe → 751 real reviews with full rating distribution, dates, and per-review flags, normalized to the exact row shape the extractor already consumes (verified/incentivized from BV Badges — never fromCampaignId, which is a display-context tag, not an incentive).fetch_slugdispatches on source type; Target stays primary where present, BV is the brand-site fallback. This opens the entire derm/pharmacy mass catalog (CeraVe, La Roche-Posay, Cetaphil, Aveeno, Neutrogena, …) at zero cost. -
Coverage structure that emerged (the real scale-up map):
- Target r2d2 → mass-market + indie carried at Target (Western head).
- Bazaarvoice brand sites → derm/pharmacy brands (overlaps Target's head).
- Shopify brand stores (
/products.jsonopen; Judge.me/Yotpo/Okendo widgets) → the K-beauty / DTC tail Target and BV don't reach. - Hard-walled, not free: Sephora, Ulta, iHerb, Amazon (bot/Cloudflare gated).
-
Pilot-50 yield of BV: zero net-new products — the 50's Western brands are already Target-covered, and its actual gaps are K-beauty (Shopify apps), Obagi (Okendo), Paula's Choice (PowerReviews), Exuviance (no active reviews). So the pilot's collected set stays at 13; BV is a reusable capability banked for the broader catalog, not a pilot-50 rescue. Fable 5 confirmed as the session/orchestration model; Haiku remains the bulk extractor by design.
Third free source wired — Judge.me / Shopify (2026-07-17, addendum)
Reached the K-beauty / DTC tail Target and Bazaarvoice can't see, still $0:
- Shopify
/products.jsonis open on the US brand stores → resolve the product's Shopify id for free. Review app detected from the product page. - Judge.me public widget JSON (
judge.me/reviews/reviews_for_widget, keyed byshop_domain+ Shopifyproduct_id) returns a clean object:number_of_reviews,histogram(the pooled distribution), andreviews[]with rating /created_at/verified_buyer/transparency_badges/ text. Wired as a third source (fetch_judgeme), same row shape;fetch_slugdispatches Target → BV → Judge.me. Collected Numbuzin No.4 end-to-end (7 reviews). - Okendo (SKIN1004) and PowerReviews (Paula's Choice) noted but not wired — each is a distinct public-widget API, addable the same way.
K-beauty data-quality finding (the reason the tail is hard, cost aside): US
Shopify stores carry thin, heavily-syndicated, near-uniform-5★ review sets.
Numbuzin's catalog holds real volume (11k+ in a 40-SKU sample) but it's
concentrated on hero-SKU groups and 5/7 of the pilot product's reviews are
review_collected_from_another_provider (translated-from-Korean marketing
copy); Torriden's US store has 0 reviews. The de-bias is doing its job:
shrinkage pulls the 7-review Numbuzin star signal to the category prior (4.06 ≈
prior 4.04) and it stays Moderate/thin — a real data point, correctly discounted.
Thin-tail engine fix (found here): the F3 aspect-sentiment leaf bypassed
shrinkage — 6 glowing syndicated reviews drove it to 100 (Δ+5). Gated the
aspect leaf at n≥20 (MIN_ASPECT_N); thin sets now ride the shrinkage-protected
star signal only (Numbuzin F3 100→53.6, Δ+5→−1). Golden PASS; the
substantive Target products are unaffected (all well above n=20).
Free-source coverage map is now complete and proven: Target (retailer head)
- Bazaarvoice (derm/pharmacy head) + Judge.me/Shopify (K-beauty tail, thin) — all reachable with built-in capabilities, no paid access. Sephora/Ulta/iHerb/ Amazon remain hard-walled but are largely redundant with the three open routes for scoring purposes.
Extractor reliability audit — run (2026-07-18)
scripts/audit-extractor.py over the 100-record sample ($1.64). Two reads:
- Determinism (frozen Haiku re-run vs itself): efficacy 85% exact / κ=0.73, aspects F1=0.79, context F1=0.80 — the transcription reproduces well.
- Accuracy proxy (independent Sonnet-5 rater, identical contract): context F1=0.79 / Jaccard 0.86 (strong), aspects F1=0.71 (moderate), efficacy 61% exact / κ=0.375 (fair — below the 0.7 target).
Read: the reviewer's stated context (which drives stratification — the whole
point) is transcribed reliably; aspect bucketing is acceptable; the efficacy
direction is the weak field — and it is also the highest-leverage one (F3
quantified sentiment + stratified %improve). Two things soften it: efficacy is
only scored where explicitly stated, and per-review disagreement partly averages
out over 200-review pools (systematic bias would not — random noise does). Still,
ext-1.1 must tighten the efficacy definition (the +/0/null boundary is where
Haiku and Sonnet diverge) before full-catalog scaling. Report:
formula/outcomes-extractor-audit-2026-07-18.md.
Open for the scale-up decision (P6)
- Coverage: Target-only reaches ~⅓ of the catalog's traffic-weighted head; the rest needs a browser-based fetcher or partner/API access (Bazaarvoice display keys, Yotpo/Okendo brand-site widgets) — pick before scaling.
- Extractor: formal human-agreement κ on the 100-record sample still to be run (operator read suggests it clears 0.7); context-capture rule ("pre-existing states only") should be tightened in the ext-1.1 prompt.
- Determinism re-extraction check (same reviews, second pass) not yet run (~$8).
- Outcomes-side UI (review receipts, stratified panels) — now has real data to design against, per the sequencing call.
2026-07-18 — P4 render COMPLETE (all surfaces wired behind the flag)
The remaining P4 render surfaces are wired and verified in both flag states (flag-on dev server renders grades+confidence; flag-off is byte-identical — 0 chip elements, legacy scores). Nothing user-facing changes until P5.
- Client cards (
src/scripts/product-card.js): grade + confidence chips matchingProductCard.astro, driven by field presence (the slim index only emitsgrade/gradeCond/confwhen the engine was on at build) — so no client flag needed and flag-off renders no chips. Verified: /products SSR shows 50 A + 10 B grade chips + 60 confidence chips with the flag on, 0 with it off. - Compare (
compare-data.js+compare.astro): flag-awareoverallvia the seam, a guarded Best Grade row (only when a product carries a grade), and the Data Confidence row shows the assessment band when on / legacydata_confidencewhen off. Newcmp-gradetier CSS. - My Skin: already renders through the shared
productCardHtml, so it inherited chips with no page change. - Routine Finder (
routine-index.js+RoutineFinder.jsx): indexoverallis flag-aware (so ranking uses engine scores when on) and carriesgrade/gradeCond/conf; result cards render a grade + confidence pill. - Best-of re-ranking (
scripts/rerank-best-of.py): re-orders each list's members by condition grade (general lists by Overall), B− floor, pins kept. Because ordering bakes intobest-of-lists.json(not a flag-gated render), it is a cutover data change — the script is dry-run by default and only writes with--writeat cutover. Dry run: 0 gutted lists (cutover gate holds), 933 members fall below the B− floor, condition lists shrink to their genuinely-qualifying members (e.g. aging 40→15) — the D3-accepted honest deflation, surfaced for the go/no-go, nothing written. - /methodology (
methodology.astro): flag-aware — the legacy four-dimension copy renders when off (unchanged); when on, a v4.3 section ("Two Scores, Built From Evidence") describes the six Overall factors, the Condition Grade formula with the Early badge, and the co-headline confidence band. Meta description flag-aware too.
P4 exit: every surface renders grade + confidence at parity behind the flag;
flag-off is byte-identical; golden test PASS; the only remaining pre-P5 items are
the outcomes-side UI (deferred to real data) and the ext-1.1 efficacy-prompt
tightening. P5 (flip the flag + rerank-best-of.py --write) remains gated on
explicit go.
2026-07-17 — P4 (in progress): render wired behind the flag
Phase P4 — wire the render. All wiring sits behind the default-off
ASSESSMENT_ENGINE flag: with the flag unset (every normal npm run deploy:build),
the site is byte-identical to today — legacy scores render exactly as before.
The flag flip is P5. So P4 remains reversible.
The accessor seam (schema §2)
src/lib/assessment.js — every surface reads scores/grades through it:
getAssessment / getScore / getGrades / getConditionGrade / getBestGrade / getConfidence + display helpers (confidenceChip, gradeTier, conditionLabel).
Flag off ⇒ getScore = legacy scores.overall, getGrades = null. Flag on ⇒
reads assessments.json (dynamic-imported once, only when on — no memory cost off).
Fails soft to legacy if the sidecar is missing.
Surfaces wired this increment
- Product page (
products/[slug].astro): newAssessmentPanel.astrorenders the full audit trail — overall + confidence co-headline (§9 parity), per-condition letter grades with reason-code/contested badges, active gates with their public record, and the chapters→factors→leaves decomposition (L1–L3,<details>per factor with source + confidence + INCI hits). Legacy score section wrapped in{!ASSESSMENT_ENGINE && …}as the defensive fallback. - Payload discipline (§2):
src/pages/assessment-data/[slug].json.jsserves one product's full tree on demand (emitted only when the flag is on); the sidecar is never shipped whole to a client. - ProductCard.astro (SSR cards, incl. best-of rows): score via the seam + a
grade-letter chip + confidence band at parity. Chip CSS in
product-card.css. - Slim card index (
product-index.js): flag-aware ordering +score/grade/ gradeCond/conffields so client-rendered cards reach parity.
Verification
- Flag-on
astro buildcompiles clean — the Vite SSR bundle built in 36.9 s with no CompilerError (one caught-and-fixed issue: a<=inside a JSX expression that Astro's parser read as a tag open → moved to a JS helper). Full page render running to completion. - Accessor logic validated against the real sidecar (golden
getBestGrade= A+ rosacea; gated product showscapped_by: G4; catalog best-grade tiers sane). npm run assessments:goldenstill PASS; flag-off path unchanged.
P4 polish increment (chemistry-side, durable) — done 2026-07-17
Addressed the review feedback that the panel showed low scores, technical field names, and no study links — the durable, chemistry-side half of the UI (won't change when outcomes land):
- Study "receipts" wired — new
src/lib/evidence.jsjoins the evidence table (196 rows) at build time; each condition grade now renders a "Show the studies (N)" expansion with real titles/journals/years and live PubMed links (verified in dev: azelaic→rosacea shows BJD 2019, Cochrane 2015, JAAD 2003, …). - Humanized every internal string — friendly leaf labels (
active_evidence_quality→ "Clinical evidence"), source pointers (inci-flag-rules:v1.0#fragrance→ "Ingredient-list analysis";prior:category→ "Baseline — no reviews collected yet"), and reason-codes → prose (azelaic-acid:evidence-well-established-for-rosacea→ "Azelaic acid has well-established evidence for rosacea."). Verified: zero raw field names / table pointers leak to the page. - Grade-forward emphasis — grades are now prominent cards (letter + prose + studies); the 0-100 demoted to a summary line; an honest note explains the Early badges ("graded on formulation & clinical evidence; user-review data not collected yet").
- Verified via flag-on dev server (HTTP 200 on product + gated pages, gate G4 shown humanized). Golden test still PASS; flag-off unchanged.
Outcomes pilot — scoped (not run)
formula/outcomes-pilot-plan.md — a 50-product pilot to prove collection +
extraction, get a real cost/accuracy read (~$10–50 est.), and see the de-compressed
score distribution before scaling. Requires building collect-outcomes.py +
finishing the engine's stubbed outcomes-consumption path. This is the lever that
lifts every grade off "Early".
Still remaining for full P4 (all safe behind the off flag)
- Client card renderer
src/scripts/product-card.js— render thegrade/confchips (data already ships in the slim index). - Compare, My Skin (condition-fit panels), routine-finder — grade + confidence parity.
- Best-of list re-ranking by condition grade + B− floor — a
score-best-of.pydata change (ordering is baked intobest-of-lists.json, not render). /methodologycopy refresh + reason-code prose throughaudit-boilerplate.py.- The outcomes-side UI (review receipts, stratified panels) — deliberately deferred until the pilot supplies real data to design against.
2026-07-17 — P1–P3: engine built, calibrated, frozen, shadow-wired
Phases P1 (build engine), P2 (calibrate + freeze — review gate), P3 (shadow build). Everything here is invisible to users and reversible ("stop reading the file").
P1 — the engine (scripts/build-assessments.py)
Full F1–F6 + two scored objects + gates + confidence + fingerprints + variant
families, writing src/data/assessments.json (76 MB, keyed by slug).
products.json is never touched. Reuses the ev/dose/fit core from
simulate-evidence-impact.py; adds the real INCI-flag engine
(inci-flag-rules.json) for safety/fit/craft/floor/gates.
- Golden test (
--golden): PASS. (1) The hand-authored exemplar inassessment-schema.jsonis internally arithmetically exact (overall = round(Σ score·weight); chapters sum; every grade's components reproduce its score; letters match the provisional boundaries it was authored against — the schema's_boundaries_notestates the letter map is provisional by design). (2) The engine's real output forthe-ordinary-azelaic-acid-suspension-10satisfies every §3 invariant and reproduces the deterministic grounded values from the frozen tables (azelaic→rosacea = 100 well-established, →acne = 75 promising, →hyperpigmentation = 52 emerging; fragrance-free ⇒ 0 fragrance deduction). - Catalog run: all 9,313 products assessed, coverage 100%
(
products_assessed == products_total), all §3 invariants hold on every record. ~45 s. 307 records share a variant family; 83 carry a derived gate (G4 = 41 sunscreens with no listed UVA filter; G5 = 42 antagonism / unstabilized-avobenzone). Confidence bands: all moderate at the overall level, all grades early (outcomes uncollected — by design).
P2 — calibration + frozen boundaries gb-2026.07 (scripts/calibrate-boundaries.py)
Recalibrated the 12 letter boundaries to the published shape targets against the
honest ev-2026.07-r196 grade distribution (n = 38,046; mean 69.8, sd 7.8), then
froze them into leaf-rubrics.json → grade_boundaries.frozen_gb_2026_07.
Interpolated in percentile space between the three published anchors.
| Boundary | A+ | A | A− | B+ | B | B− | C+ | C | C− | D+ | D |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 86.3 | 82.2 | 79.6 | 72.0 | 68.8 | 66.2 | 65.1 | 63.8 | 62.2 | 59.9 | 53.9 |
- Shape targets hit: A+ 2.1% (≈2%) · ≥A− 15.3% (≈15%) · ≥B− 65.1% (≈65%).
- Best-of survival under B− = 66.2: 0 lists gutted, 0 thin (<5) across all 210 lists (median 38 survivors). ✅ cutover gate.
- Movers vs legacy: mean −4.7 (sd 7.5); 73% deflate, 41% within ±5 — the expected, D3-accepted deflation from the inflated legacy labels.
- Engine now reads the frozen set (falls back to provisional only if unfrozen).
P2 — D2: condition-fit-matrix finalized
condition-fit-matrix.json promoted draft → finalized (4.3.0) on our own
dermatological review (2026-07-17). Every row reviewed against safety-leaf
semantics + clinical rationale; the draft coefficients were coherent and
defensible, so all values kept as-is. Dr. Thomas remains post-publish QA.
P3 — shadow build wiring (no render)
package.json: added assessments, assessments:golden,
assessments:calibrate, and deploy:build:shadow (generates the invariant-gated
sidecar, then builds — nothing renders it yet). deploy:build left
unchanged so the normal deploy path is untouched; the shadow build is opt-in.
src/data/assessments.json added to .gitignore (derived/regenerable, 76 MB).
Open items surfaced for the P4 go/no-go (not blockers):
- Curated gate register
src/data/scoring-gates.jsonis an empty stub — G1/G2/G3/G6 need real public-record entries before they can fire. - G4 fires on titanium-dioxide-only and incomplete-INCI sunscreens — a borderline
inci-flag-rules.jsonfilter-list stance (that file is still1.0.0-draft), worth a human/Dr.-Thomas call. Effect is aformula_qualitycap, notoverall. - Outcomes (F3 / grade Outcomes / Usability sensory) run on neutral priors
everywhere → every grade is
early. Lifts to Moderate/High as collection lands (P6).
Exit P1–P3: ✅ golden test passes · engine runs clean over all 9,313 with 100% coverage and all invariants · boundaries frozen · best-of 0-gutted · shadow build wired, nothing rendered. Stopping for go/no-go before P4 (render wiring).
2026-07-17 — P0: inputs locked, decisions D1–D3 recorded
Phase: P0 (Lock inputs + decisions — no code). Per scoring-cutover-plan.md §5.
Six engine inputs — resolved
| Input | Resolves to | Status |
|---|---|---|
| Evidence table | formula/ingredient-evidence-table.json (table_version: ev-2026.07-r196, 196 rows) |
✅ present |
| Leaf rubrics | formula/leaf-rubrics.json (4.3.1-draft) |
✅ present |
| Condition-fit matrix | formula/condition-fit-matrix.json (4.3.0-draft) |
✅ present — finalized in P2 (D2) |
| INCI flag rules | formula/inci-flag-rules.json (1.0.0-draft) |
✅ present |
| Claims vocabulary | formula/claims-vocabulary.json (2.0.0-draft, three-family) |
✅ present |
| Output contract | formula/assessment-schema.md + .json (1.0.0) |
✅ present |
Gate list src/data/scoring-gates.json (curated G1/G2/G3/G6 register) and the
outcomes store src/data/outcomes/<slug>.json are not cutover blockers:
absent gate file ⇒ only the rule-derivable gates G4/G5 fire; absent outcomes ⇒
category prior at Early confidence. Chemistry is computable for every product on
day one, so coverage goes 0→100% in one engine pass (schema §11 coverage gate).
D1 — "No admissible evidence for a claim" rule → Option B (neutral prior 50 + Early confidence)
When a (product, condition) grade has no evidenced hero for that claim (the evidence component cannot be computed), the evidence component takes a neutral prior of 50 and the grade is badged Early. It is never zeroed (Option A, which would falsely fail honest products) and never refused (Option C, which would gut best-of lists and leave blanks). Consistent with design rules 8 & 22 and methodology §3.3 ("the grade still exists; chemistry + fit are day-one computable"). This is the single highest-leverage call — it moves thousands of grades and best-of memberships.
- no-evidence (nobody ran the trial) ⇒ excluded from the evidence mean; if no hero contributes for the claim, the whole evidence component falls to the neutral prior 50 + Early.
- contradicted (a trial found no effect, tier-1/2 sourced) ⇒ scored at its 8-point tier and is gate-G2-eligible if it is the headline claim.
D2 — Finalize condition-fit-matrix ourselves → internal task, not an external gate
The fit matrix drives 30% of every condition grade. Its values are reviewed and finalized on our own expertise/analysis in P2, before render (P4). Dr. Thomas is post-publish QA only and never gates the cutover — we do not wait on him (consistent with the standing "Dr. Thomas = post-publish QA" guidance). The matrix finalization is tracked as an internal task; the P2 review is our sign-off.
D3 — Cutover scope & boundary shape → ship Overall + Condition grades together; expect deflation vs legacy
- Ship the Overall score and the per-condition grades in the same cutover — best-of ranking needs condition grades, so an Overall-first cutover would leave every "best for [condition]" list unranked.
- Letter boundaries are cutover-calibrated to the published shape targets
(A+ ≈ top 2%, ≥A− ≈ top 15%, ≥B− ≈ 65%) then frozen as
boundaries_version: gb-2026.07(methodology §3.5). - Grades will deflate vs legacy. Legacy LLM
scoresran mean 81.7 / sd 9.2 (inflated — every active looked good). Honest evidence has real spread; lower, better-calibrated grades are the intended outcome, made defensible by the co-headline confidence bands + the/methodologynote. Editorial framing for lower grades is accepted.
Architecture reaffirmed (schema §1)
The engine writes two sidecars and never touches products.json:
src/data/assessments.json (DERIVED, keyed by slug, regenerable) and reads
src/data/outcomes/<slug>.json (COLLECTED, durable input, absent for now).
Rollback through P3 is "stop reading the file." First irreversible-ish step is
P4/P5, gated by the cutover plan §7.
Exit P0: ✅ six inputs resolve; D1–D3 recorded here.
2026-08-11 — 4.7.0-pre: hair & nails groundwork (dandruff + nail-fungus claims)
- New condition claims
dandruffandnail-fungusinclaims-vocabulary.json(family: clinical), with endpoints (ASFS / mycological cure) and transfer guards (no scalp-psoriasis transfer; no tinea-pedis→nail-plate transfer).plausible-actives.jsonscopes the queue: 10 dandruff + 9 nail-fungus actives; 19 pairs staged informula/evidence-queue/(3 batches). RINSE_OFF_TRIAL_CONDITIONS = {"dandruff"}in build-assessments.py: the 4.6.1 contact-time rank ceiling is skipped for conditions whose clinical evidence is generated in rinse-off format — dandruff RCTs are timed shampoo-exposure trials, and ketoconazole 2% shampoo is first-line. Every other condition keeps the ceiling unchanged.shampoocounts as rinse-off ininci_facts(fragrance/alcohol penalties at rinse-off weights). New categories (shampoo, scalp-treatment, hair-care, nail-care) have no job specs yet — they deliberately ride the evidence/dose/craft path; revisit after the first ingestion canary.condition-fit-matrix.json: dandruff entry (fragrance/irritant/ preservative/harshness at 1.5) — scalp seb-derm is inflamed skin. nail-fungus stays on defaults (nail plate is dead; tolerability leaves mostly moot).- No re-run yet: zero catalog products carry the new conditions. First real grades appear after evidence rows + standing + phase-3 ingestion.