How these files are graded

Every check is deterministic, cites the CMS hospital price-transparency rule (45 CFR part 180, in particular § 180.50) or the CMS schema documentation, and can be explained in one sentence. There is no model anywhere in the grading path.

What is checked

Five independent dimensions per file: retrievability (one identified, bounded download attempt), conformance (selected CMS v3 envelope or general-element, structure, and accepted-value checks), completeness (presence and usability of the fields that make a rate interpretable), interpretability (whether rates are usable amounts or require separate treatment), and freshness (the file's own last_updated_on against the assessment date). JSON publications are assessed under the CMS JSON v3 profile and CSV publications under the CMS CSV v3 profile (Tall and Wide); the two are never pooled. The complete finding catalogs with citations are in docs/how-we-grade.md.

What is deliberately not checked

• This is not the official CMS validator and not exhaustive schema validation; an A means the implemented checks emitted nothing, not that the file is valid.
• No determination of legal compliance with 45 CFR part 180 or any other law.
• No verification that published prices, payer names, or attestations are factually accurate.
• No price comparison of any kind: dollar, percentage, and algorithm rates are structurally separated, and no rate statistic is published without the suppression and uncertainty work this project has not yet done.
• No long-run availability measurement: retrievability reflects one bounded attempt from one vantage on one date.

The letter grade

In evaluation order: a failed download is an F with the dated reason; a local limit (invalid input, the project's size ceiling, cache trouble) is not graded and never conflated with an F; an incomplete stream is an F; then errors (or missing evidence) in zero dimensions is an A (warnings make it a B), one dimension a C, two a D, three or more an F. Tolerated INFO observations never lower a grade. Each cohort names the exact policy its grades were minted under below.

Retrieval conduct

Hosts' robots.txt files were checked for every retrieved URL before the run; a site that blocks is recorded and skipped, never circumvented. Retrieval of these files is what the transparency rule exists to allow: CMS requires them to be public, machine-readable, and accessible without barriers.

The CMS hospital JSON v3 cohort (2026-09-12)

Grade policy cms-hospital-json-v3-file-grade-v1, fingerprint df7943ac4f005297…; the full rule table ships in this cohort's machine-readable comparison.

One serial, operator-invoked run of `mrf-honest scorecard` per target under the default retrieval policy: identified User-Agent with a contact address, HTTPS only, robots.txt fetched and obeyed in code before every request, a per-host minimum interval, conditional revalidation against the verified cache, retry backoff, and the default 1 GiB decoded-size ceiling. The policy is byte-identical to the one 2026-08-19 was collected under -- same fingerprint -- which is what makes the two cohorts comparable. Every target's URL was re-resolved from its own origin's `cms-hpt.txt`, retrieved the same day, rather than replayed from the earlier cohort: three of the seventeen are published today at a URL that is not the one graded on 2026-08-19, because the HCA storage account those three sit behind rotated its shared access signature, and each was fetched at the URL its hospital publishes now. Of the seventeen attempts, 9 revalidated against the verified cache and returned HTTP 304, moving no body bytes; 7 transferred a body; 1 answered with an HTTP error and is an F with the dated reason.

How these subjects were chosen

Two strata, each a complete enumeration, carried forward unchanged from 2026-08-19: this cohort re-measures those subjects and draws nothing new. Stratum A is every subject published in the 2026-08-14 cohort (6 files, 4 systems) and is a convenience sample. Stratum B is the seeded uniform random sample of 48 facilities drawn on 2026-08-19 from the 3024 acute-care, non-federal, 50-states-and-DC hospitals in the CMS Hospital General Information dataset; every drawn facility was attempted then and every outcome is recorded. The draw itself is dated 2026-08-19 and is not re-run here -- re-drawing would make the two cohorts describe different hospitals and the change between them unreadable. Cohort-wide totals mix the two strata and are descriptions of this cohort, not estimates. docs/SAMPLING-FRAME.md states it in full.

A target is graded only where the publication is a CMS hospital JSON document. Format is read from the publication, never from a failed request: the URL's file extension decides where it has one; where it has none, a specific declared media type decides; where the declaration is generic, the retrieved document decides. A hospital publishing a conforming CSV is not a hospital with a problem, and is recorded as an exclusion rather than graded against the wrong profile. A .json publication whose retrieval failed stays in the cohort and is graded F with the dated reason.

Finding codes emitted in this cohort

The CMS hospital CSV v3 (Tall and Wide) cohort (2026-09-12)

Grade policy cms-hospital-csv-v3-file-grade-v1, fingerprint eadf930dd97d123b…; the full rule table ships in this cohort's machine-readable comparison.

One serial, operator-invoked run of `mrf-honest scorecard --profile csv` per target under the default retrieval policy: identified User-Agent with a contact address, HTTPS only, robots.txt fetched and obeyed in code before every request, conditional revalidation against the verified cache, retry backoff, and the default 1 GiB decoded-size ceiling. The policy is byte-identical to the one 2026-08-19 was collected under, which is what makes the two cohorts comparable. Every target's URL was re-resolved from its own origin's `cms-hpt.txt`, retrieved the same day; twenty-four of the twenty-five are published today at the same URL, byte for byte, as on 2026-08-19. The exception is Whitfield Medical Surgical Hospital: msh.ms.gov's robots.txt answered HTTP 301 on 2026-09-12, so under RFC 9309 2.3.1.4 this tool could not retrieve that origin's cms-hpt.txt at all and the origin no longer declares the file. The file itself is still served at the URL the 2026-08-19 TXT published, which is the URL this row was retrieved from, and that discrepancy is stated here rather than hidden behind an unchanged grade. Of the twenty-five attempts, 13 revalidated against the verified cache and returned HTTP 304, 7 transferred a body, 2 were refused by robots.txt, 2 exceeded this project's own size ceiling, and 1 answered HTTP 404.

How these subjects were chosen

One stratum, a complete enumeration: every facility of the committed 2026-08-19 seeded random draw (48 facilities from CMS's enumeration of 3024 acute-care, non-federal, 50-states-and-DC hospitals) whose cms-hpt.txt publication the sibling JSON cohort recorded as a CSV-retrievable format_outside_profile exclusion - 25 of the 32 format exclusions; the other 7 are ZIP archives and remain excluded there. Every target is a row in this cohort, graded or stated. Totals describe this cohort, not hospitals as a class, and must never be pooled with the JSON cohort. This cohort re-measures those same 25 targets on 2026-09-12; nothing was re-drawn and no target was added or dropped. docs/SAMPLING-FRAME.md states it in full.

A target is graded only where the publication is a CMS hospital CSV document (Tall or Wide). Format was read from the publication during the sibling cohort's review, never from a failed request: the URL's file extension decides where it has one; where it has none, a specific declared media type decides; where the declaration is generic, the retrieved document decides. A ZIP archive is a container, not a CSV file, and grading one against this profile would measure the wrong thing; the 7 ZIP publications remain recorded exclusions of the sibling cohort. A .csv publication whose retrieval failed stays in this cohort and is graded F with the dated reason.

Finding codes emitted in this cohort

A grade describes one published file under one stated policy on one date. It does not rank hospitals, price care, or determine compliance with 45 CFR part 180, and it is not the official CMS validator.