Panoptic Quality: exact rules for inclusion, removal and duplication, the sharpness of the matching threshold, and the annotator ceiling

lax-303562·formalized by Joel Cruz Cabrera @joelcanary·registered·created ·GitHub @4db471d·Lean v4.33.0 epoch · mathlib db584cd6d46c

Loading review…

Sign in with ORCID

Community review

Flags

Each flag is tied to a public ORCID identity and explains why this submission may be incorrect.

No flags have been submitted.

    Community review

    Flag this submission

    State precisely what appears incorrect. This explanation will be public under your ORCID name.

    Abstract

    Panoptic Quality (Kirillov et al., 2019) scores an instance segmentation as the sum of the matched intersection-over-union values over TP+12FP+12FN|TP| + \tfrac12|FP| + \tfrac12|FN|. This submission treats the metric as a function of those four counts and proves the exact identities that govern how it moves when one prediction is added, removed, or duplicated.

    The inclusion rule: a candidate that matches with probability pp and IoU jj raises the expected score if and only if pj>PQ/2p\,j > \mathrm{PQ}/2, because the denominator rises by exactly 12\tfrac12 whether the candidate matches or not. The removal rule is its mirror image, and the two are inverse. Under evaluators that accumulate every pair above the threshold instead of a one-to-one matching, a duplicate of a matched prediction with IoU vv raises the score if and only if v>PQv > \mathrm{PQ}, and no reweighting of false positives and false negatives removes this incentive; under one-to-one matching the same duplicate is a false positive and lowers the score exactly when false positives carry positive weight.

    Two further statements concern the matching itself and the data. With the strict threshold IoU>12\mathrm{IoU} > \tfrac12 a prediction matches at most one of any family of disjoint annotations, and the threshold is sharp: at 12\tfrac12 a prediction can tie with two. When two annotators of the same image disagree in their instance counts, the Panoptic Quality of any prediction scored against both is strictly below 11, by a margin of half the disagreement.

    All statements are elementary and their proofs are short; their value is that a competition, a benchmark, or a paper can cite the exact condition rather than an intuition. They were used to select the operating point of a solar-filament segmentation pipeline and to diagnose a metric-inflating evaluator.

    Concepts

    Concept map
    7 concepts
    100%
    Proven claimDefinitionThis submissionA → B: B builds on A

    Proofs

    Proof networkview on GitHub

    100%
    assumptions conclusionProven claimStatement 1, 2, … of a claim with several statementsClaim from this submissionProof — open large view for details
    Proof list

    Lean sources for these proofs: proofs/ on GitHub

    Proof code is not displayed; the archive records each proof's checked relationship between claims.

    Related submissions

    No other submission in the archive builds on this one, and this one builds on none.

    Cite this

    This is only the formalizers. The authors of the formalized results may be different (see References).

    @misc{lax-303562,
      author = {Joel Cruz Cabrera},
      title = {Panoptic Quality: exact rules for inclusion, removal and duplication, the sharpness of the matching threshold, and the annotator ceiling},
      year = {2026},
      howpublished = {Lax Archive, lax-303562},
      url = {https://laxarchive.org/lax-303562/},
    }

    Discussion

    Ask a question or add context. Endorsements and structured flags are kept in the review panel above.

    Loading discussion…