The ceiling imposed by two disagreeing annotators
Lax303562.AnnotatorCeiling · concepts/Lax303562/AnnotatorCeiling.lean · lax-303562
No public endorsements yet.
Loading review…
Sign in with ORCIDNatural Language Statement
Theorem
When the same image is annotated twice, with and instances, a prediction with instances can be scored against both. Its true positives against the first annotator are at most , and against the second at most . If the annotators disagree in their counts, , the total number of true positives stays strictly below the combined denominator , by a margin of : half the disagreement.
Since each matched IoU is at most , the combined numerator is at most the number of true positives, so the combined Panoptic Quality is strictly below for every prediction. The ceiling is set by the annotators, not by the model.
Concept map
Evidence
Lean source view on GitHub
| 1 | import Lax303562.PanopticQuality |
| 2 | |
| 3 | /-! |
| 4 | --- |
| 5 | title: The ceiling imposed by two disagreeing annotators |
| 6 | type: theorem |
| 7 | --- |
| 8 | When the same image is annotated twice, with `a` and `b` instances, a prediction |
| 9 | with `m` instances can be scored against both. Its true positives against the |
| 10 | first annotator are at most `a`, and against the second at most `m`. If the |
| 11 | annotators disagree in their counts, `a < b`, the total number of true positives |
| 12 | stays strictly below the combined denominator `m + (a + b)/2`, by a margin of |
| 13 | `(b − a)/2`: half the disagreement. |
| 14 | |
| 15 | Since each matched IoU is at most `1`, the combined numerator is at most the |
| 16 | number of true positives, so the combined Panoptic Quality is strictly below `1` |
| 17 | for every prediction. The ceiling is set by the annotators, not by the model. |
| 18 | -/ |
| 19 | |
| 20 | namespace Lax303562.AnnotatorCeiling |
| 21 | |
| 22 | /-- With `a < b` annotated instances and `m` predicted ones, the true positives |
| 23 | against both annotators are strictly fewer than the combined denominator. -/ |
| 24 | axiom ceiling_two_annotators (m a b tpA tpB : ℕ) |
| 25 | (hab : a < b) (hA : tpA ≤ a) (hB : tpB ≤ m) : |
| 26 | ((tpA : ℝ) + tpB) < (m : ℝ) + ((a : ℝ) + b) / 2 |
| 27 | |
| 28 | /-- The combined Panoptic Quality against two disagreeing annotators is strictly |
| 29 | below `1`, whatever the prediction. -/ |
| 30 | axiom pq_lt_one_of_disagreement (sA sB : ℝ) (m a b tpA tpB : ℕ) |
| 31 | (hab : a < b) (hA : tpA ≤ a) (hB : tpB ≤ m) |
| 32 | (hsA : sA ≤ tpA) (hsB : sB ≤ tpB) (hm : 0 < m) : |
| 33 | (sA + sB) / ((m : ℝ) + ((a : ℝ) + b) / 2) < 1 |
| 34 | |
| 35 | end Lax303562.AnnotatorCeiling |
| 36 |
Discussion
Ask a question or add context. Endorsements and structured flags are kept in the review panel above.
0 comments