| paper | d4-conf-ecai-2023-0082 |
| authors | — |
| venue | ? |
| filed under | unclassified |
| judged by | gpt-6-luna / xhigh (triple__luna__xhigh__c6r1) |
| judge confidence | high |
| authors would recognise it | yes |
Neither analyst identifies a numbered theorem, lemma, corollary, or proposition asserting an algorithmic, hardness, or computable-formulation result; the procedures are described and compared empirically. The grouped maximum-likelihood construction is a plausible new question, but the paper neither states nor analyzes it, and its predictor-training component needs additional model choices.
fails bit a — no named computational result to mirror
The paper presents EM, inferred and simulated consensus, a cutoff method, and experimental comparisons; none is stated as a numbered computational theorem to anchor a mirror.
The paper contains no numbered Theorem, Lemma, Corollary, or Proposition asserting a computational complexity result. Its EM equations and inferred and simulated consensus procedures are presented in Sections 3.1–3.3, and its main claims are empirical comparisons. I can’t honestly name one of those as a qualifying anchor, so there is no paper-anchored continuous problem to state or classify as tractable, hardness-transferring, or continuum-specifically hard.
A plausible mirror suggested by the paper’s subject would concern a high-volume stream of standardized cases. Let a type specify a case’s feature category and its pattern of which expert roles annotated it and what they reported; roles group experts with shared sensitivity and specificity. The mass of a type is its fraction of cases. Given rational masses for observed types and a fixed feature-model family, ask for a maximum-likelihood estimate of the latent class distribution and expert reliabilities, together with posterior class probabilities for each type. That preserves the paper’s central task—reconciling sparse, noisy expert labels while estimating reliability—and its outputs.
This scenario is most plausible when many cases share a finite feature schema and experts work in stable roles, so the number of cases is large relative to the number of distinct types. But the optimization problem is a proposed extension motivated by the algorithms, not a numbered result of the paper. The paper does not analyze its complexity, and I would not claim the authors established any tractability or hardness for it.
The strongest case against is that this paper gives no computational result to continuize. It states no complexity theorem or formally posed optimization problem. The EM, inferred-consensus, and simulated-consensus procedures are evaluated experimentally; the paper’s claims about their performance are not claims about worst-case complexity. So the proposed maximum-likelihood problem is a new research question motivated by the paper, not a continuous mirror of one of its results.
The proposed type-level likelihood also misses the paper’s distinctive machinery. Grouping cases by feature and annotation pattern gives weighted inputs for estimating latent labels and annotator reliability, much like a grouped Dawid–Skene model. But simulated consensus also trains a separate predictor for each annotator on that annotator’s labeled cases, then predicts on cases they did not label. If the type description is rich enough to preserve those feature relationships and training subsets, the task becomes running a chosen learning pipeline on weighted data; its computational difficulty comes from the model family. If it is coarse enough to discard those details, it no longer mirrors the algorithm. Neither route yields a complexity result from this paper.
There is a plausible stronger scenario: a large stream of cases drawn from a finite feature schema and handled by stable annotator roles. One could define a population version of simulated consensus over masses of feature-and-annotation types, with role-specific training on labeled strata and prediction on the complementary strata. That is coherent, and high multiplicity is not the problem. But with a fixed finite schema it looks like a weighted version of the existing pipeline; with richer features, the main computational question shifts to learning over that feature space. Either could support useful statistical work, but the paper gives no basis for expecting a new computational-social-choice landscape.
So the negative case is firm about the missing paper-anchored result, but weaker as a universal claim. A continuous statistical treatment may be worthwhile in its own right. What is hard to defend is that this paper supplies a computational result whose continuous mirror belongs to ChoCo’s stated programme.
The adversarial triple: the proponent anchors on up to three named results; the opponent sees that case and must defeat every anchor; the judge decides which case convinced it. These are the pipeline’s own outputs, generated by tools/triple_run.py — no human edited them. The paper’s own text is not reproduced here beyond the quoted statement above.