| paper | d4-conf-aaai-2020-1595 |
| authors | — |
| venue | ? |
| filed under | unclassified |
| judged by | gpt-6-luna / xhigh (triple__luna__xhigh__c6r1) |
| judge confidence | high |
| authors would recognise it | yes |
The paper contains no numbered theorem, lemma, corollary, or proposition asserting a computational result; its score comparisons and ranking correlations are empirical findings without a stated guarantee. The population version of annotator score aggregation is a plausible extension, but it is not a continuous analogue of a qualifying computational result in this paper.
fails bit a — no named computational result to mirror
No qualifying named result is covered; the closest lead is Section 4's aggregation of binary annotations, while the dataset analysis and BERT ranking experiments have no qualifying computational anchor.
There is no qualifying named anchor in this paper: it contains no numbered Theorem, Lemma, Corollary, or Proposition asserting a computational complexity result. The closest results are empirical comparisons: Section 4 compares the MACE-P and WA scoring functions, and Section 6 reports ranking correlations for BERT models. Those are useful findings, but they do not establish that an underlying problem is in P, NP-hard, FPT, or similar. So the anchor set is empty, and I cannot give a compliant per-anchor continuous problem or predict its place in the trichotomy.
The nearest plausible mirror, if treated as an unanchored lead, is crowd aggregation. Fix a topic and stance and a set \(A\) of arguments. Let each evaluator type \(t\) specify a complete binary judgment vector \(b_t:A\to\{0,1\}\) and a reliability weight \(r_t\); let \(\mu_t\) be that type’s share of a large evaluating audience. For each argument \(a\), compute
\[
q_\mu(a)=\frac{\sum_t \mu_t r_t b_t(a)}{\sum_t \mu_t r_t},
\]
then return the arguments ordered by \(q_\mu\). A plausible regime would be a large civic consultation whose many participants fall into a moderate number of stable judgment and reliability types. This is a natural population version of the paper’s binary-annotation scoring task, and its computation is straightforward. But it changes the focus from the paper’s text-based BERT ranking experiments to score aggregation, and the paper provides no named complexity result for that problem. That missing anchor is the case’s weakest point—and under the requested standard, it prevents this from being a supported positive case.
The strongest case against a mirror is that there is no named computational result to mirror. The paper’s contributions are a dataset, empirical comparisons of ways to turn annotations into scores, and neural ranking results measured by correlation. Its scoring experiments and BERT results are findings, but the paper does not pose or classify a computational problem such as bribery, control, or winner robustness. So the proponent’s anchor set is empty.
Crowd aggregation is the closest lead, and it deserves the strongest version of the case. A large civic audience with repeated evaluator types could be modeled by a distribution \(\mu\); the paper’s weighted-average score becomes the corresponding population average for each argument. That is a plausible continuous-population model, and the authors might recognize the connection to their label-induction pipeline. But the pipeline uses those labels to train a text-based predictor. Replacing its finite annotators with evaluator types changes the annotation regime; it does not continuize the paper’s main ranking task or mirror a computational result it establishes.
A stronger proposal might study how rankings change when mass shifts between evaluator types, or how to manipulate that mass. Those would pose computational social-choice questions, but they add an intervention objective the paper does not study. Alternatively, treating arguments as the population would retain the text-specific objects the ranker must score; grouping them into types would generally discard distinctions the task depends on. Neither route supplies a clear continuous mirror of a result in this paper.
The negative case is not airtight: a continuous evaluator population is sensible, and a study of its effect on label induction could be useful. Its weakness is that the paper offers no computational question there to carry into the programme. That makes the proponent’s lead an adjacent research direction, not a supported anchor.
The adversarial triple: the proponent anchors on up to three named results; the opponent sees that case and must defeat every anchor; the judge decides which case convinced it. These are the pipeline’s own outputs, generated by tools/triple_run.py — no human edited them. The paper’s own text is not reproduced here beyond the quoted statement above.