| paper | d4-conf-aaai-2024-0137 |
| authors | — |
| venue | ? |
| filed under | unclassified |
| judged by | gpt-6-luna / xhigh (triple__luna__xhigh__c6r1) |
| judge confidence | high |
| authors would recognise it | unclear |
The paper presents MBO-APL in Algorithm 1 and reports empirical results, but contains no numbered Theorem, Lemma, Corollary, or Proposition asserting a computational result. The classifier-selection question is a proposed extension, not a computational result of this paper, so it cannot anchor the required mirror.
fails bit a — no named computational result to mirror
The suggested extension draws on the fairness-aware classifier hyperparameter experiment in §5.2.3; it does not cover a named computational result.
There is no anchor in the required sense. The paper contains no numbered Theorem, Lemma, Corollary, or Proposition asserting that a problem is in P, NP-hard, FPT, or otherwise classifying its complexity. Algorithm 1 proposes MBO-APL, and Sections 5.1–5.2 report experiments, but neither is a named complexity result. Equation (1) states the underlying optimization problem; it is not an anchor. So the paper supplies zero qualifying anchors.
The best positive case I can make is an unanchored mirror of the paper’s preference-based classifier-selection problem. Consider a public service choosing one classifier configuration for a large recurring population. A type records everything relevant to the metrics under consideration—for example, a person’s protected group, label, and prediction response across the candidate configurations. If \(n\) people fall into only \(\tau\) such types, let \(\mu_t\) be type \(t\)’s share, with rational masses summing to one. For configuration \(x\), population accuracy and group fairness metrics are computed from these masses; call the resulting vector \(f_\mu(x)\). A single decision maker retains the paper’s preference model, for example
\[
U_w(f)=\min_{\ell\in[L]}\frac{f_\ell}{w_\ell},
\qquad
w\in\Delta^\circ_{L-1},
\]
and supplies pairwise comparisons or improvement requests as in the paper.
This yields a concrete problem: given a finite set \(X\) of candidate configurations, rational type masses \(\mu\), noisy expensive evaluations of \(f_\mu(x)\), and access to the same preference queries, find an \(\hat x\) such that
\[
U_w(f_\mu(\hat x))\geq \max_{x\in X}U_w(f_\mu(x))-\epsilon
\]
with probability at least \(1-\delta\), while minimizing the tradeoff between classifier evaluations and decision-maker queries. The population size \(n\) need not drive the computation when evaluation can use the \(\tau\) type masses and their metric contributions. I would expect this finite-candidate, bounded-noise version to be tractable in a query-complexity sense. That is a forecast about the proposed problem, not a result of this paper.
This is a plausible extension of the paper’s own fairness-aware hyperparameter experiment in §5.2.3: the shared decision remains choosing \(x\) to maximize a preference over accuracy and fairness, while the evaluation population becomes an explicit distribution over types. I would scope the mirror to that classifier-selection setting and the paper’s core preference-based optimization framework, not to every experiment.
The weak point is substantial: the paper treats the metric vector as an expensive black box and has one decision maker; it does not make the people whose data determine the metrics into a population model. Making their type distribution explicit may look to the authors like a more detailed evaluation model, rather than a new computational question. And without a qualifying named result, this mirror cannot support the requested anchored case for continuization.
The proponent’s audit is decisive: this paper has no named result classifying the complexity of a computational problem. It proposes a Bayesian optimization algorithm and reports empirical performance; it proves no complexity theorem for that algorithm or for a social-choice problem. So there is no result here to anchor a continuization. Any candidate would have to introduce a new computational question.
The strongest candidate is the proponent’s classifier-selection scenario. It is plausible to represent a large evaluation population by types and rational masses, then compute accuracy or fairness metrics from those masses. But those people supply data for evaluating a classifier; they do not express preferences that the method aggregates. The paper has one decision maker, who supplies pairwise comparisons or improvement requests about objective vectors. Recasting the evaluation data as a distribution does not turn that problem into computational social choice.
It also does little to change the paper’s computational question. For a fixed classifier configuration, metrics such as accuracy can be calculated by summing type contributions, and fairness measures by taking ratios of such sums. That may speed up metric calculation when many records share a type. But the proposed mirror still has a finite set of candidate configurations, expensive noisy evaluations, and one decision maker. Without additional assumptions, finding the best candidate requires queries governed by the candidate set and the objective functions’ noise and structure; the type masses alone do not reduce that query burden. If the proposed gain is faster classifier training or fewer evaluations because of repeated types, that needs a new model and analysis absent from the paper. If instead the types become voters whose preferences are aggregated, the problem has changed.
This leaves a useful adjacent question about evaluating classifiers over large populations, but not a strong continuous mirror of this paper’s computational contribution. I would not claim to prove that no imaginable reformulation could be worthwhile. The negative case is strongest on scope: the paper’s population is an evaluation dataset, while its decision problem is single-user black-box optimization, and neither its results nor its main computational burden turns on population multiplicity.
The adversarial triple: the proponent anchors on up to three named results; the opponent sees that case and must defeat every anchor; the judge decides which case convinced it. These are the pipeline’s own outputs, generated by tools/triple_run.py — no human edited them. The paper’s own text is not reproduced here beyond the quoted statement above.