I Will Have Order! Optimizing Orders for Fair Reviewer Assignment

Justin Payan, Yair Zick · IJCAI 2022 (ijcai22-00063)

no mirror
paperI Will Have Order! Optimizing Orders for Fair Reviewer Assignment
authorsJustin Payan, Yair Zick
venueIJCAI 2022
filed underfairalloc · indivisible
judged bygpt-5.6-luna / xhigh (triple__luna__xhigh__c22r1)
judge confidencemedium
authors would recognise ityes

Why no mirror

Proposition 3.2 clearly supplies the required computational result, and a finite-type, high-throughput review service is a credible multiplicity regime. The proposed order-and-welfare mirror remains recognizably RRR, so the opponent has not shown that no continuous analogue exists. But the fluid RRR allocation, reviewer-identity capacities, first-agent dynamics, and EF1-up-to-one-reviewer semantics are not yet canonically defined by the stated data.

fails bit none — no continuous question survives

The objection that survived

The proponent's appeal to replicated finite instances does not specify whether different microscopic orders and tie-breakings yield the same limit, or whether the attempted-set invariant and EF1-up-to-one-reviewer condition survive it.

fatal: False

Judge caveat

It is undecided whether a fixed finite-type replication scheme and tie-breaking yield a well-defined limit of RRR that preserves its attempted-set invariant and EF1 semantics, rather than defining a new fluid control problem. A precise limiting construction plus an equivalence result, or an explicitly accepted redefinition of RRR, would settle the grade.

What the mirror covers

The mirror targets Proposition 3.2's welfare maximization over RRR orders; it leaves Theorem 3.1 and Proposition 3.1 as unproved fluid correctness and completeness questions, while Theorem 3.2 remains only a possible approximation template.

Open questions for a prover

The case FOR (proponent)

My lead anchor is Proposition 3.2: “Maximizing welfare subject to round-robin (and RRR) is NP-hard.” This is asserted by Payan and Zick and proved in the full version, using techniques from Aziz et al.; it is not merely a result cited from elsewhere.

A natural continuous mirror is a high-throughput review service handling a very large stream of papers and a large standing reviewer pool. Papers are not treated as individually idiosyncratic: a paper type \(t\in T\) specifies its demand \(k\), its conflict and eligibility parameters, and its affinity \(a_{tq}\) for every reviewer type \(q\in Q\). A reviewer type specifies expertise, workload capacity, and any other property relevant to assignment. There may be \(N\gg |T|\) papers and \(M\gg |Q|\) reviewers, with \(\mu_t\) the fraction of papers of type \(t\), \(\nu_q\) the fraction of reviewers of type \(q\), and \(M/N\to\rho\).

This is plausible for a federated journal or conference service using a fixed subject taxonomy, discretized affinity scores, recurring reviewer roles, and standardized conflict categories. The claim is not that every real conference has this structure. The claim is that reviewer assignment has a credible high-multiplicity regime in which many papers are indistinguishable for the purposes of the mechanism, while the number of paper and reviewer types remains moderate.

The continuous problem I would propose is Continuous RRR-Order Welfare.

An instance consists of finite type sets \(T\) and \(Q\), rational distributions \(\mu\) and \(\nu\), a reviewer-to-paper population ratio \(\rho\), a common paper demand \(k\), affinity values \(a_{tq}\), and reviewer load limits \(u_q\). Think of the reviewer identities of type \(q\) as an atomless pool of total mass \(\rho\nu_q\), each identity able to review at most \(u_q\) units of paper mass.

A solution chooses a measurable order \(\pi:[0,1]\to T\) satisfying \(\lambda(\pi^{-1}(t))=\mu_t\). Thus \(x\in[0,1]\) is a paper position in the order and \(\pi(x)\) is its type. The order is then fed into a fluid version of RRR: in each of \(k\) rounds, sweep through \(x\in[0,1]\), assigning the highest-affinity still-admissible reviewer identity to the paper at \(x\), using the same capacity, distinctness, and EF1-preservation test as Algorithm 1 in the paper. Let \(r_\ell(x)\) be the reviewer identity assigned at round \(\ell\), and let \(q(r_\ell(x))\) be its reviewer type.

The allocation must satisfy reviewer capacities, each paper position must receive \(k\) distinct reviewer identities, and the resulting bundles must satisfy continuous EF1: for every pair \(x,y\), there must be some reviewer \(\ell\) in \(y\)’s bundle such that \(x\)’s value for \(y\)’s remaining bundle is no greater than its value for its own bundle. Formally, \(\sum_{h\ne\ell}a_{\pi(x),q(r_h(y))}\le\sum_{h=1}^k a_{\pi(x),q(r_h(x))}\).

The objective is to maximize normalized utilitarian welfare \( \operatorname{USW}_\infty(\pi)=\int_0^1\sum_{\ell=1}^k a_{\pi(x),q(r_\ell(x))}\,dx \). The decision version asks, given a rational threshold \(W\), whether some continuous paper order produces welfare at least \(W\). A solution is the order \(\pi\) together with its induced measurable RRR allocation; an \(\varepsilon\)-solution has welfare within \(\varepsilon\) of optimum.

This is not merely a fractional matching problem. Each infinitesimal paper still receives a discrete bundle of \(k\) distinct reviewer identities, and the order remains the mechanism’s central decision variable. Mass only records how much of each paper and reviewer type exists. That makes it recognizably the same question as Proposition 3.2: optimize welfare by choosing the order in which paper-agents invoke round robin or RRR.

For the genuinely high-multiplicity regime with fixed or moderately sized \(T\), \(Q\), and \(k\), I would expect this problem to have a Class A slice. The remaining reviewer-slot capacities form a finite-dimensional fluid state, while the order is a control over type mass. A discretization of \([0,1]\) gives a finite type-position scheduling problem. Theorem 3.2 suggests a possible approximation route: if the fluid welfare function remains monotone and \(\gamma\)-weakly submodular with a controlled \(\gamma\), the greedy order argument could yield a \(1/(1+\gamma^2)\)-type guarantee. The theorem itself does not establish this continuum extension, but it gives a concrete technical test rather than a vague analogy.

The expected tractability is therefore conditional on the high-multiplicity structure. If the number of affinity and reviewer types grows with the input, the problem may instead be Class B: the combinatorics could survive in the type-level affinity matrix and reviewer-resource structure rather than in the number of repeated papers. Determining whether the discrete NP-hardness of Proposition 3.2 survives this fluidization, or disappears when \(N\) grows with \(T\) and \(Q\) fixed, is precisely the useful complexity question.

This mirror covers Proposition 3.2 and the paper’s central optimization target, but not every result. Theorem 3.1 and Proposition 3.1 concern correctness and completeness of RRR; their continuous counterparts would be feasibility and EF1-preservation theorems. Theorem 3.2 supplies a possible approximation framework, but I would not claim it as a second anchor.

The weakest point is that RRR’s proof is intrinsically discrete in two places: the “up to one item” condition and the attempted-set argument for named reviewers. In a fluid model, one must define the measurable limit carefully, especially when reviewer capacities become tight or many identical papers are interleaved. Also, requiring papers to have identical affinities to every reviewer type is a substantial modeling assumption.

That weakness does not destroy the mirror, because it identifies the right boundary. A rigorous construction should define Continuous RRR as the limit of finite replicated instances, preserve reviewer identities inside each reviewer type, and prove convergence and EF1 under suitable capacity slack. If that limit fails to preserve the RRR invariant, the correct conclusion is that the continuous analogue is a fluid round-robin mechanism with a modified fairness notion—not that reviewer-order optimization has no continuous form.

The case AGAINST (opponent, writing after the proponent)

Proposition 3.2 is the strongest possible anchor, and the negative case against it is necessarily limited. It is a genuine computational result, and reviewer assignment does admit plausible high-multiplicity stories. The best objection is structural rather than complexity-theoretic.

RRR’s order is an order of individually valued papers consuming indivisible, named reviewers. A paper’s complete type must include its entire affinity and conflict profile, while the RRR invariant depends on which named papers previously attempted which named reviewers. If those profiles are sufficiently heterogeneous to preserve the paper’s order-optimization problem, then almost every paper is its own type; the putative society has masses \(1/N\), not meaningful multiplicities. If profiles are collapsed into a small number of recurring types, the individual permutation disappears. What remains is a schedule of type masses, not the paper’s optimization over paper orders.

The proposed fluid RRR also has no canonical limit. A measurable order \(\pi:[0,1]\to T\) has no first paper, whereas RRR’s assignment and EF1 test are defined by a finite prefix of named agents. Replicating finite instances does not resolve this: one must additionally specify a limiting sequence of microscopic orders, or replace it by a measurable control. Different such controls can have different reviewer-capacity trajectories even when they induce the same type distribution \(\mu\).

Every repair changes one of the paper’s essential objects. If reviewer identities and finite bundles are retained, the decisive structure remains atomic and identity-level; the distribution is merely bookkeeping. If reviewers become mass, then “remove one reviewer” has zero mass, distinctness becomes vacuous, and EF1 turns into a different fairness condition. If one retains discrete bundles through a distribution over bundle types, the central state is no longer the society distribution but a history-dependent allocation measure. The result may be an interesting fluid matching or control problem, but it is no longer a continuous mirror of Proposition 3.2’s RRR-order problem.

That said, this is not a decisive universal refutation. A federated review service with genuinely repeated paper, reviewer, affinity, and conflict types could support a defensible type-level scheduling model. The honest negative conclusion is therefore only that the paper does not supply a clean, canonical continuization: its best proposed mirror is a substantial new hybrid mechanism whose relation to RRR must be created rather than inherited.

The adversarial triple: the proponent anchors on up to three named results; the opponent sees that case and must defeat every anchor; the judge decides which case convinced it. These are the pipeline’s own outputs, generated by tools/triple_run.py — no human edited them. The paper’s own text is not reproduced here beyond the quoted statement above.