headerphoto

Evaluation of Administrative Officials

Normative prediction markets help neutralize ideology by making recommendations that are ideologically balanced and independent of incumbent officials. An alternative strategy for reducing the impact of ideology on administrative decisions would focus on inputs rather than outputs, on the personnel who make decisions rather than the decisions those personnel make. A reasonable goal of an appointment process should be to produce officials who are moderates rather than ideologues and who have high degree of technical competence in the relevant fields. Existing approaches for the selection of administrative and judicial officials, however, might not adequately seek to advance these goals.

Consider, for example, the U.S. Constitution’s Appointments Clause. The Senate’s confirmation power places a check on the president, although when the president’s ideology is aligned with the Senate’s, this check seems unlikely to lead to many appointments of officials in the middle of the political spectrum. Many legislators, meanwhile, believe that there should be some degree of deference to the president, absent an appointee who is unqualified or “out of the mainstream.”28 This attitude often reflects a view that it is normatively beneficial for the president to be able to choose officials of the same ideology, even for nominally independent agencies and for independent judges. This view confuses the inevitability of large ideological shifts based on small shifts in the electorate’s preferences with the desirability of such shifts. If no one held this view, the system still would not be conducive to appointment of genuine moderates. One reason to grant substantial deference to the president is that positions need to be filled and stalemate can be destructive.

Appointments in the United States thus depend on the degree to which a prospective nominee’s views match the president’s and the prospective nominee’s qualifications. And the old joke about a judge being someone who knows a senator suggests that connections matter, too. Alternative appointment systems seek to focus on merit and thus minimize ideological criteria and connections. One approach, used by countries including Germany to select judges, is to base appointment in large part on an examination that seeks to identify the most legally knowledgeable test-takers.29 A plausible objection to this approach, however, is that even if an exam is unbiased, it may not be the only relevant variable. Other factors such as experience, demeanor, and judgment might also be relevant, and some of these cannot be assessed entirely mechanically. The intellectual challenge is to develop a system that can take such considerations into account in a truly nonideological way.

Some institutions appear to be relatively successful at nonideological personnel selection. For example, in the United States, the Office of Personnel Management administers the system for the selection of administrative law judges (ALJs).30 Although the system is called an “examination,” the score depends not only on a written test but also on qualifications, a panel interview, and a check of references. The absence of complaints from politicians about this system suggests that it has not become greatly politicized. Nonetheless, there are two reasons to think that the model is not easily transferred to other contexts. First, if ALJs were more important– as important, say, as federal appellate judges or administrative agency heads–it seems likely that the system would be politicized, and the president would seek to install OPM officials with an ideological agenda. Second, although focusing only on the merits tends to reduce ideology, a system that focused directly on ideology might in theory be able to produce more mainstream applicants.

Prediction markets could provide an alternative approach to rating prospective administrative officials or ALJs. A normative prediction market could be used to forecast two evaluations by ex post decision makers, again perhaps a decade away, of the overall quality a particular candidate, conditional on the candidate’s appointment. The ex post decision makers in one market would consist only of Republican members of the legislature (or their designees or some other pool of relatively conservative individuals). The parallel market would consist of Democrats. The greater the degree of divergence in expected evaluations between Democrats and Republicans, the more ideological the candidate.

Meanwhile, the two predictions could be combined to provide an assessment of overall quality independent of ideology. They might simply be averaged, or they might be combined with weights based on a prediction market forecast of the future Democratic-Republican balance in Congress or perhaps in the population at large. A somewhat more elaborate system would use simple statistical methodologies to untangle ideology from assessments of merit. For example, a regression model could be used to predict a particular nominee’s average rating by the Democrats and Republicans as a function of the candidate’s predicted ideology. The error term for each observation of the regression would identify the corresponding candidate’s score controlling for ideology, so more qualified candidates should receive higher scores.

It is true that similar approaches might be used without the benefit of prediction markets and perhaps could represent an improvement. Members of the legislature could simply be asked to rate individual candidates, and the above-mentioned techniques could be used. Indeed, political scientists sometimes look at the difference between the votes of the two political parties to produce a proxy for the ideology of nominees.31 There are, however, at least two advantages to the use of prediction markets. First, they provide incentives for third parties to gauge the performance of potential candidates and thus to provide information to the legislature. Second, ex post evaluations that have no function other than to resolve the prediction market payouts are likely to be less politicized than evaluations that determine who is selected. With contemporaneous evaluations, actual quality might factor very little into the evaluations, and so numbers produced using the regression method would be very noisy.

The weight ultimately placed on the ideology measure (with more neutral candidates more likely to be selected) and the weight placed on the quality measure could be disputed. But assuming the continued use of existing appointments systems, prediction markets along these lines at least might provide information that could help influence the political debate, once the relative accuracy of such markets is established. It will be difficult to defend someone as being within the mainstream if prediction markets say that the person is not or to claim that someone is unqualified if prediction markets forecast high quality ratings. Of course, prediction markets also could be used if being ideological were viewed as a positive attribute. For example, it may be better in a multimember decision-making body such as the Federal Communications Commission to ensure that the median member is relatively moderate but that there are also more ideological members to ensure viewpoint diversity.

Normative prediction markets for gauging prospective appointees might be useful not only in selecting people for positions in administrative agencies but also in selecting people who might serve as ex post decision makers in other such markets. The premise of the normative prediction market proposals offered so far is that because the identity of the decision maker is unknown, the market will tend to produce relatively average decisions. This approach increases the risk cost borne by market participants, raising the cost of the market subsidy needed to obtain any given level of accuracy. The alternative is to commit to appointing the decision makers through a normative prediction market designed to select relatively moderate ones. Quality should matter, too, but not as much since nonsystematic analytical errors they make will tend to cancel each other out in expected value terms. Selecting ex post decision makers in this way might be particularly beneficial if it is impractical to wait a decade or so before resolving a normative prediction market.

 

Leave a Reply