February 17, 2010. The Return of the Phantom Judge. In 2006, Yale professor John Emerson demonstrated the role played by the computer's random selection of judges' scores in close figure skating competitions. With the start of the 2010 Olympic competition in Vancouver, the details may have changed but the story remains the same: close figure skating competitions may be influenced by the computer, not just the skaters' performances.

At the 2006 World Championships, the competition between Chinese gold medalists Pang and Tong, silver medalists Zhang and Zhang, and Russian bronze medalists Petrova and Tikhonov was so close that the medal standings were determined by the computer's random selection of judges' scores. Under International Skating Union (ISU) rules, the scores of three judges were discarded at random in the Short Program, and another set of three judges' scores were eliminated in the Free Skate.

Unfortunately for Petrova and Tikhonov, the randomness induced by the ISU rules (and not simply the skaters' performances on the ice) made a real difference. Emerson showed that the competition was close enough that different panels of judges could easily have awarded them the gold medal. Had the full set of scores from the complete panel of 12 judges been used, Petrova and Tikhonov would have won silver instead of bronze. Emerson's full analysis appeared in CHANCE Vol. 20, No. 2, 2007.

Fast-forward to 2010. The ISU has tweaked the system, but the problem remains. Judging panels now consist of nine judges (instead of 12), and scores of two judges (instead of three) are discarded at random. The role of the computer is unchanged – it's like a “phantom judge” in close competitions. Emerson has called for the ISU to follow the lead of U.S. Figure Skating, which uses panels of nine judges without any random selection. The ISU has acknowledged that the random selection of panels "may affect to a small degree" the overall score, and the 2010 system leaves open that real possibility.

In this week's Olympic Pairs competition in Vancouver, the medal standings weren't close enough to be influenced by the system. However, only a fraction of a point separated the 4th and 5th-place finishers, and the random selection of judges can influence scores by a full point or two. Emerson showed that the use of the complete panel of nine judges, and likely a number of alternate panels consisting of seven out of nine judges, would have changed the rankings. The actual panel used for scoring the Free Skate gave the lowest possible scores to Zhang and Zhang, and the panel used in the Short Program was among the worst for the Chinese. It was very bad luck – the “phantom judge” at work in Vancouver.

Emerson will repeat his analysis for the Men's and Ladies' competitions. He credits graduate students Taylor Arnold and Ankur Sharma for their assistance in processing the scoring sheets. For more information, see: Jay's web page.