clock menu more-arrow no yes mobile

Filed under:

Study: Skating Judges Collude

An article from the Martlet points out a study that was recently published in the Journal of Sports Economics that figure skating judges alter their scores to conform with other judges' scores. The data was taken from figure skating world championships between 2001 and 2003.

From the article:

Author Jungmin Lee, a labour economics professor at Florida International University, found that judges try to choose scores close to their counterparts, especially when their scores for the previous skater were very different from their colleagues’. Judges try to pick similar scores because they are responding to criteria used to evaluate their own performance, Lee suggested, adding that this trend hurts the skaters who performed the best.

"The case of figure skating judging clearly shows that there is a bias toward agreement, because the degree of agreement among judges is used as a measure of reliability of the evaluations and to assess individual judges themselves," the study said. "Judges tend to rank skaters in accordance with preperformance information, putting less weight on the actual competition."

Lee also found that female judges are more generous on average than male judges, but that male figure skaters tend to receive higher scores than female and pairs skaters. Judges also tend to favour skaters from their own country.

Although the paper does not appear available online, you can find other of Lee's figure skating papers here. In Lee's words from his abstract: "I test for the existence of "outlier aversion" in which subjective evaluators avoid submitting outlying judgments. I find that judges manipulate scores to achieve a targeted level of agreement with the other judges."

It should be noted that the new Code of Points debuted in 2004, upon acknowledgment that the old judging system was unacceptable. I'm not sure if Lee has analyzed the new system but I would be quite interested in seeing what the results of that were.