RateMyProfessors.com

2008 Top 10 Ranking Methodology

RateMyProfessors.com is the Internet's largest destination for collegiate professor ratings with over 6,500 schools, 1,000,000 professors and 8,000,000 ratings. The site uses a five-point Likert scale as well as a binary scoring system for students to rate professors. Below is a summary of the overall methodology used to compile the 2008 rankings as well as notations for each of the categories1.

For each of the professor lists, each individual rating value was first standardized around its mean. Using the standardized scores from the years 2006, 2007 and 2008 to date, weighted scores were computed using the following weights: 15% for 2006, 25% for 2007 and 60% for 2008. The weighted score reflects a combined, long-term performance rating of just over a two-and-one-half years, with an emphasis on the most recent ratings2.

Using the weighted score, professors were ranked from high to low. Only professors with 10 ratings or more (in the years 2006-present) were included to provide statistical significance. All professors were verified as actively teaching in the current semester by each school at the time the lists were compiled.

In an attempt to break ties, professors with a greater number of ratings were ranked higher; the rationale is that a larger amount of information typically results in an estimate closer to the true parameter. Nevertheless, ties (i.e. professors with the same score and the same number of ratings) still occur. In that case, if two professors are tied for the same place - say 5th - then the next available rank is 7th3.

It should also be noted that school size does not affect the outcome of the lists nor does it give professors from larger schools an advantage over their corollaries from smaller schools. We performed a regression analysis on school size vs. number of ratings and found no noteworthy correlation.

Here now is a look at how each of the lists were compiled:

Highest Rated Professors

Students on RMP rate professors on several dimensions: clarity, helpfulness, easiness and rater interest (interest level prior to attending the class). However, overall professor quality (which informs the highest rated professor list) is determined by an equal weighting of only two criteria: clarity and helpfulness. 5 is the highest rating and 1 is the lowest rating for each of the above-mentioned dimensions.

Hottest Professors

A professor who received a chili pepper is considered "hot." Chili peppers are awarded based on the sum of positive and negative (hot or not) ratings. For instance, if a professor receives 7 "hots" and 6 "nots" the hots will be counted as "+" and the "nots" as "-". The sum of these (7-6) equals 1, meaning the professor will receive a chili pepper. If the result had been negative, the professor would not receive a chili pepper. Professors are ranked by highest numerical value in this case. For instance a -3 result is higher than a -4.

Schools with the highest rated faculty

Schools with the highest rated professors were ranked using a very similar methodology. Selecting only schools with a minimum of at least 30 rated professors, we computed the average professor rating for each school (only professors with 30 ratings or higher). Schools were then ranked from high to low according to their average rating. Using a similar rationale as described above in the "Highest Rated Professors" category, we broke ties using the average number of ratings per school.

1  Data analysis for these rankings was performed by Dr. Wolfgang Jank, Associate Professor in the Department of Decision & Information Technologies at the University of Maryland's Robert H. Smith School of Business

2  Please see both BusinessWeek's and the Financial Times' ranking methodology for Business Schools for an examples
http://www.businessweek.com/magazine/content/06_43/b4006008.htm
http://www.ft.com/cms/s/2/39626150-ad5a-11db-8709-0000779e2340,dwp_uuid=991cbd66-9258-
11da-977b-0000779e2340.html

3  See an example of this type of ranking in the Financial Times European Business School Rankings
http://www.ft.com/cms/s/2/39626150-ad5a-11db-8709-0000779e2340,dwp_uuid=991cbd66-9258-
11da-977b-0000779e2340.html