I know, I know, not what you would expect to hear. For several years now, I have felt that university and school rankings do a significant disservice by purporting to have the one methodology that determines the true ranking. Ranking schools is a good thing – don’t get me wrong. I support comparing schools and outcomes, but think the only appropriate ranking is one whose component weights are done by the end users directly.
Nearly all rankings determine a list of schools overall, and maybe some specialty lists in sub-groups, but students are not homogenous interchangeable customers and cannot be lumped together and serviced with a one size fits all product. With available technology, it is entirely feasible to conduct the same research and to publish the results in a form that lets end-users determine how important each category and component of the data actually is. That is to say that prospective students and those using the rankings should be able to change the weights of different categories and weights of individual components so the ranking engine generates a list of schools entirely appropriate to the user. For example, students may not care a single iota about publication history of faculty and may be far more concerned about job placements or teaching quality and they should be able to manipulate the algorithm to create a list that matters – rankings that matter.
The first place I’ve seen this attempted, and to date the only place, is on an iPhone app by Times Higher Education for their World University Rankings. The app has a “customize” option in the iPhone app that allows the user to change the weighting of five different categories and look at the new rankings and the original rankings.
While not perfect, this is a tremendous step forward in providing rankings that matter to students, and no longer decreeing to the world that you will you have the one true methodology that matters. The next step is for a rankings publisher to provide a tutorial about the implications of each ranking component and enable user choice in the weights. I wonder how long it will be before user generated rankings replace the static, homogenous lists now routinely published.
I won’t start talking about how appropriate is it for students to look at a ranking list and to make a decision on where to apply to grad school. I certainly have a lot to say about the subject however so watch for a future post.