Einstein admonished us that we tend to measure only what we can count easily and thus fail to measure what really counts.
We see Einstein’s challenge in a number of efforts to evaluate institutions in higher education: from the input-heavy methodology of the U.S. News & World Report rankings to the survey methodology of the Princeton Review, to the newer output-focused economic models of Payscale and others.
Each metric implies a theory of value: schools with stronger inputs (student selectivity, alumni contributions, faculty ratios) are better; schools with superior survey results (best campus, most accessible faculty, happiest students) are better; schools with stronger outputs (high graduation rates, post-graduate salaries, and low debt) are better. The use and reliance on any one ranking is a choice that depends on which theory of value is deemed most important at a particular school. It is also a methodological choice based on whose empirics seem most transparent, accurate, or aligned with the proposed value.
In a phrase, metrics are in the eye of the beholder.
Beyond this observation, we can see a series of traps for leaders in higher education. We tend to champion the favorable rankings and criticize the unfavorable ones. The rankings and the metrics we consume are inherently and at times profoundly imperfect and yet they are modestly useful as proxies of value. But the championship and criticisms of rankings lean to the extreme. Rankings have become a type of sport in which we dance in the virtual end zone after the touchdown or else we complain about the refs when we are on the losing end of the results.
So then, how should we navigate the rankings and think about our own forensics of value here at Claremont McKenna? Here are four thoughts for your consideration.
First, we need to stay focused on our own theory of value regardless of what external rankings seek to impose, so that we can consider competing theories in comparison.
Second, we need to be principled in our approach to the power and limits of quantitative comparison, by focusing on the accuracy and merits of the underlying methodologies.
Third, we need to recognize what is easily forgotten: the use of rankings is an imperfect proxy for value, especially in the absence of better metrics for applicants and their families to use as they sort through complex data in preparation for making a large financial investment and long-term commitment.
Finally, if we are critical of measurements used by others, we have the responsibility to develop our own, and that is where we have hard work to do. For example, at CMC, can we develop a working measure for the cultivation of responsible leadership? Through the new president’s Mellon Foundation grant program to be announced by Dean Warner, can we develop ways to measure creativity, empathy, and courage in pursuit of central learning objectives? If, as Professor Kind suggested in her convocation lecture Beyond Happiness, discomfort is a vital condition for intellectual and emotional growth and fulfillment, can we develop a discomfort index that demonstrates how dedicated we are to get outside our comfort zone?
As we enter the new academic year, please join me in reflection on Einstein’s challenge: how should we choose among alternative metrics, illuminate the underlying choice of value they reflect, and weigh methodological strengths and weaknesses in the measurements applied?
Please feel free to share your questions as well as your ideas about developing new ways of measuring our special value.
“First, we need to stay focused on our own theory of value regardless of what external rankings seek to impose, so that we can consider competing theories in comparison.”
Couldn’t agree more. What makes CMC great is it’s unique commitment to the belief that the liberal arts are too valuable not to be applied in business and public affairs.
I agree completely with the sentiments of “patwater”. I entered CMC in the fall of 1967, when rankings were not a significant factor unless you got into an Ivy League college. While developing a theory of value is a very serious endeavor, rankings are a marketing issue and one that could essentially be ignored by having a marketing strategy that communicated CMC’s success at being the kind of place a cohort of good students wanted to be. CMC’s record of graduating successful business people and people who succeeded in government (and individually represent multiple strains of political philosophy), as well as other fields, is, to me, the most important demonstration of its long held educational values. I personally took a path into science, engineering and risk management which was strongly enhanced by the literature, history, economics and political science I learned at CMC. I believe it is important for applicants and incoming students to understand that graduates follow many diverse paths, some not even contemplated while in college, and an institution that provides a broad understanding of our culture and as many of its component parts as can be crammed into four years can be of exquisite value in navigating the forks in the road, plus the enjoyment that comes from engagement in many things outside of our current jobs. I understand that a college president cannot ignore rankings but a focus on providing consistent value (and following a set of values) would be invaluable in pursuing a track that is most worthwhile and least buffeted by external fads.