This week I received the annual U.S. News and World Report survey that asks me to provide a “peer assessment” of hundreds of liberal arts colleges and universities in America. My response and those of thousands of other presidents, academic deans and directors of admissions, will be compiled by the editors of the magazine for their 2010 edition of America’s Best Colleges and Universities.

I am not a big fan of these ratings magazines because they can lead to very arbitrary findings. Saint Vincent ranked very well in Forbes magazine’s college and university guide but not as well in the USNWR guide – and in large part, this is because the most heavily-weighted criterion in USNWR is this “peer assessment” that I am about to fill out.

Here’s how it plays out. The editors want me to opine on Texas Lutheran University and Allen University (in South Carolina) and Warren Wilson College (north of the border) and Judson College in Alabama – to name just a few.

How can I respond to the solitary survey request to “rate the academic quality of undergraduate programs” and decide, for example, whether those at Sheldon Jackson College in Alaska are distinguished, strong, good, adequate or marginal? The obvious answer is that I can’t, and so of course, I am left to check the remaining “don’t know” box provided. It seems rather ludicrous to base a rating of colleges and universities on nearly blind assessments – I could check any box and it would be counted.

Among the hundreds of institutions on my ten-page survey booklet are places like Davidson, Furman, and Wheaton where I have spent time – but even in those cases, how would a visitor, as I was, get a real sense of the quality of the undergraduate programs there?

Last Thursday I gave the McNerney-Hanson Ethics lecture at the University of Portland. I met with the president there, Fr. Beauchamp, and with senior administrators and distinguished faculty at the dinner that followed my lecture. The campus was lovely, these individuals were very impressive, and the students I met with told me they enjoyed their undergraduate experience there. But does that put me in a position to intelligently respond to the survey question with regard to the University of Portland?

Tonight I will be at Grove City College, one of the institutions that I am asked to evaluate. I will be meeting with its president and then attending a lecture by General Dick Tubb (a remarkable man who served as physician for presidents Clinton and Bush). There will be a dinner and an opportunity to tour the campus and visit with folks – but again, would this enable me to choose between distinguished and strong in rating the academic quality of Grove City College’s undergraduate programs?

And then in two days, I will at one of the most highly esteemed universities in America – Harvard University. I have been invited to engage in a debate at Harvard Law School on the faith-based initiative. Because of Harvard’s long established reputation, I am sure the folks checking USNWR boxes are going to bless them with yet another stellar rating. One would think such a rating is well-deserved – but upon what basis is it given? Reputation? Name familiarity? It seems to me that unless I actually worked at one of those institutions, I would be hard-pressed to render an informed judgment on this question. It is amazing to me that all of these surveys come back with any boxes other than “don’t know” checked!

My point is that these ratings are highly unreliable and may invite the worst kind of politics – where some may be tempted to rate competitor colleges poorly to increase the relative advantage in the marketplace of their own institutions. It sure seems the methodology used by USNWR supports the status quo and makes it difficult for small colleges like ours to rise.

Nonetheless, Saint Vincent is on the move. One of the reasons our professors and administrators attend conferences and forums away from campus is to increase the visibility of Saint Vincent College and enhance our reputation. You may have seen national news stories over the weekend that repeatedly mentioned the fact that Saint Vincent hosted a sitting United States president two years ago (in the context of Notre Dame’s controversial decision to host President Obama). These stories certainly help position our college’s name in the public square and give it more prominence, but I am not sure that this would inform a single one of my colleagues’ decisions on rating Saint Vincent.

As president I cannot ignore USNWR’s ratings or those of other periodicals or web sites that rate colleges and universities. High school students and their parents use these resources as they navigate among thousands of choices, and therefore these ratings instruments clearly can affect enrollment. The fact that Forbes magazine ranked us ahead of nearly all area schools and alongside Carnegie Mellon University surely has helped in our recruitment efforts.

So while I cannot ignore USNWR and the other ratings entities, I also cannot put too much stock in what they have to report.

Our students know well the academic quality of our undergraduate programs – even if my colleagues at American Jewish University, Harvey Mudd College, and Pitzer College (to name three institutions in California that will each have three individuals rate Saint Vincent for USNWR) may be left to guess or check “don’t know.”

Our faculty knows. Our alums know. The employers, med schools, law schools, doctoral program administrators, and others who welcome our students fresh out of Saint Vincent know.

And to me – the bright futures and satisfaction of the young men and women who study here and leave intellectually, morally and spiritually better for the experience – are the best measures of all.