The final feature I have identified (for now) that should be included in any Federal college rating system relates more to the submission process. At the moment, I am responsible for submitting figures for over two (2) dozen different reports and surveys. Almost every single one of these asks for the calculated figures, rather than the data those figures are based upon. The only report that asks for individual student records is required for our state appointed board of regents (and is thankfully supplied by our Vice President of Information Technology). The nature of this reporting is both a good and a bad thing. On the positive side, I would be very uneasy about handing over so much data about our students to any external, non-governmental agency and I suspect many of my colleagues (and students) would express the same reservation. The bad news is that these are difficult or impossible to revise and can at least temporarily be subject to inaccuracy. I understand that we are all guided by a code of ethical and professional conduct and none of us would ever even consider falsifying numbers. But I am not worried about my ethical counterparts; it’s those other guys, I worry about. This is particularly the case with high-stakes assessments as the federal rating system might turn out to be. We are all human, and when faced with a big enough reward for unethical behavior a few of us might stray from the straight and narrow (I am reminded of a certain state governor that implemented a code of ethics for all state employees and a few years later was convicted of being somewhat less than completely ethical).
In order to prevent or at least reduce the temptation and ability to falsify or otherwise “adjust” reported figures, I believe the Federal ratings system ask for student records directly, rather than pre-computed values. With these records in-hand the Federal college ratings body can apply all calculations directly to the data, thus ensuring consistency and accuracy.
Aside from promoting ranking figure consistency, this process has the benefit of facilitating future modifications to said rankings. Suppose future research suggests a slightly different calculation of graduation rate. At the moment only new reports can employ the new calculations making historical figures difficult to interpret and comparisons to said data difficult. Recalculating historical figures with new procedures would only be possible at the level of each organization and would likely be labor intensive enough that few would bother to try. On the other hand, if the Federal ranking body retains individual student records new figures can be calculated as needed and applied to previously published reports.
The final and perhaps most significant benefit of this submission model is that it would create a single repository of student records available to researchers and regulators alike. At the moment research can only be conducted at the level of each reporting unit, be it university, college or satellite campus. While group-level research can be valuable, individual-level data offers an opportunity to investigate hypotheses that were impractical or impossible to explore (Note: I could not find a decent link to describe methodological issues related to levels of analysis, so consult your friendly-neighborhood statistician). Lack of individual level data limits how insights can be generalized to individual students, which can be a problem for educators, regulators and legislators. Even more significant is the possibility that higher education databases at the federal level could be linked to other governmental databases, enabling more definitive research into outcomes like post-baccalaureate earnings, incarceration rates and others.
Article Concludes: Federal College Ratings: Overview and Concluding Remarks