Accountability, Assessment and the Public Policy Environment

Speaker: David Ward, Former President, American Council on Education and Chancellor Emeritus, University of Wisconsin

Over the course of the fifty years or so since the end of World War II, the success of US higher education has been recognized and celebrated primarily in the growth of research productivity and expanded access.  Access not only included a much larger segment of traditional college age students between 18 and 24 but also many who chose to start or continue their higher education later in life. More recently this positive perspective has been qualified by a growing national interest in the cost of research intensification and of enhanced access. The most compelling questions are related to the degree to which students actually complete their programs and the adequacy of their general educational competency. This focus on student performance has resulted in calls for accountability within higher education from both internal and external sources. So far the diverse range of efforts to initiate or enhance accountability and assessment is extremely varied. The extraordinary costs of research intensification have also limited the number of institutions able to sustain a comprehensive research mission, and the assumption that all institutions of higher learning will necessarily sustain major research programs was never fulfilled. Consequently, US higher education is so diverse in its offerings and structures that it is difficult to develop uniform metrics to gauge effectively every student and every institution’s performance. 

This interest in accountability is not new, but changes in the relationship between higher education and government have given this interest a more imperative and potentially regulatory tone. Over the past two decades it is clear that the policy ideals of the immediate post World War II era have reached the limits of their fiscal capacities. The simultaneous commitment to expand research capacity and to massively increase access to higher education has necessitated both a diversification of revenue streams and a greater differentiation of missions within higher education. The exponential growth in research costs has made it difficult for all but a selective group of well funded universities to maintain a commitment to a comprehensive research mission, while the need to supplement educational needs by means of higher tuition and financial aid have undermined or at least confused the degree to which it is possible for higher education to provide low cost programs for all, irrespective of need.

Concerns about costs and prices remain a central focus of policy discussions about higher education, and it is inevitable that these discussions will spread to issues about performance and accountability. This expanding array of concerns has been amplified by growing anxiety about international competitiveness, for in a global economy based on the discovery and application of knowledge, higher education is often regarded as a key competitive asset. The somewhat flawed national comparisons of the performance of pre-college students and of their subsequent degree completion rates reveal significant weaknesses in the US educational pipeline. The temptation to mandate and regulate standards and performance will inevitably be a continuing part of the public policy debate about higher education.

Over the past three years these questions have been addressed by several commissions convened by major foundations, testing services, business states and most visibly by the Secretary of Education in the second Bush administration, Margaret Spelling.  Each commission had a different focus, but fears about access to education due to high cost and reduced state investment, as well as concerns about competitiveness in research and specifically preparation in the STEM disciplines, were common to all of them.  All assumed that higher education and economic development were interrelated and that the performances of all levels of education were critical to the future national interest.  While this connection certainly enhances the justification of the value and role of education, it also raises expectations and perhaps inadvertently narrows the range of purposes for which education has been responsible.

A desire to use age-specific standardized tests to measure performance resulted in an emphasis on six or seven competencies that would define the key content of higher education. This objective presented a number of problems.  While there are currently measures of cost, price, time to degree, and a variety of other indicators of student demographic characteristics, there are few if any adequate measures of learning outcomes capable of standardization. Certainly the quality and standardization of some data could be improved and made more transparent and comparable, but the sophisticated measurement of academic progress requires much further research before any outcomes are likely to be reliable. Standardized tests may be helpful in defining minimum standards below which future success may be difficult or they may assist in directing students to alternative programs, but they are rarely effective in predicting future performance. Moreover, tests tend to focus on the proportion of students falling below a certain pre-determined level, but this finding does not in itself define the cause of the outcome. One result of mass higher education is an increase in the variability of student academic performance and a greater variability of the age at which students may develop any specific competency.

The high costs of collecting, maintaining and reporting data on an infinite variety of institutional and student behavior has made higher education somewhat skeptical of any effort to increase this burden. Much data is collected independently and is rarely cross tabulated; and if we are to measure rates of program completion effectively, it will be necessary to collect information on specific individuals on a longitudinal rather than on cross sectional basis. A growing proportion of students attend college part-time, drop out periodically before dropping back in to complete a program. A large proportion of all students now take courses at more than one institution, sometimes simultaneously, and completion rates that do not account for the high rate of student transfer will result in misleading under-estimates of performance and an excessively negative interpretation of drop out rates. We do need improved data calibrated at the individual level and it would also be prudent to review the value of much other data that is currently collected at great expense.

American post-secondary education today needs to respond more directly to these external pressures for accountability and assessment by harnessing institutionally driven efforts to measure and improve performance and to develop more effective communication and collaboration among institutions with similar missions. In these efforts, we need to clarify the demands for accountability with respect to cost, productivity and access for which there are appropriate measures from the assessment of learning outcomes which will require much more nuanced and prudent research and reflection. Moreover, we need to be clearer about the balance between a legitimate public interest in transparent availability of useful information on higher education and the responsibility of colleges and universities for academic standards and curricula. The genius in American higher education in my judgment is based on an appropriate but sensitive recognition that this public interest cannot be achieved by government alone. Accreditation and disciplinary and professional associations do have an indispensable role in those aspects of accountability that involve learning assessments.

What steps might we take to respond to the challenges presented in a new public policy environment that is increasingly influenced by the heightened focus on higher education as a critical competitive element in the global knowledge economy?

1. Clearly distinguish between institutional accountability for which there are currently adequate measures and the assessment of student learning outcomes for which we need a more convincing research base.

2. Improve the degree to which data tracks individual students rather than relying on periodic cross-sectional information that fails to account for transfer and discontinuous college attendance.

3. Much effective work in accountability and assessment is based on campus-specific initiatives, and some appropriate system of sharing best practices among institutions with similar missions needs to be encouraged.

4. While there may be a few standardized measurements appropriate to all of higher education, data presented on the basis of mission and discipline are more likely to seed improvements in learning outcomes.

5. The tendency to present data on a rigid ordinal basis without any recognition of the magnitude of difference measured by pure ranking seems to have little diagnostic capacity to identify the sources or means of improvement.

6.  Accreditation is clearly the arena where the interests of government, the public and institutions should be defined, and the process of accreditation itself should emphasize institutional and perhaps even mission-based inter-institutional efforts to improve the transparency and effectiveness of accountability and assessment.

In addition to these shared steps, the Reinvention Center could further enhance its efforts:

a) to amplify its role in improving communication about assessment and accountability among institutions with a special commitment to publicize best practices and to foster better articulation of institutional accreditation and of institutional strategic planning.

b) to engage directly in the debate about the role of accreditation in providing the kind of institutional accountability that meets professional and public expectations.

c)  to foster approaches to accountability and assessment that engage simultaneously all segments of higher education rather than the current tendency for a bifurcation in advocacy and implementation between faculty and administration and between institutional efforts and public policy proposals.