News: Feasibility study report on cross-national measurement of student learning in HE (AHELO)

AHELOpic OECD recently published the first volume of AHELO feasibility study report, focused on design and implementation. AHELO stands for Assessment of Higher Education Learning Outcomes and is in essence an attempt to conduct cross-national comparisons of learning outcomes independent of different cultures, languags and institutions.

The aims are similar to the PISA test that measures student learning in schools in a cross-national perspective – with an important difference – the comparisons will not be on national, but institutional level. In addition, the study will not provide a ranking. The feasibility study took 5 years to conduct, and this volume marks the first of three that will be published in upcoming months. The first volume focuses on design and implementation, second on data analysis and national experiences, and third will give further insights and conference proceedings.

Currently, the AHELO project has gone through a feasibility study in two discipline specific areas – economics and engineering, in addition to a measurement of generic skills. ¬†The feasibility study tested 23 000 students in 248 institutions in 17 countries on a voluntary basis. One of the countries particiapting in the generic skills evaluation was Norway, J. Levy who is a member of the permanent Norwegian delegation to OECD explained thatIf successful, it will increase our knowledge of higher education institutions, and thus give additional tools for quality development.” Furthermore, he argued that AHELO can provide “new information, supplementing existing information from rankings mainly based on data on research activities“. Similar arguments about more information and data about learning were also emphasized by the Mexican and Australian representatives.

The feasibility report highlights that the basis for a project like AHELO to emerge lies on global trends such as expanding higher education systems, widening participation, new kinds of educational providers, diversification of students, new technologies, internationalisation, new funding modules, marketisation, and governance that puts focus on performance and accountability. In this context, there is arguably a gap in information regarding the quality of student learning outcomes as the student body is diversified, they attend a number of new kinds of institutions and in many cases there are concerns about considerable drop out rates, the report argues. In this context, the idea for AHELO was proposed in 2006, and it is argued¬†(p.34) that it marks a “paradigm shift” for higher education.

The report also examines some of the critical issues that emerged during the process of operationalisation and development of measurement, including issues such as ranking (concerns over AHELO becoming another ranking), misuse of information (financial reallocations of funding based on results), complexity of comparisons (assuring that institutions compared are comparable), issues with standardisation (oversimplification), and potential adverse effects on autonomy and academic freedom. The third section examines some of the methodological challenges with designing and managing the survey, and finally the report gives insights into the development of the specific measurement instruments.

For more information, download the whole report here (.pdf).