Ljiljana Krstić is a recent graduate from the UiO’s Higher Education Master’s Programme. Her prior education includes a degree in the Greek language and literature as well as the human resources management. Main research interests involve organisational change within universities, management and internationalisation of higher education.
The idea to write the thesis about learning outcomes resulted from the article I read on how research has pointed out to the discrepancy between the narrative and actual application of the concept in practice, and how in fact application has turned out to be the slow and difficult (Adam, 2008). In addition, a CEDEFOP study (2012) confirmed that interpretations of the concept vary throughout Europe, and even within individual institution (Dobbins, Brooks, Scott, Rawlinson, Norman, 2014). Research indicates that application of such a broad concept may cause a variety of interpretations, misconceptions and misuses.
In general the problem with the reform rhetoric and changes that follow, are empirically unverified beliefs and assumptions of the reform policies. Some are expected to be adopted even if they lack empirical verification, normative agreement and clear theoretical propositions (Maassen, Olsen, 2007). Learning outcomes exemplify how a policy debate throughout Europe has the tendency to become more similar, despite the different traditions and varieties between the counties, implying the willingness of national actors to follow the new terminological fashion (Teichler, 2004) and to emphasise the “European perspective’.
Therefore I decided to write about perspectives necessary for understanding the concept of learning outcomes and empirically verifying whether some of them are more dominant than the other. Additionally, I wanted to hear the voices of academics and academic leaders as the ultimate recipients of the policy, responsible for its reshaping in practice and enquire about their interpretation of the concept and its embeddeddness within the institutional context of the University of Belgrade. Lastly, I used the data to find patterns in the perceptions of changes which occurred as a result of the application of learning outcomes in practice.
As for the methods of inquiry, University of Belgrade was treated as an embedded single-case study, with three faculties as sub-units integral to the University as a whole. The selection of faculties reflected the classification of disciplines into four broad headings: hard-pure, soft-pure, hard-applied and soft-applied (Becher, 1989; Neumann, Becher, 2002).
Understanding and interpretation of learning outcomes may vary respectively to the perceived learning orientation and purposes of the concept among academic community who assume different functions within the University. Thus, it was essential for the study to explore the perceptions of academics and academic leaders. Empirically, the thesis is built upon twelve in-depth semi-structured interviews and relevant university and legislative documents. The respondents are academics and academic leaders from: soft-pure, hard- pure and hard-applied faculties respectively.
Divergence of formulation among the academics and University documents
According to Prøitz (2010) the dominant debate involves the ongoing disputes of the two classical perspectives of learning, behaviourism and social-constructivism. The behaviourist tradition suggests that learning should be measurable, pre-formulated and result oriented. On the opposite, social-constructivism asserts that learning is an open ended process, with limited measurability and process-orientedness.
Findings indicate the divergence in the formulation of learning outcome definitions between the academic community and the University documents. The official documents use well-established definitions, formulated by behaviourist propositions which emphasise instructional planning and specific learning activities one must engage in, in order to complete a required task (Gagné 1974). The common denominator of all the documented definitions is precision to specific knowledge, skills and competences a student actually acquires at the end of the learning process (Adam, 2004).
Interviewees provided similar responses, although assertions varied from uncertainty what learning outcomes are, to the questionability of their measurability and formulation. The uncertainty originates from the following premises:
- The fashionable term is de facto an old familiar concept
- Confusion between the terms ‘aims’ and ‘outcomes’
- Philosophical objections, especially by the respondents of the soft-pure faculty
- The formulation guidelines were vague and scarce
On the other hand, in terms of purpose, learning outcomes perspectives vary according to two opposite approaches, the pragmatist movement and assessment approach. According to the pragmatist movement, learning outcomes serve as tools for ‘educational and instructional planning and curriculum design (Prøitz, 2010, p.123). The assessment perspective appears as a response to the governmental urge and pressures for more accountability and increased effectiveness of higher education.
Document analysis and interviews undoubtedly provide empirical evidence that learning outcomes are employed as tools for educational, instructional planning and curriculum design. Primarily LOs are utilised in the programme and individual course descriptions. Study programmes define generic and subject-specific competences, and try to assign number of ECTS according to learning outcomes. A national higher education report from 2012-2015 however, asserts that in practice defining ECTS has encountered practical problems in the terms of workload-number of credits assignment.
Finally, from the assessment standpoint, higher education institutions have to be held more accountable for the educational outputs and strive to raise the effectiveness of education. Hence, outcomes of education should be measurable and correlated to educational inputs (Biesta, 2009). Legislation and university steering documents fail to provide the empirical support of learning outcomes formulated and defined as accountability tools in neither of the study’s outlined categories: political, legal, bureaucratic, professional and market (Proitz, 2010; Darling-Hammond, 2004). Nevertheless, some of the respondents objected to the ‘must’ as a requirement to formulate and define learning outcomes as accreditation committee requirement. Although the process results from the departmental co-operation, heads of departments steer and manage partly as a requirement of the function they perform.
The following figure shows the expected and emerging patterns of the empirical evidence.
Interpretation and embeddeness of learning outcomes at the University of Belgrade
According to the study findings, academic leaders interpret learning outcomes in the context of the ‘ultimate outcome’, referring to the result of study programme completion- a competitive graduate in the labor market. This inference might be explained by the managerial function of vice deans of education, who perform administrative activities participate in the setting of the Faculty’s strategic plans and keep track of teaching processes and curriculum changes. Their responses support the narrative of the outcome approach to increased University’s responsiveness to the societal needs. Emphasis on the improvement of the educational outcomes is often notable in transitional countries, because of the increased political pressures to create better links between higher education and labor market (Brankovic, Maassen, Stensaker, Vukasovic, 2014).
Academics interpret learning outcomes in the context of curriculum design, course descriptions, teaching methods and assessment. All of the academics asserted that agreement within the department had to be reached upon the course design. hence, the cooperation on the departmental level can be attributed to the same belief systems, as they share disciplinary and professional cultures (Clark, 1983).
For the organisational level though, those inferences might be misleading, due to the system fragmentation of the University of Belgrade. From the cultural-institutional perspective, the acceptance of an idea depends on the congruence of its intrinsic value, norms and principles with the organisational culture. Although universities’ declaration of the student-centred approach (CONUS, 2011) praises the merits of the learning outcomes, it cannot be inferred whether the concept is accepted or rejected by the institutional actors, and whether it is on a path of institutionalisation. Given the fact that the Declaration was issued by the Conference of the Universities in Serbia as a strategic plan to promote the concept and organise the trainings for the academic community, it can be concluded that the University of Belgrade intends to introduce the essential organisational changes to enable the ‘paradigm shift’ as the University’s strategic goal.
On the course/module level academics pointed out to practical problems encountered since the introduction of learning outcomes. According to the data, academic agree that teaching practices had to be accommodated to the new programme structure and the one-semester length of the courses. However, respondents did not clarify whether courses were designed according to the syllabus content, literature and number of direct contact hours, or by learning outcomes. Academic leaders emphasised the transparent descriptions of learning outcomes on the course level present a positive development for students, academics and the content review for the internal and external quality evaluation processes.
In terms of teaching practice, the views of respondents were polarised, from complete disregard of learning outcomes as empty meaningless phrases, a column to fill in, to envisioning the degree by defining learning outcomes and working in reverse down to the individual courses, to make sure they acquire them in the end
The empirical evidence does not point out to specific statements on learning outcomes as performance indicators and/or monitoring devices. However, closer look at the documents, specifically at the standards for quality assurance classify student surveys as evaluation devices, an obligatory practice after completion of the course. From the perspective of the one academic leader, student evaluation of the course might indicate its successfulness, but cannot oversee all the aspects of learning.
Patterns of change processes
The subjective nature of the last research question, takes all of the inferences with caution and with respect to the internal and external validity of the study. The following themes emerge from the data: uncertainty of the concept’s usefulness, change of accreditation requirements, structural changes, teaching-learning-assessment activity changes, transparency and accountability perceptions of change.
Ultimately, this is a study about understanding, interpretation and embeddeness of a concept of learning outcomes in the context of a country in transition. Moreover, it focuses mainly on academic community and their interpretations of the concept, a research topic not explored enough. From a personal point of interest a longitudinal study on possible institutionalisation of the learning outcomes at the University of Belgrade could provide an analysis of the intrinsic and extrinsic values of this particular policy.
Do not hesitate to contact for details about the study, you can contact Liliana at firstname.lastname@example.org.