Kwaliteit HO Europees borgen

Nieuws | de redactie

17 december 2007 |

Om tot echte Europese kwaliteitszorg en borging te komen moeten de écht substantiële verschillen tussen de accrediteringssystemen van de landen hanteerbaar en overzichtelijk zijn. NVAO- voorzitter Karl Dittrich heeft in een rede in Barcelona aangeven, dat hier nog maar beperkt problemen blijken te bestaan. “It is safe to conclude that 90 to 95% of the themes assessed within the various assessment frameworks are identical, although wordings may differ in some cases. The ECA partners concluded that the number of ‘substantial differences’ is actually extremely small! Several organisations have reached the point where they can formally recognise each other’s accreditation results or decisions. Achieving this milestone has required a great deal of extra work over the past months.”

Op deze basis is een structurele samenwerking mogelijk geworden tussen de borgingsorganen en kunnen stappen gezet worden naar een Europese kwaliteitszorg van het hoger onderwijs. “This means the idealism of the Bologna signatories can be realised in practice, although the process may take considerably longer than they originally believed in all their idealistic innocence back in 1999,” concludeert Dittrich.

Hij ziet voor de komende jaren wel nog veel huiswerk. Bij voorbeeld “in dealing with the quality labels, the EU is so keen to stimulate. Rather than existing in parallel to the national accreditation systems, these labels should be incorporated within them. Institutions and programmes must invest a great deal of manpower and means in order to obtain even a single accreditation. National accreditations are often required by law; the simultaneous obtainment of “professional accreditations” would mean a great improvement in terms of efficiency and effectiveness.”

Dittrichs volledige analyse en overzicht van de actuele stand van zaken leest u hieronder.

---

At the time of our first meeting in June of 2003, we were either vague acquaintances or even complete strangers. Now, just over four years down the line, we can rightfully claim to have built mutual respect and trust, and conclude – despite differences in terms of our administrative and legal constellations – that our organisations would all assess the quality of institutions and programmes correspondingly! No mean feat, to be sure!

My story here today is one of trust that has been earned. It is about the way in which strangers set out to find similarities, meaningful differences and the consequences they could or should have. Despite having yielded results, this quest took a great deal of effort and energy, and was founded on both objective observations and subjective emotions and views. The joint efforts and results can be attributed to our organisations, but we must also thank those people within them that worked to create this mutual trust and respect!

Our activities appear – especially in hindsight – to have unfolded along an extremely rational path, made up of structured steps that were implemented in sequence. Rolf Heusser’s presentation outlined our “pyramid” of mutual recognition, which – as you saw – seems to be built up along very rational lines. Every step appears to follow logically from the previous one. This does seem to be the case and can be regarded as such in retrospect, but this rational approach was hardly in place in June of 2003. The first logical step was to get to know one another, based on an initial comparison by one of the first gurus of external quality assurance, Ton Vroeijenstijn.

This was primarily intended to draw up an accurate description of the various accreditation systems and the role of our organisations. These initial efforts to take stock of the situation brought to light a large number of similarities between our procedures, but also obvious divergences between the legal frameworks within which we operate. For instance, where some organisations could take independent decisions with legal force, others had to obtain government approval before assessments could take on legal force.

Nevertheless, paper is patient, and the reality behind those paper agreements and comments often emerges in the day-to-day practice. This is why the ECA partners invited observers to take part in each other’s accreditation procedures in an early stage. In the Netherlands and Flanders for instance, we played host to observers from our Irish colleagues at HETAC, from our Norwegian colleagues at NOKUT, the French CTI, the Swiss OAQ and the Polish PKA, while NVAO staff took part in accreditation procedures in Ireland, Norway, Spain, France, Switzerland and Austria. This helped bring the procedures and approaches to life, allowing us to gain practical insight into the shape these accreditation procedures have taken in the various countries. Obviously, this is extremely helpful in building mutual understanding and trust between the organisations!

We can basically regard these steps as the necessary preconditions in order to achieve sufficient mutual understanding and knowledge. It is thus the first part of our quest, or – in more rational terms – our roadmap.

The Code of Good Practice which was signed by all organisations in December of 2004 and can be seen on screen now, is vital in further developing trust between our organisations. The Code consists of 17 standards, relating to our organisations or the procedures and accreditation criteria they apply. The Code obliges us to draw up a mission statement, explain how our independence has been assured, apply an internal quality assurance system, provide transparency as to our procedures and results and cooperate with other organisations. We have also set ourselves certain rules with regards to accreditation procedures: these should be underpinned by self-assessments conducted by an institute or programme, followed by a site visit by an independent panel. We promised each other that the ECA partners would finish implementing the Code by the end of 2006, allowing this to be included in the external assessment of all organisations in 2007 and 2008 as a part of assessments for ENQA membership or national assessment obligations, as is the case in Germany. Here, the Akkreditierungsrat is charged with the assessment of regional or supraregional accrediting organisations.

Slightly less spectacular, but equally important are the Principles for the selection of experts agreed upon in Dublin in June 2005. No matter what accreditation system is applied, after all, it is the experts that have the final say when it comes to assessment of institutions or programme(s). This places a great deal of responsibility in the hands of the experts, who must be independent and expertly trained. However, the makeup of the panel – which must be well balanced and capable of assessing all relevant aspects of an institution or programme – is also essential. Conducting assessment requires skill and professionalism, and putting together a panel is no easy task. On the contrary, the makeup of the panel and quality of the assessment procedure are vital with regards to the assessment and its legitimisation! This requires a professional approach and must not be underestimated.

In follow-up to the above agreements between the various ECA members, an agreement was reached at the session in Vienna in December 2005 and later confirmed at the meeting in Bruges in June of 2006 specifying the need for more intensive focus on the various assessment frameworks used by the accreditation organisations. The initial approach was to identify similarities between the various frameworks, in parallel with joint accreditation by two or more organisations: CTI, OAQ, and ANECA were especially active in this area, with the assistance of NVAO, while a number of joint accreditations were also conducted by German organisations, most notably ZEVA and ACQUIN in cooperation with NVAO.

The comparisons yielded a great many similarities between the various assessment frameworks. It is safe to conclude that 90 to 95% of the themes assessed within the various assessment frameworks are identical, although wordings may differ in some cases. The next question was whether to emphasise the similarities or the inevitable differences. The ECA members sought guidance from their collaborating ENIC/NARICs: these organisations have much more experience in dealing with national differences in the recognition of qualifications and degrees according to the Lisbon convention. They do so on the basis of the “substantial differences” concept: the key issue is not the differences themselves, but whether or not these are substantial enough to affect the quality or content of the qualification or degree. Using this concept, which has proven to be very workable within the day-to- day practice of the ENIC/NARICs, the ECA partners concluded that the number of “substantial differences” is actually extremely small!

The accreditation organisations and ENIC/NARICs in Austria, the Netherlands, Flanders, Switzerland, Norway and Poland formulated a joint declaration in Vienna, the Joint declaration concerning the automatic recognition of qualifications, which specifies that the mutual recognition of accreditation decisions could be recognised by the ENIC/NARICs as a step towards automatic recognition of qualifications and degrees by countries whose national accreditation organisation and ENIC/NARICs have signed the cooperation agreement.

The accreditation organisations have also come to the realisation that their cooperation would offer advantages to higher education institutions. Initially, this would apply to the joint programmes, such as initiatives developed as a part of ERASMUS MUNDUS. The institutions often complained that all cooperating institutions had to obtain national accreditations on the basis of the same joint programme. Obviously, this means doing the same work many times over. This is why agreements were made at the meetings in Paris (December 2006) and Berlin (June 2007) to recognise – taking into account the relevant national legal provisions – accreditation by any of the ECA partners.

As it stands today, several organisations have reached the point where they can formally recognise each other’s accreditation results or decisions. Achieving this milestone has required a great deal of extra work over the past months. The organisations feel that every formal agreement on mutual recognition should be based on a thorough bilateral analysis of the accreditation frameworks and assessment rules. A large number of these analyses were made over the last few months. As a result, the parties about to sign bilateral agreements will have access to:

–   A thorough description of both Higher Education systems

–   A thorough description of both accreditation systems

–   A signed Code of Good Practice

–    A signed declaration on the selection of experts

–   A direct or indirect comparison between the various accreditation frameworks and assessment rules

–   An external review demonstrating that the organisation complies with the Code of Good Practice

In my view, a major achievement has been reached in the space of four years. Even those parties that will not be signing today due to incompatibility with their national legislation, are sufficiently aware of the procedures and results of their fellow accreditation organisations to take rapid action when approached by individuals or institutions. This is a win-win situation for all those involved in the process of accreditation!

I should make one general comment in this regard: the accreditation organisations will not be adopting the same procedures from now on. There will always be differences, due to the diverse nature of national administrative frameworks. However, it is safe to say that our cooperation has given accreditation organisations the confidence that they would come to the same conclusion as their colleagues when assessing the same programme or institution on the basis of their own methodology and procedures! The concept of “earned trust” in a nutshell, in other words”

What can we learn from the ECA experiences?

Firstly, it has become clear to all ECA partners that “trust” is an extremely important prerequisite for international cooperation. It has also become clear just how much effort this same “trust” requires. Partners may well decide to trust one another from the outset. With a system in which external quality assessment plays an important role and a large number of new players on the higher education market, such ‘blind trust’ would have sent out an inappropriate signal, however. This is why the ECA members have made a conscious decision to opt for the concept of “earned trust”. A wholly justifiable decision was made to opt for a combination of formal agreements, analytical benchmarking and learning about each other’s regulations, procedures and working methods on a personal basis. This combination takes a relatively great deal of time but has led to excellent results. These results are based on empirical methods and are thus acceptable to those who deal or work with them in practice.

Another benefit of the ECA partners’ efforts over the past few years has been the recognition that seemingly divergent systems can be unified within a single system of mutual recognition. Our systems, after all, differ greatly: our colleagues at CTI have been accrediting engineering programmes for over 70 years. Our colleagues at NOKUT have a range of assessment tools at their disposal, of which accreditation is but one. Our colleagues at the Austrian Akkreditierungsrat are charged with accrediting Austrian private universities. This can concern extremely small institutions that initially require assessment at an institutional level. The Dutch- Flemish NVAO is charged with assessing well over 3000 Dutch and almost 1500 Flemish programmes within the space of approximately 7 years. On paper, at least, these differences in terms of remit, units to be assessed and working methods seem so great that mutual recognition of accreditation procedure results would not prove a simple task, let alone self-evident. It has taken in-depth analyses to build the sort of trust that can yield results.

A third result of the ECA approach is the fact that we can now translate our results into transparent information for institutions, students and the labour market. Call us idealistic, but we believe that an information-tool currently being developed by a number of ECA members – of which you will be hearing more today and tomorrow – will benefit all three stakeholder categories. The institutions will gain easy access to information on the quality of programmes at fellow institutions, allowing them to set up a high-quality network of programmes. The students will be able to arrange for the continuation of their studies abroad on the basis of reliable information on the quality of assessed programmes. The labour market will also benefit, as employers will be able to check the assessed quality of the programmes completed by potential employees and actively recruit graduates from specific programmes.

The ECA experiences clearly demonstrate to national and international politicians that there is room for more cooperation within the desired European framework for higher education. This will, however, require long-term investments in terms of manpower and means. The ECA quest for mutual recognition potential has clearly demonstrated that this can be achieved, but only on the basis of hard work, dedication and respect for the existing differences. This means the idealism of the Bologna signatories can be realised in practice, although the process may take considerably longer than they originally believed in all their idealistic innocence back in 1999.

Which tasks lie ahead in the years to come?  

The ECA partners are highly interested to see whether the path they have chosen is also open to countries and organisations that do not apply accreditation to ensure external quality assurance. A large number of quality assurance systems are applied throughout Europe. Each of these is in line with the relevant Higher Education systems and has its own specific characteristics and background. It would be extremely valuable to assess whether the ECA approach is suitable for broader application and whether the mutual recognition of accreditation results could also be applied to the results of other external quality assurance systems.

A second challenge lies in dealing with the dynamics of the Higher Education systems themselves. It has become clear over the past few months that several accreditation-based systems are to undergo major changes. Germany may well be the most poignant example, but the Netherlands, Norway, Spain, France, Switzerland and possibly Austria will also see more or less radical modification of their current systems. This means the ECA partners will have to invest in obtaining trust in the newly developed systems over the coming years. This may be a daunting task, but the existing basis of “earned trust” will obviously form a solid basis for the development of the phase to come.

The third challenge may well be the most important of all: although the mutual recognition of accreditation resultants and decisions may be a noble and useful goal, organisations do not go to such lengths for their own benefit. The objective is to increase the mobility of their staff and students, while contributing to the establishment of a culture of quality within the higher education sectors and institutions themselves! If this is to be achieved, the accreditation systems and accreditation organisations must be regarded as partners in the higher education system, despite their role as assessors of educational and institutional quality. Despite these sanctioning powers, I am well aware that the great majority of European higher education institutions are working to monitor and improve the quality of their own educational programmes and appreciate the role of external impulses. The accreditation organisations’ theoretical conflict of interest will thus require a continual focus on potential practical implications and faith in the institutions’ own capacity to improve educational quality.

A fourth challenge lies in dealing with the quality labels, the EU is so keen to stimulate. Rather than existing in parallel to the national accreditation systems, these labels should be incorporated within them. Institutions and programmes must invest a great deal of manpower and means in order to obtain even a single accreditation. National accreditations are often required by law; the simultaneous obtainment of “professional accreditations” would mean a great improvement in terms of efficiency and effectiveness.

The fifth and final challenge is to ensure that the ties between European higher education and higher education sectors in other continents are improved. We believe our methodology and working methods can serve as an example, and are aware that the issue of mutual recognition is also under serious consideration in South America thanks to ANECA. There are also various other initiatives in Asia and the Pacific region, which we are duty bound to examine and learn from. Hopefully, we will learn how the various continents view our performance and results tomorrow.

In closing, ladies and gentlemen, I can only say that it gives me a certain sense of pride to have outlined the ECA partners’ quest and the results it has yielded. We can now “sell” this quest as a roadmap that has proven successful and will continue to do so in future. We know there is still a lot of work ahead, but are confident our efforts will benefit our countries, higher education institutions, students and labour markets. A great source of satisfaction to be sure …….