higher quality 3

March 1998

An agenda for quality

John Randall, Chief Executive

I welcome the Government's positive response to the Dearing Report's proposals on quality. The Government paper Higher Education for the 21^st Century endorses all of the specific recommendations concerning quality, and acknowledges the work already undertaken by the Agency to prepare for their implementation. Now it is for the sector and the Agency, working together, to turn recommendations into reality.

This edition of higher quality sets out for consultation the Agency's agenda for quality, based on the Dearing proposals. The Agency will work with institutions and other stakeholders to provide public assurance of the quality and standards of higher education. In doing so it will define:

a framework for qualifications and awards at all levels of higher education;
threshold standards, relative to that framework, across all subject areas;
guidance to support institutions in providing clear specifications for each of the programmes they offer;
codes of practice designed to secure the quality of the student learning experience within institutions;
the public information that should be available to meet the needs of all stakeholders.

Quality will be assured and enhanced by:

reviews of institutional systems for safeguarding the quality of provision and the standards of awards;
reviews of programmes in all subject areas, to measure the outcomes they achieve against relevant threshold standards and the objectives specified for the programmes;
promulgation of good practice.

The Agency seeks to develop an effective system for assuring quality and standards that:

is efficient, economical and reduces the burden of external scrutiny of institutions;
satisfies the information needs of a wide range of stakeholders;
enables funding bodies to satisfy themselves that value is being obtained for the public funds they distribute;
enhances quality through continuous improvement;
promotes public confidence in United Kingdom higher education.

To achieve this the Agency will first consult upon, and then carry out trials of the quality assurance model proposed by the Dearing Report. The paper that follows develops the Dearing model, adapting it where appropriate to take account of stakeholder needs and operational feasibility. Views are invited on the Agency's proposals.

The consultation addresses a number of audiences. Some will be concerned with the operational detail of new procedures and will wish to comment specifically on this. Some will wish to know of progress made in general terms, and may wish to respond with general comments. Some, particularly those in key decision making positions, will wish to address broad matters of principle.

Because the paper seeks to cater for all of these audiences it is fairly full, and perhaps too detailed for those concerned primarily with the broad thrust of the new agenda. So for key players and stakeholders, concerned with the larger issues, let me identify points on which views would be particularly welcome:

Employers, potential students, funders and other users of public information

What types of information about the quality of courses and of the student learning experience are important to you? Do you need comparative information about courses in the same subject in different institutions? Do you need information about individual programmes that may be multidisciplinary and in some respects unique to the providing institution?
The Dearing Report recognised that setting out clearly what programmes are intended to achieve is important for students and employers alike. A draft of a programme specification, showing how intended outcomes might be set out, is at Annex A. Does this give you the information you need?
Is there other quality information that you need? Is information about placement into employment important to you?

Heads of Higher Education Institutions

Given flexible application to take account of the differing nature of individual institutions, is the overall model one that could work constructively with the internal procedures of your institution?
Do you agree that it is sensible to seek to draw upon the expertise of external examiners to help meet the need for information whilst minimising external scrutiny processes? Are there measures, additional to those suggested in the paper, you would wish to propose to safeguard the integrity of external examining in your institution?

All responses to the consultation will be welcome, and will be taken fully into account. It will be particularly helpful to have views submitted on behalf of higher education institutions, employer organisations, professional bodies, subject associations and organisations representing students and staff.

Midway through the consultation period the Agency will hold a major consultative conference, with keynote contributions from Government, employers and institutions. The date for your diary is 20 April, in Glasgow - full details are given on the back cover of this edition of higher quality.

Your views matter, whether on broad principle or operational detail. I look forward to hearing them.

John Randall, Chief Executive

Part I

Consultation: developing the quality assurance and standards framework for UK higher education

Introduction

1
The report of the National Committee of Inquiry into Higher Education (NCIHE) chaired by Sir Ron Dearing - 'Higher Education in the Learning Society' - and the report of the NCIHE's Scottish Committee chaired by Sir Ron Garrick were published in July 1997. The QAA's response to both reports was published in the November 1997 edition of higher quality.

2
The Dearing and Garrick Reports outline a model for assuring quality and standards in UK higher education, and propose that the Quality Assurance Agency (QAA) should manage its development and implementation. The model is expressed in general terms; its implementation requires the development of much operational detail.

3
The model is most likely to be successful if there is a broad consensus of support for it within the academic community and amongst the stakeholders in higher education, including potential students and employers as users of public information generated from quality assurance processes. There are also statutory requirements to be met: the funding bodies (HEFCE, HEFCW, DENI, SHEFC) have a duty to secure that provision is made for assessing the quality of education provided in institutions they support. HEFCE, HEFCW and DENI have contracted the Agency to undertake this assessment on their behalf, and to ensure that the quality of information provided satisfies public information and quality enhancement needs. Discussions on a future contractual relationship with SHEFC are proceeding.

4
The Agency is now requesting the views of the higher education sector and of stakeholders on developing the main elements of the proposed model. Parts II-VII overleaf report on progress so far and the Agency's proposals in relation to:

developing qualifications frameworks in the UK (Part II);
developing a template for programme specification (Part III);
developing benchmark information on subject threshold standards (Part IV);
defining the subject areas for benchmarking work (Part V);
developing the codes of practice and institutional review (Part VI);
strengthening the external examiner system and developing the role of the registered external examiner (Part VII).

5
Parts II and IV report largely on work in progress and any general comments would be welcome. In Part III views are sought on a draft template for programme specification. In Parts V-VII there are specific questions on subject areas, codes of practice and external examining, on which the Agency would welcome your views. Part VIII summarises the issues and questions on which comments are sought. Comments may be made on any aspect of the proposals, whether or not highlighted in a specific question.

The Dearing/Garrick model

6
The Dearing and Garrick Reports propose a revised approach to quality and standards that builds upon the experience of earlier models, but which is broader in scope and provides an integrated model that takes account of the needs of all stakeholders. It offers the potential for a reduction in the burden of external scrutiny on institutions, to be achieved in three ways:

by a clearer specification of what is expected of institutions;
by a focus on outcomes rather than a detailed investigation of teaching and learning processes;
by combining the information needs and review processes of different external agencies.

7
The focus on outcomes does not ignore the importance of the processes that lead to them, nor the need for rigorous action to assure quality and standards. However, a greater emphasis is placed upon the responsibilities of institutions in these areas, with the discharge of those responsibilities to be monitored through a process of institutional review linked with national codes of practice. Verifying the robustness of an institution's procedures in practice should mean that greater reliance can be placed upon them, thus allowing a lighter touch at the subject or programme level.

8
Academic standards will be made more explicit through several mechanisms:

the qualifications framework will be developed, which will have generic qualifications definitions and defined level descriptors, linked to awards and credits, at all levels of higher education;
the qualifications framework will be linked to subject benchmark information in order to provide a broad statement of the student attainment that is expected at the threshold level within a broad academic discipline, for specific awards and types of programme;
the specific objectives and intended outcomes of each programme of study will be set out by institutions in a programme specification;
individual student attainment could be set out in a Progress File recording learning and performance grades.

9
Verification of the attainment of standards on taught HE courses, at award and programme levels, and achievement of the programme objectives stated by the provider, will be undertaken by registered external examiners (REEs), whose reports will form the basis of information to be published by the Agency.

10
A range of codes of practice published by the Agency will provide a basis for assuring the quality of the student learning experience. Institutions will be free to adopt these or to demonstrate that they have in place measures that have equivalent effect. Compliance will be verified through periodic institutional review. In addressing achievement of programme objectives in taught courses, the assurance process will also be concerned with aspects of the quality of the student experience at the subject/programme level, for example, design, content and delivery of the curriculum, and the approach to teaching, learning and assessment.

11
The Agency will also wish to consider ways to take account of student and employer views in its assurance process. This could include use of statistical data such as first destination.

12
This proposed approach will enable the Agency to contribute significantly to public information and to quality enhancement through published reports arising from institutional and subject reviews and from wider thematic reviews. The information will also be available to contribute to institutions' own reviews.

13
There is a considerable body of experience and previous work in the higher education sector on which the Agency will be able to build in taking the Dearing/Garrick model forward, for example: the experience of audit and assessment; the work in the Graduate Standards Programme (GSP); the Harris review of postgraduate education; existing initiatives in developing programme specifications and qualifications and credit frameworks; HEQC review of the external examiner role; and existing codes of practice and guidelines.

The purposes of quality assurance

14
Judgements on external quality assurance processes must have regard to the purposes those processes are intended to serve. Those purposes include, inter alia:

to assist institutions in enhancing the quality of their provision;
to promote public confidence, at home and overseas, in higher education and the standards of awards;
to enable the funding councils to fulfil their statutory responsibilities;
to generate reliable public information that is helpful to potential students, employers, parents, government, funding councils and the institutions themselves;
to ensure that there is clarity and transparency about the purposes of programmes and the meaning of awards;
to provide a measure of accountability for the resources provided by the public purse and individuals to fund institutions.

15
It is against these requirements, and a concern to reduce the burden and costs of external scrutiny of institutions, that the proposed model should be judged.

16
The Dearing and Garrick Reports and the Joint Planning Group Report (JPG) (that resulted in the establishment of the Agency) seek a lighter touch in external quality assurance. The key to this lies in a reduction of duplication of effort. In part this will come through better co-ordination of external scrutinies that are now separate (for example subject/programme level review by the Agency and accreditation by professional and statutory bodies), by building on existing collaboration, using common timetables, paperwork and personnel. It will also come through a better inter-relationship between external and internal scrutiny processes at institutional and programme levels. There could also be greater differentiation in external scrutiny generally, with provision previously judged to be good being subject to review on a longer cycle, or at a reduced intensity (see also paragraphs 29-31).

Testing the model

17
The Dearing/Garrick model offers a way forward to secure a multiplicity of purposes and meet stakeholder requirements, whilst at the same time reducing the number of external engagements that institutions have to deal with and thus reducing burden on them. However, the detail has still to be worked through, and the Agency wishes to consult widely with institutions and others and to test carefully as the work proceeds. The present consultation exercise is the first step.

18
There will need to be careful planning and trialling to test the central method for scrutinising standards proposed by Dearing, namely the use of REEs to make judgements on provision by reference to subject threshold standards, a national qualifications framework, and declared programme objectives. The trialling in 1998-99 will be concerned with establishing whether the method is robust and capable of generating information that satisfies the purposes of quality assurance and the needs of the stakeholders. Trialling will take place initially in three subject areas - chemistry, history and law - and benchmarking groups in these subjects are now being established. The host institutions for the trialling will be mainly in Scotland and Wales where quality assessment will be completed by the end of the current academic year. External examiners will be drawn from throughout the United Kingdom for the trials, in consultation with the host institutions.

19
The outcomes of the trials will be reports on method, not reports on quality and standards in the host institutions.

Generating information

20
It is a characteristic of quality assurance systems in UK higher education that they generate information that is published. This is an important part of accountability, transparency and meeting the information needs of stakeholders.

21
Considerable amounts of information can be generated from scrutiny at the institutional and programme levels. However, the more information that is required to be generated, the greater is the cost. That cost is in part the direct cost of conducting and receiving external scrutinies, and in part the opportunity cost to the institution of dealing with a high level of external scrutiny. If that cost is not to escalate it is necessary to define carefully the information that is required, by reference to the purposes for which it is needed, and then to design efficient, effective and economical ways of generating that information.

22
An important question for the consultation is whether the Dearing/Garrick model will be capable of generating all the information that is required by the different stakeholders, without imposing undue burdens on institutions. Public information is required to inform student choice of institution and programme, to inform employers about the attributes they might reasonably expect in a graduate of a particular programme, and to promote confidence in the value of the public investment in higher education. Dissemination of self-generated information by institutions is insufficient: there must be independent, external verification.

23
The funding councils will require information from the new quality assurance process for a number of reasons. They, like the Agency, wish to ensure that sufficient information is available to potential students, employers and other users of higher education so that they can make informed choices. In this connection, they believe that the quality of the student learning experience is a central issue. They need to satisfy themselves that value is being obtained for the public funds they are distributing.

24
Most importantly, the funding councils wish to take opportunities to enhance quality. With this in mind, they are looking for a quality assurance process that will provide universities and colleges with incentives for continuous improvement. They intend to use the outcomes from the quality assurance process to reward quality through their distribution of funding, a practice which already exists in Scotland and Wales. The HEFCE will be consulting institutions later this year on developing a link between quality and funding. The joint SHEFC-COSHEP ('Miller') Review in 1997 addressed this matter in Scotland. The HEFCW will be consulting on a new funding methodology for Wales in early summer 1998.

25
These requirements from the funding councils mean that information about quality is required on a consistent, comparable basis. It needs to be sufficiently robust to withstand challenge. This implies a consistency of process, managed by the Agency, with a clear link between judgements and the evidence on which they are based. Improved information on outputs will reduce the need for direct gathering of information on the teaching and learning process. But the funding councils will wish to be satisfied that, in total, the information gathered does satisfy all the needs identified above. In particular, they believe that evidence about the delivery of the curriculum as experienced by the student should be an important part of the picture.

26
The Agency believes it should be possible to generate this evidence through strengthening the external examiner system. Some external examiners - the REEs - will have a role of reporting to the Agency as well as reporting to the institution. They will be able to draw upon their participation in monitoring and internal review processes and on direct discussions with students and staff to provide independent and informed commentary on the quality of the student experience.

27
The Agency envisages requiring the REEs to report to it on a cyclical basis such that all subjects are covered in a five or six year period. In the year in which the Agency was reviewing a given subject, the REEs in each institution would produce a report, which would form the basis of information to be published by the Agency, dealing with that subject in that institution. It would focus upon the extent to which subject standards were being achieved in relation to the benchmark information and the extent to which the programme was offered at the right standard for the award, and would provide some commentary on the student learning experience. The Agency would produce also a sector-wide subject report covering all provision.

28
The Agency recognises that there are some concerns about the proposed REE role and dual reporting responsibility. This is discussed further in Part VII.

The review cycle

29
The JPG recommended that reviews should be conducted on an eight-year cycle. This would have implications for the currency and utility of the public information generated over such a period. At present the Agency envisages that both institutional and subject level scrutinies would be conducted on an approximately five-year cycle, in line with the Dearing/Garrick recommendations. However this could be varied.

30
It is at the subject level that the major interaction with professional and statutory bodies occurs. Such bodies have their own cycles of accreditation and review. An assumption that the only basis for co-ordination is an adoption of the Agency's preferred cycle would limit opportunities for collaboration and streamlining. There will have to be flexibility on both sides to ensure the maximum co-ordination, and hence reduction in the burden created by having separate scrutinies. There will be a considerable reduction in burden if the purposes of two forms of scrutiny can be satisfied through a single process.

31
There could be differentiation of approach to reflect the relative strengths and weaknesses of institutions based on the evidence of quality audit and quality assessment so far and the evidence to be generated by the Agency's assurance processes. Criteria for using this evidence to determine future review arrangements will need to be developed. A differential response to the relative strengths of institutions might involve either varying the length of the review cycle, or maintaining the frequency but varying the intensity of scrutiny. Either would reduce the burden on institutions. At the same time, the Agency intends to respond promptly, at any point in the cycle, to indications that quality and standards may be at risk in an institution.

Timetable

32
Dearing proposes that the new model should be in place by the year 2000. This is a challenging timetable. It will only be possible if the Agency and the institutions work together on all the main elements of the new model over the next couple of years. The outline timetable for the main elements of the developmental and trialling agenda is shown overleaf. It may change in the light of the results of this consultation.

Request for views

33
The Agency would welcome comments on progress so far and on the approaches and proposals that are discussed in more detail overleaf. In Part III and Parts V-VII there are some specific questions on which views would be particularly welcome. The issues for consultation are summarised in Part VIII.

Responses should be sent to Martin Johnson at QAA, Southgate House, Southgate Street, Gloucester GL1 1UB, to be received by Friday 22 May 1998.

Part II

Developing qualifications frameworks in the UK

Purpose

1
This Part outlines the Agency's proposed strategy for taking forward the development of HE qualifications frameworks in the UK.

Dearing/Garrick

2
The Garrick and Dearing Reports propose that parallel, but distinct, qualifications frameworks should be established in Scotland and the rest of the UK.

3
The Garrick Report recommends (recommendation 1) 'to providers of higher education programmes in Scotland, the Quality Assurance Agency, the Scottish Qualifications Authority and the Scottish Advisory Committee on Credit and Access that they should together consider and adopt an integrated qualifications framework based around level of study and Scottish Credit Accumulation and Transfer Scheme credit points.'

4
The Dearing Report proposes (paragraph 10.45) 'a framework of qualifications for institutions in England, Wales and Northern Ireland ('other UK' institutions), as shown in Chart 10.1. It describes higher education qualifications in terms of levels of achievement - from H1 to H8. It is based on credit points, which means that different levels can be regarded as equivalent, although achievement will not guarantee automatic progression to a higher level or entry to any one institution at a particular level. The final responsibility for admissions and progression will rest, as now, with the receiving institution, and decisions will depend upon the appropriateness of the credits achieved and the prerequisites for particular pathways.' Recommendation 22 recommends 'that Government, the representative bodies, the Quality Assurance Agency, other awarding bodies and the organisations which oversee them, should endorse immediately the framework for higher education qualifications that we have proposed.'

5
There is a strong background of work in this area from which the Agency is able to draw:

i) the credit-related national developments in Scotland, N. Ireland, and Wales, and the regional developments in the credit consortia in the south and north of England;

ii) the Graduate Standards Programme (GSP) of HEQC, which explored issues to do with the understanding of first degrees, and recommended the construction of an awards framework;

iii) the Harris Review, which made a number of suggestions at the postgraduate level;

iv) the inter-consortium credit agreement (InCCA) project;

v) a HEQC/QAA project on credit levels, which includes a number of recommendations on the relationship between credit levels and: awards and awards frameworks; structures of higher education programmes; learning achievements and outcomes; assessment criteria; progression of learning; academic, professional and vocational learning; and core/key skills. It also provided an illustrative series of generic descriptors of the learning associated with each level and the ways in which such descriptors might contribute to the quality assurance and standards of credit-based learning.

6
The Agency's response to Dearing and Garrick welcomed the recommendations, although it recognised the need for the detail of the frameworks to receive further development before implementation. It is clear that there is broad stakeholder support, in principle, for the qualifications frameworks proposed by Dearing and Garrick, with some reservations in relation to some of the detail suggested in the reports. The Agency will consult fully with HEIs and other stakeholders as part of the development process.

The role and key elements of the qualifications frameworks

7
The qualifications frameworks are central to the achievement of the vision for higher education embodied in the Dearing and Garrick reports. They provide coherent and consistent structures within which all HE qualifications and their standards may be located, their purposes made explicit, their agreed credit values determined and their articulation with each other and other qualification frameworks plotted. They lie at the heart of the Dearing/Garrick strategy for 'Higher Education in the Learning Society' and, potentially, of related aspects of Government strategies on wider access and life-long learning. The qualifications frameworks will provide:

a basis for consistent public information on the definition of awards and the award of credit;
a point of reference for benchmark information for the subject benchmarking groups;
a foundation for the template within which programme specifications will be constructed;
a point of reference for registered external examiners;
a key structure to which codes of practice and institutional reviews will relate.

8
The qualifications frameworks, based on credits and levels, will be designed to support the wide range of activities undertaken within HE across the UK. The frameworks will not assume any particular pattern of course design or student learning. A key rationale of the frameworks is that they should support diversity by being capable of recognising appropriate learning however, wherever and whenever that learning takes place. Key elements of the qualifications frameworks are likely to include:

a definition of level;
a definition of credit;
a number of levels and types of awards;
generic levels descriptors;
generic awards definitions;
guidance on appropriate award nomenclature.

Progress on postgraduate qualifications

9
Work on qualifications frameworks at the postgraduate level is already under way as a consequence of the 1996 HEFCE-CVCP-SCOP Review of Postgraduate Education (the 'Harris Review'). The Harris Review identified widespread confusion, following a period of rapid expansion, about the level and nature of study recognised by postgraduate qualifications. It recommended:

a typology of postgraduate awards and the standardisation of award nomenclature;
the development of a national directory of postgraduate programmes;
a code of practice for postgraduate research.

10
Following national consultation, CVCP and SCOP endorsed the Harris recommendations in principle and referred their detailed development to HEQC/QAA. The work, overseen by a broadly constituted steering committee, has involved a project group of senior representatives of nine institutions, reflecting the diversity of the sector, who have agreed to supply policy advice and trial options. Work to date has sought to identify a national approach to achieving greater clarity about postgraduate provision.

11
The Dearing Report further endorsed this work and made a number of specific recommendations with particular relevance to postgraduate education, including:

the incorporation of a number of explicit postgraduate levels within the national qualifications frameworks;
a code of practice 'for postgraduate education';
explicitness among institutions about the development of 'professional skills' (eg, communication, self-management and planning) in postgraduate research students.

12
The Agency's current programme of work on postgraduate qualifications therefore reflects both the Dearing and Harris reports, and is now being taken forward as part of the development of the national qualifications frameworks. It is expected that, at the postgraduate level, the qualifications frameworks in the UK will converge. A discussion paper on postgraduate matters, containing further detail on principles, emergent issues and options for consideration, is currently in preparation.

Progress on credit frameworks

13
Much progress has been made on developing credit frameworks through national developments in Scotland, Northern Ireland and Wales and through regional credit consortia in the north and south of England. The inter-consortium credit agreement project (InCCA) has made considerable progress and its work will provide a valuable resource to support further development.

Development strategy for the qualifications frameworks

14
The Agency is now embarking on a development programme, building on the above work, to develop the parallel qualifications frameworks in Scotland and the rest of the UK.

15
In Scotland, the QAA Advisory Committee for Scotland, in conjunction with COSHEP and the Scottish Qualifications Authority (SQA), has now established a qualifications framework development group to take this development forward. It is important to note that there is a parallel joint initiative in Scotland with COSHEP, QAA, SQA, and SOEID to integrate the higher education qualifications framework within a comprehensive post-compulsory credit and qualifications framework for Scotland.

16
Similarly, a qualifications framework development group is being convened to take forward the development of the parallel framework for the rest of the UK. A joint forum on qualifications is also being established with the Qualifications and Curriculum Authority (QCA) to ensure appropriate articulation between qualifications frameworks at other levels and of other types. It is envisaged that a representative group of institutions will be constructed in due course to advise the Agency and to test options as they are developed.

17
There will be an element of common membership at both officer and member level, as well as common reporting to the QAA Board, to ensure effective parallel and consistent development over the UK. It is intended that the initial focus of the group will be at the honours degree level in order to provide draft information to the trial subject benchmarking groups at an early stage.

18
Developing (and subsequently maintaining) the qualifications and credit frameworks will involve:

describing and agreeing the 'architecture' of the frameworks in the form of number and nature of levels, credits, and qualifications;
describing levels and qualifications in terms which are easy for all stakeholders to understand and to work with;
ensuring articulation between the development of the qualifications and credit frameworks and the identification of benchmark information for standards at subject and programme levels (a process of continuing iteration);
ensuring articulation with further education, vocational and international qualifications and credit frameworks;
describing and agreeing the principles and procedures that relate to the operation of the frameworks;
producing guidelines on the location of qualifications on the frameworks;
producing framework documentation, linked to a code of practice;
proposing and agreeing mechanisms for maintaining the frameworks over time to ensure their continuing currency and credibility.

19
A number of key issues will need to be addressed by the development groups as their work proceeds. These include:

the nature, number and form of the dimensions to be used in constructing the frameworks;
the extent of flexibility in the frameworks;
how the frameworks can encompass all HE awards at all levels;
whether the Dearing/Garrick models have specified the number and types of level and qualifications appropriately (for present and future needs);
the extent to which adoption of the frameworks will be required of institutions as a basis for sectoral self-regulation and for receipt of public funding;
the concerns expressed about aspects of the specific frameworks proposed by Dearing and Garrick.

Request for views

20
Comments are invited on progress so far and on the issues identified in paragraph 19.

Next > Part III

higher quality 3

March 1998

Contents