The handbook for a pilot study of an integrated quality and enhancement review

QAA 126 09/06

Part one: Background and development

Introduction

1 The mission of the Quality Assurance Agency for Higher Education (QAA) is to safeguard the public interest in sound standards of higher education qualifications and to inform and encourage continual improvement in the management of the quality of higher education. QAA undertakes reviews of higher education provision delivered in further education colleges (colleges) on behalf of the Higher Education Funding Council for England (HEFCE) in the context of the latter's statutory responsibility for ensuring that provision is made for assessing the quality of the education that it funds. Colleges are not themselves awarding bodies, but work in partnership with such bodies, in particular Edexcel and/or one or more higher education institutions (HEIs). As awarding bodies, HEIs retain ultimate responsibility for maintaining the academic standards of their awards and assuring and enhancing the quality of students' experience. Further details of an HEI's responsibilities for the quality and standards of its awards may be found in the Code of Practice for the assurance of academic quality and standards in higher education (Code of practice), Section 2: Collaborative provision.

2 This Handbook has been written to support the pilot study of integrated quality and enhancement review (IQER) only. Part one provides an operational description and Part two provides more detail of the stages of IQER. The contribution of the Handbook to the IQER process will be evaluated as part of the evaluation of the method.

The IQER method

Background

3 HEFCE has decided that during the period 2007-08 to 2011-12 colleges delivering publically-funded higher education programmes will have some engagement with QAA. The ending of the current cycle of academic review of subjects has provided an opportunity to design for the first time a review method specifically for higher education in colleges. Following a period of careful consultation, it has been decided to pilot the IQER method described in this Handbook. A wide range of considerations informed the development of IQER, including the work of the Office of Public Services Reform, Public Services Expenditure Inspection, Higher Education Regulation Review Group, the Bureaucracy Reduction Group and the Department for Education and Skills/HEFCE/Learning and Skills Council Joint Progression Strategy Group. In addition QAA has taken account of evaluations of reviews and other sources of feedback from colleges and their representative bodies. They have expressed a preference for:

a peer review, rather than an inspection process
a process with self-evaluation at its heart
a development from Academic Review
an external review process comparable to Institutional audit, used within HEIs
a method which draws on evidence from college inspections and other external reviews and provides evidence for future inspections
an approach which recognises the specific needs and circumstances of colleges and the environment in which they operate.

4 Both with the range of demands on colleges in mind and reflecting current attempts to reduce the regulatory burden on educational institutions, IQER attempts to avoid duplication of effort by encouraging colleges to use existing procedures and reports rather than preparing lengthy bespoke documentation for the purpose of review. It is, however, appreciated that in some cases the preparation of additional material may be unavoidable: we say more about this later (see in particular Annex B).

5 IQER will be carefully evaluated both during and at the end of the pilot, in the light of the views and experiences of a range of stakeholders. Following this evaluation process, the IQER method will be revised as appropriate, so that it can be published and introduced from January 2008. It is the particular intention of QAA to minimise any duplication of effort between IQER, Institutional audit and Collaborative provision audit (details of these methods are available at www.qaa.ac.uk/reviews/institutionalAudit/handbook2006/default.asp and www.qaa.ac.uk/reviews/institutionalAudit/collaborative/supplement/default.asp) The development of a new method of Collaborative provision audit will not start until the IQER pilot has been evaluated, so that the evaluation can inform both the revision of IQER and the development of the next process for Collaborative provision audit. In addition, the IQER and Institutional audit processes will be reviewed and evaluated jointly at the end of the IQER cycle, to enable QAA to advise HEFCE as to the most effective and efficient way of proceeding. QAA is also currently exploring with Ofsted the possibility of carrying out IQER for colleges with less than 50 full-time equivalents (FTEs) students funded by HEFCE at the same time as college inspections. QAA anticipates that this model will be included in the pilot.

The method of review

6 IQER has a dual focus. While its overall aim is to safeguard the public interest in the quality and academic standards of higher education delivered in colleges by providing objective, independent and accessible information by way of reports, its complementary stages in the method: enhancement-focused Developmental engagement and formal assessment through Summative review, have been designed to offer a supportive framework within which colleges can continue to develop their own quality assurance and enhancement processes. We explain Developmental engagement and Summative review later (see paragraphs 14-20), but for the moment it suffices to say that in most cases Developmental engagement teams will include one or two members of the college itself, and that colleges' first Developmental engagement (in many cases their only Developmental engagement) will focus primarily on their evolving approaches to student assessment. This attempt to explore, in an open and collegial way, the challenges colleges face and the manner in which they are meeting them, is designed to provide an understanding of, on the one hand, colleges' internal culture, policy development and practice and, on the other, the manner and extent of their engagement with the Academic Infrastructure.

7 The Academic Infrastructure is a set of nationally agreed reference points relating to effective practice in the setting and management of academic standards and quality in higher education. It comprises: the Code of practice; The framework for higher education qualifications in England; Wales and Northern Ireland (FHEQ) and its companion for Scotland; Subject benchmark statements; and the Foundation Degree qualifications benchmark, all published by QAA. Programme specifications and progress files also contribute to the Academic Infrastructure and QAA provides guidance on these. The centrality of the Academic Infrastructure to IQER means that reference is made to it throughout this Handbook.

8 IQER reports aim to be useful to both colleges and their awarding bodies in maintaining standards of provision, enhancing quality and building colleges' capacity to manage delegated responsibilities for their HEFCE-funded provision. The reports have also been designed to facilitate an overview analysis of the operation of higher education in colleges in England, to identify and promote good practice, and ultimately, in conjunction with the Higher Education Academy, to develop good practice guidelines.

9 IQER acknowledges, first and foremost, that colleges are not awarding bodies. QAA appreciates that ultimate responsibility for the outcome of student assessment and the standards of academic awards lies with awarding bodies, whether Edexcel or HEIs; it is aware of the wide variety of collaborative arrangements between HEIs and partner institutions; it understands that, just as some HEIs have multiple partnerships, so do some colleges engage with a number of HEIs; it appreciates that some directly funded colleges have no HEI partners and that all their higher education programmes are validated by Edexcel. QAA is aware that the level of delegated responsibility for assessment procedures varies widely on the basis of factors which may include the size and maturity of the partnership, the geographical location, the nature of the educational provision and the normal practices of the awarding body. Given this context, and the fact that QAA has no preferred contractual arrangement other than that any such arrangement must permit the awarding body to be able to assure itself as to the quality and standards of its collaborative provision, IQER takes as its starting point each college's partnership agreements and the manner in which these frameworks are operated.

10 While IQER aims to build on the strengths of Institutional audit and other review methods, it reflects also the particular circumstances and characteristics of the large and diverse further education college sector. The IQER method has been designed within the context of QAA's mission, standards and values as set out in Annex A. IQER shares a number of features of Institutional audit and other QAA review methods in that it:

focuses on the management of higher education quality assurance in colleges
is based upon the Academic Infrastructure
expects that colleges and their awarding bodies will carry out reviews and evaluations of their own programmes
involves an evidence-based peer review process characterised by a continuing dialogue with, and the provision of regular feedback to, the college
relies on a college's self-evaluation and visits to colleges to meet students, staff and, where appropriate, external stakeholders
acknowledges, in part through the facility for the student representative body to make a written submission, the centrality of students' experiences
uses available data for higher education in colleges
carries out selective and focused enquiries
results in oral and written reports based on evidence-based evaluations, judgements and recommendations
requires the review team to take collective responsibility for judgements, conclusions and recommendations about the higher education provided in each college
identifies good practice and areas for development or improvement in the management, quality assurance and enhancement of higher education in colleges.

11 IQER is not a predominantly discipline-focused activity and only involves teaching observation or the direct scrutiny of student work where this is directly relevant to a particular Developmental engagement subject or theme. Its primary focus is on all levels of a college's management of higher education; it has been designed as a flexible, negotiated and transparent process which reflects and values the diversity of the college sector as well as exploring colleges' engagement with the Academic Infrastructure; it provides independent and accessible public information about directly, indirectly and consortium funded college-based higher education; it encourages improvements in the quality of higher education in a supportive and developmental manner, it emphasises active student participation. It encourages public awareness of Foundation Degrees. It is economical of time in its risk-based approach, its use of information which is already available and its contribution to future reviews by QAA and other bodies, it recognises the status and requirements of awarding bodies and dovetails into QAA's existing audit procedures; and it provides the opportunity for HEFCE to link judgements relating to quality and standards to funding decisions, should it so wish.

12 In short, IQER is designed to:

assist colleges in identifying and confirming their strengths and limitations in their higher education quality assurance management and self-evaluation processes
provide independent and reliable information about higher education provided in colleges to present to prospective students and to other interested stakeholders
contribute to colleges' increased and enhanced capacity to manage their responsibilities for higher education, as set out in partnership agreements
identify features of good practice
deliver constructive reports to colleges, their awarding bodies and HEFCE

The IQER method

13 IQER applies to all HEFCE-funded higher education in colleges and it is structured around three core questions, each focusing on colleges' contractual obligations, which, it is appreciated, vary widely. The core questions are:

i Does the college discharge its responsibilities effectively, as set out in its partnership agreement(s), for the management and delivery of the standards of the award(s) it offers on behalf of its awarding body(ies)?

ii Does the college discharge its responsibilities effectively, as set out in its partnership agreements, for the management and assurance of the quality of the learning opportunities to enable students to achieve the intended learning outcomes?

iii Can reliance be placed on the accuracy and completeness of the information that the college is responsible for publishing about itself and the programmes it delivers?

14 IQER embraces the full portfolio of HEFCE-funded higher education provision, regardless of whether funding is received directly, indirectly or through a HEFCE funded consortium. It is a peer review method that considers the manner in which colleges satisfy themselves that they are discharging the responsibilities assigned to them by their awarding bodies in relation to the assurance of the standards and quality of the higher education provision they deliver, with particular reference to the Academic Infrastructure.

The following flow chart summarises the IQER method for the pilot study, though for a fuller diagram of the IQER process proposed for a full five year cycle, see Annex D:

Activities
Planning
	QAA provides information to all colleges about the IQER method.
	Where necessary, QAA negotiates the number and timing of reviews.
	QAA confirms the number of reviews for each college.
	QAA provides the college with a deadline for indicating its preferences for the order of reviews and the remit for the Developmental engagement(s) in relation to internal college events.
	The college submits its preferred order of reviews and Developmental engagement specificiation(s) to QAA.
	QAA collates all responses into a schedule, making any necessary amendments following negotiation with colleges.
	QAA produces a definitive schedule, listing the timing and the outline Developmental engagement specification(s) for all colleges. QAA confirms each college's pilot study schedule and provides dates for the submission of the self-evaluations.
	Recruitment and training of coordinators takes place. QAA assigns a dedicated coordinator to each college.
	Recruitment and training for specialist reviewers (SRs), Institutional nominees (INs) and facilitators takes place. Teams are composed.
	The IQER cycle college planning meeting takes place. The coordinator and a QAA officer visits the college to brief staff and students, agree lines of enquiry for the Developmental engagement(s), the structure of the self-evaluations and the nature and extent of review activity. If the college and its partners consider it appropriate, the awarding body(ies) also take part in this meeting.
Developmental engagement process
	The college submits its self-evaluation for the Developmental engagement to QAA; students submit their optional written submission; QAA sends the self-evaluation to the appropriate coordinator, who confirms its appropriateness as a basis for the visit.
	The college sends its self-evaluation to the Developmental engagement team. The team comments on the self-evaluation, and forwards its comments to the coordinator.
	The coordinator meets appropriate college staff and student representatives for a Developmental engagement preparatory meeting. This includes discussion of the self-evaluation and the detailed planning of the review. If the college and its awarding bodies consider it appropriate, the awarding bodies also take part in this meeting.
	The Developmental engagement proceeds to conclusion. The report and action plan are finalised and distributed.
	The college prepares for any further Developmental engagement and for the Summative review.
Summative review process
	The college submits its self-evaluation for the Summative review to QAA; students submit their optional written submission. QAA sends the self-evaluation to the coordinator, who confirms its appropriateness as a basis for the review.
	The college sends its self-evaluation to the reviewers. The reviewers comment on the self-evaluation and forward their comments to the coordinator.
	The coordinator meets appropriate college staff for a Summative review preparatory meeting. This meeting involves discussion of the self-evaluation and the identification of a date for a second visit if this becomes necessary. If the college and it partners consider it appropriate, the awarding body(ies) also take part in this meeting.
	The review team meets at the college for Summative review visit 1.
	The review team writes its report off-site and formulates its provisional judgements.
	The coordinator sends the draft report including the provisional judgements to the college and its awarding body(ies) for comments on factual accuracy.
	The college responds to the report with additional evidence if required.
	Either the review team members confirm judgements off-site; or if they have insufficient evidence or in the case of a potential no confidence or limited confidence judgement, the coordinator informs the college that the provisional second visit is required.
	The review proceeds to conclusion. The report and action plan are finalised and published.
	The IQER cycle is complete unless a second Developmental engagement has yet to take place.

15 As the above chart indicates, IQER involves desk-based analysis and evaluation of documentary evidence, scrutiny of the accuracy of published and other available information and college-based meetings with staff, students and other stakeholders. These meetings may include representatives of awarding bodies, however, this decision rests with the college and its awarding bodies and should be taken in the context of the college's partnership agreements. IQER's two complementary processes, Developmental engagement and Summative review, engage with key aspects of colleges' discharge of their responsibilities for quality and standards in the HEFCE-funded higher education provision they deliver on behalf of their awarding bodies. They also evaluate how colleges engage with the Academic Infrastructure and meet any contractual obligations they may have to ensure the standards of higher education awards and provide high quality learning opportunities for their students.

16 The IQER method also involves an evaluation of the information about HEFCE-funded higher education programmes which colleges may publish or other information they make available to stakeholders. Review teams will not make a formal judgement about the accuracy, integrity, completeness and frankness of any information awarding HEIs publish about the quality of their programmes delivered through their partnerships with colleges, and will not replicate the enquiries of Collaborative provision and Institutional audit into the manner in which HEIs are meeting their obligations in respect of published information.

The IQER cycle in the pilot study

17 In the pilot study, colleges delivering HEFCE-funded higher education will usually participate in a minimum of one and, other than in the exceptional circumstances specified below, a maximum of two Developmental engagements and one Summative review in 2006-07. Colleges with less than 100 FTE students funded by HEFCE may elect to have a Developmental engagement. The proposed number of Developmental engagements for the pilot will be set out in a letter to each college from QAA, with a deadline for a response following consultation by the college with its awarding body(ies). QAA will copy all correspondence it sends to the colleges to their awarding bodies to assist effective communication. The following principles apply in calculating the number, timing and scope of Developmental engagements for each college:

all colleges with a minimum of 100 FTE students on HEFCE-funded programmes will have at least one Developmental engagement
other than in exceptional circumstances the maximum number of Developmental engagements in the pilot study will be two
colleges, in consultation with their awarding body(ies), will have the opportunity to negotiate the timing and scope of Developmental engagements (although QAA reserves the right to specify part of their focus and their timing)
the scope and number of Developmental engagements for individual colleges will be identified through a risk-based approach.

18 QAA uses the following risk factors to determine the number of Developmental engagements for each college with 100 or more FTE students funded by HEFCE. Where one or more of the following factors are present the college will have two Developmental engagements.

The risk factors are:

a no confidence and/or a failing judgement in a QAA academic review since January 2002, not redressed through a satisfactory re-review of the relevant provision
a no confidence and or a failing judgements in two or more QAA academic reviews since January 2002 irrespective of the outcomes of re-review
a no confidence judgement in 2004-05 in a review of a Foundation Degree involving teaching provided by the college in question
a no confidence judgement in a QAA Major review of healthcare not redressed through a satisfactory re-review
a no confidence or limited confidence judgement in a Collaborative provision audit or Institutional audit of a partner HEI's management of collaborative provision of higher education in further education colleges
a 4 grade or unsatisfactory judgement in leadership and management or in any subject in which higher education programmes are provided in a college's most recent inspection by Ofsted/ALI, not redressed through satisfactory re-inspection.

19 Exceptionally, usually resulting from an essential recommendation made in a previous Developmental engagement and thereby signalling that quality and/or standards are considered currently to be at risk, the number may be increased by one to a maximum of three.

20 Each Developmental engagement, normally of two consecutive days' duration, is structured around the three core questions and focuses on a developmental theme. The theme of the first Developmental engagement in each college is student assessment, as the analysis of earlier reviews shows that this is the element of higher education most in need of development. All Developmental engagements will include the reviewers' consideration of the extent to which each college has engaged with the Academic Infrastructure. Earlier reviews have also identified the use of the Academic infrastructure as an area which needs significant development in colleges.

21 Colleges with a second Developmental engagement are invited, on the basis of discussion with their awarding bodies, to propose either a subject focus or a theme from the following list. The list links closely to elements of the Academic Infrastructure and has been designed to focus on processes for quality assurance and enhancement, to support the identification and dissemination of good practice and to facilitate the recognition of risk to standards and/or quality of learning opportunities.

Academic appeals and student complaints
Accreditation of prior experiential learning
Careers education, information and guidance
Communication of information
Curricula
Employability
External examining
Flexible and distributed learning
Learning resources
Management information, student tracking
Partnerships, collaborative provision
Placement learning
Programme approval, monitoring and review
Programme specifications
Quality enhancement
Recruitment and admissions
Registration for practice
Staff capacity, qualifications and scholarship
Student achievement
Staff development
Student experience
Student involvement in quality assurance and enhancement
Student progression
Student support, progress files
Students with disabilities
Teaching and learning
Widening participation
Workplace learning

22 All subjects and themes are considered in the context of the awarding bodies' requirements as set out in the framework of the partnership agreement and will not address aspects of provision for which the awarding body retains responsibility. Nonetheless, in the developmental and participative spirit of IQER as a whole, within each subject or theme colleges have the opportunity to negotiate the specific lines of enquiry to be addressed, (examples of lines of enquiry will in due course be listed in more detail on QAA's website). As far as is practicable, colleges also have the opportunity to negotiate the timing of their Developmental engagements in the light of how they fit with other external and internal reviews, including the Summative review, and the colleges' revisions to quality assurance processes.

23 Each Developmental engagement and Summative review results in an oral report and a written report containing conclusions, features of good practice, recommendations for enhancement and matters for further consideration. Developmental engagement reports are evaluative and advisory in tone. Summative review reports contain judgements on core questions one and two and an evaluative comment on core question three (for an explanation of the core questions see paragraph 13 and for an explanation of the criteria underpinning judgements see Step 19 of Part Two below). Summative review judgements are expressed as levels of confidence, and colleges will be advised whether reviewers have confidence, limited confidence or no confidence in the soundness of their present and likely future management of relevant aspects of their responsibilities. Should they judge a college's management of and/or quality and/or standards of only one awarding body to be below the required threshold, reviewers will award a differentiated judgement of limited or no confidence to that provision alone.

24 In order to support development and capacity building, the Developmental engagement reports will not be published. They will be provided to the colleges concerned, their awarding bodies, HEFCE and QAA, and their outcomes will be summarised in the published Summative review report for the college. Both Developmental engagement and Summative review reports trigger an action plan, to be devised by colleges in consultation with their awarding bodies.

Reference points for IQER

25 The Academic Infrastructure provides the main set of external reference points for IQER and is at the heart of the IQER process. IQER teams will draw upon the Academic Infrastructure as a source of reference when considering colleges' approach to the management of their responsibilities for the delivery of academic standards and for the quality of provision. They will not do so mechanistically or to ensure compliance but in order to explore whether colleges have carefully considered the purpose and intentions of the various elements of the Academic Infrastructure.

26 All IQER teams will ask colleges about their engagement with the Code of practice, and self-evaluations should include an account of this, drawing attention to any resulting changes in practice which have taken place, any benefits accruing, any areas of difficulty encountered and how they have been addressed.

27 Reviewers will also explore colleges' use of relevant qualification and award descriptors and subject benchmark statements. As the large majority of subject benchmark statements apply to single subject honours degrees, their general application by colleges would be inappropriate. However, they do provide an authoritative reference point, and reviewers are likely to be especially interested in whether, and if so how, they have been used to inform the development of programmes such as HNC, HND and Foundation Degree.

28 Programme specifications contain definitive information on the aims, intended learning outcomes and expected achievements of programmes of study, and reviewers will explore their accuracy and usefulness to students and staff. In particular, reviewers will wish to see how programme specifications make use of other reference points in the Academic Infrastructure to define expectations for teaching, learning and assessment.

29 The following diagram provides an illustration of how college quality assurance management systems and practices interrelate with the Academic Infrastructure to help assure the student learning experience.

The relationships of the student's learning experience to the Academic Infrastructure

Relationship of learning experience to the Academic Infrastructure

HEFCE-funded consortia and lifelong learning networks

30 The primary unit of IQER reviews is normally the individual college. Nonetheless, a number of arrangements involving shared programme ownership, including HEFCE-funded consortia and lifelong learning networks designed to facilitate progression between colleges and higher education institutions, have been established. Such partnerships take a variety of forms, normally reflecting local and regional agreements and traditions, and operate with varying degrees of coordination. In such cases IQER will necessarily be adapted, and QAA will discuss with members of consortia and life-long learning networks how to ensure an optimal fit between the desired review outcomes and the characteristics of a particular arrangement. Such discussions are likely to involve the coordination of Developmental engagements by timing and/or theme or subject.

31 It should also be noted that Summative reviews, while focussing on the individual college, are also linked to the ownership of programmes and learning resources, and may, therefore, report on arrangements for the management and administration of lifelong learning networks. This includes the distribution of student numbers and other network resources between participating institutions.

IQER personnel

32 QAA will assign an officer to manage each review and a coordinator to each college to provide continuity. In the interests of building a constructive dialogue, it is recommended that colleges also nominate the same member of staff as the Summative review facilitator and as one of the institutional nominees (IN) for all review activity within the pilot study. The INs, as full members of a Developmental engagement team, will participate in all Developmental engagement activities, including the analysis of the self-evaluation, meetings with staff and students, the forming of conclusions and recommendations and report drafting. The INs will be pivotal in the formulation and implementation of the action plans and their implementation after the review.

33 Developmental engagement teams normally comprise four members: the coordinator, one reviewer and two INs. Colleges are invited to nominate a second IN, preferably on the basis of expertise and seniority in relation to the management or coordination of the college's higher education. It is hoped that the experience gained will subsequently enable the IN to contribute to enhancing the college's management of quality and standards; if, however, a college is unable to nominate a second IN, QAA will provide a second reviewer.

34 Summative review teams typically comprise the coordinator and three peer reviewers. Teams do not have INs, though it is helpful if a Developmental engagement IN acts as the Summative review facilitator. The facilitator is the single point of contact between the college and the coordinator and supports the smooth running of the review. (See Annex C for full details of IQER roles and team composition.)

Abridged method for colleges with fewer than 100 full-time equivalent students funded by HEFCE

35 Adaptations have been made to the method for colleges with fewer than 100 FTEs on HEFCE-funded programmes. Other than in exceptional circumstances, colleges with fewer than 100 HEFCE-funded FTEs will have a Summative review only, complemented, if the college and its awarding body(ies) so desire, by one Developmental engagement.

36 Where a Developmental engagement takes place, the review team will normally have only three members: one coordinator, one specialist reviewer appointed by QAA and one IN. QAA will provide two specialist reviewers if a college is unable to appoint an IN. The Developmental engagement will take place over 1.5 consecutive days in the case of colleges with 51-99 FTEs on HEFCE-funded programmes and in a single day for colleges with 50 or fewer FTE students.

37 Summative review teams will also normally comprise three members: the coordinator and two specialist reviewers, all appointed by QAA and the main event (known as the Summative visit 1 meeting - see Annex D) will take 1.5 days for colleges with 51-99 FTE students. If the reviewers need to return to the college to complete the review, the second visit will be for a maximum of two days. For colleges with less than 50 HEFCE funded FTE students, the Summative review will either take place either as a discrete event of 1 day for Summative review visit followed by a second visit of one day if this is necessary, or as a pilot event carried out at the same time as the college inspection. In this case, the team will consist of one coordinator and one specialist reviewer.

Part Two Operating the IQER process

Part two of this Handbook offers guidance to institutions, reviewers and other parties responsible for implementing or otherwise participating in the IQER pilot study. For ease of reference, it has been arranged in the following colour-coded sections:

Section one: IQER five-year planning cycle

Section two: Developmental engagement process

Section three: Summative review process

The three sections provide a full description of IQER, and outline the sequence of steps in the process. The information, key activities and outcomes within each step are set out before proceeding to the next step. Annex D provides a further overview of IQER for the proposed five year cycle after the pilot by setting out all three colour coded sections and all 24 steps.

Section one: IQER pilot study planning cycle. Steps 1 - 8

This section of the Handbook sets out the eight steps that outline the information and activities required to plan the pilot study cycle.

1 QAA briefs all colleges and their awarding bodies

QAA will brief all colleges and their awarding bodies about IQER before the start of the pilot cycle through: a briefing event for pilot colleges and their awarding body(ies), a presentation in each college as part of the cycle planning meeting and through this Handbook and QAA's website (www.qaa.ac.uk). Throughout the IQER process, QAA will copy all correspondence it sends to each college to its awarding body(ies).

2 QAA confirms the number of Developmental engagements for each college

QAA will collect as much information as possible about colleges' HEFCE-funded provision from indirect sources, though some information, including details of HE enrolments, will be collected direct from colleges through a Scope and Preference survey. This information will enable us to establish the Developmental engagement programme, select review teams and notify each college of the number of Developmental engagements it will receive. In addition, QAA will use the information it collects to publish a digest of key statistics about higher education in further education colleges.

3 Each college is given a deadline for indicating its preferences for the nature and timing of its Developmental engagement(s) and the timing of its Summative review

QAA will provide a standard template to assist colleges, following discussions with their awarding bodies, to indicate their preferred scheduling. QAA advises colleges to propose dates for IQER which enable colleges to align IQER with internal review events or to make good use of documents colleges and their partners are preparing for other purposes.

While first Developmental engagements will focus on student assessment, colleges receiving a second Developmental engagement will be invited to suggest a subject or theme for it. It will be helpful if, in doing so, colleges will remember the desirability of appointing an institutional nominee with relevant expertise.

4 Colleges submit their preferences to QAA

Colleges will submit their Developmental engagement preferences to QAA to a standard deadline. Receiving all preferences simultaneously will help us balance colleges' scheduling requests and our operational need to ensure a steady activity flow across the pilot period.

5 QAA collates all responses into a schedule, making any necessary amendments through negotiation with colleges

In what we hope will be the minority of cases, where we are unable to accommodate colleges' preferences, we will contact the college to renegotiate dates suitable to QAA, the college and its awarding body(ies).

6 QAA produces a definitive schedule of all visits and Developmental engagement themes and subjects

When the pilot year schedule of review activity has been finalised, QAA will inform each college of the dates of review visits, submission dates for self-evaluations and provisional arrangements for a return visit to the college within the Summative review, should this be necessary (see step 12). QAA will copy all correspondence to each college to its awarding body(ies).

7 QAA recruits and trains coordinators, reviewers, institutional nominees and facilitators, assigns a dedicated coordinator to each college, and composes teams

Recruitment and training

QAA's policy for the recruitment of all review team members and of other personnel contributing to reviews (collectively described in this Handbook as IQER personnel) is provided at Annex E, and arrangements for recruiting and training such personnel have been developed in the context of this policy.

Role descriptions and person specifications for all IQER personnel are available at Annex C. Both colleges and awarding bodies are encouraged to nominate members of staff with current expertise and experience of management and delivery of HE as coordinators and reviewers. Institutional nominees may also apply to be reviewers or coordinators. Personnel with previous QAA review experience will receive training which builds on review skills from other QAA review methods; those new to QAA will receive more extensive training and support.

In the past taking part in the review process, training, review visits and engaging with peers from other institutions, has been cited as bringing significant benefits both to the individuals concerned and to their institutions. For the IQER process to succeed, it is vital that colleges and HEIs are generous in their nominations. IQER is based on peer review and so, IQER personnel should all have current or very recent experience as managers or coordinators or administrators or teachers of higher education in further education colleges. This is so that reviewers carry the respect of those being reviewed. Accordingly members of Developmental engagement teams will be appointed, as far as possible, on the basis of a close match between their expertise and the subject or theme of the review.

Team composition

Team selection will be made with reference to a college's HEFCE-funded HE provision. QAA will avoid known conflicts of interest, including having worked in or for the college under review during the previous five years; having undertaken validation, external examination or consultancy work for it during the previous three years; having recently applied for a post or having a close relative working or studying there. We will send brief details of proposed teams to colleges not less than four working weeks prior to each Developmental engagement or Summative review, allowing them two weeks to draw to our attention in writing any conflicts of interest they believe to exist.

8 IQER pilot year cycle planning meeting

Preparing for IQER

During the Autumn term 2006 the coordinator, accompanied by a QAA officer, will visit the college to lead a planning session designed to prepare staff and students of the college. If the college and its awarding body(ies) consider it appropriate, representatives of the college's awarding body(ies) may also attend. QAA will provide the coordinator with a standard presentation about IQER, and the meeting will involve discussion of the IQER process, including advice on self-evaluation and students' written submissions. This discussion will include areas which might be addressed under each core question. There will be plenty of scope for questions and discussion.

The specific lines of enquiry for any future Developmental engagement(s) the college may have, will be negotiated according to their developmental value to the college and its awarding body(ies), their ability to address the three core questions and test the college's self-evaluation processes, and, where appropriate, the selected developmental theme or subject.

Demonstrating evidence of self-evaluation processes for IQER

IQER takes as its starting point the normal college and awarding body processes of monitoring and evaluation supporting the management and delivery of standards and quality in HE provision according to how they are apportioned between the college and the awarding body. Colleges will be asked to provide a self-evaluation addressing the three core questions: for Developmental engagement they should do so in the context of the developmental theme or subject, for Summative reviews they should embrace all HEFCE-funded HE provision.

IQER teams will have available to them:

the college's self-evaluation, outlining its approach to managing and delivering the academic standards and quality of its HE provision in accordance with awarding body requirements, and offering a view of the effectiveness of that approach, including areas where scope for development or enhancement exists; this will normally be presented as a portfolio of existing documents
the college's index to the self-evaluation, providing references to existing documents that illustrate the college's approach and evidence to support its effectiveness, including, where relevant, details of its progress in responding to previous IQER action plans or reviewers' conclusions in Academic review reports
any key documents, for example a strategic plan or quality manual, which provide background or reference material to the self-evaluation
validation documents
relevant external examiners' reports
all agreements governing the college's operation of higher education provision
all reports on the awarding bodies, the college or its provision produced by QAA and other relevant bodies, including professional statutory and regulatory bodies, Ofsted and ALI over the previous three years
an optional students' written submission (see next section)
all information placed by the college on its website
samples of assessed student work.

Student participation

IQER offers students a variety of opportunities to explain their involvement in the quality assurance and enhancement of their programmes and to communicate collective views on the quality of their learning experience.

First, students will be invited to participate in discussion between the coordinator and the college about the IQER process. At this meeting they will receive advice on how to optimise the value of their input into IQER as a whole.

Secondly, they will be invited, if they wish, to present a written submission on behalf of the HE student body, summarising in particular their views on the accuracy and completeness of published information, their experience as learners and the extent to which they are able to engage effectively in the management of the quality of HE provision. Guidance on all aspects of the submission will be provided at the cycle planning meeting, although this information can also be found at Annex F of this Handbook and in a separate pamphlet to be published in the near future. While we hope the student body will provide reviewers with a written submission, should students decide against doing so they will have other opportunities to share their experiences with the review team, and the reviewers will do their best to ensure that their views are appropriately represented.

Thirdly, students will be invited to meetings with the review team in both Developmental engagement and Summative reviews, so that they can raise relevant matters of particular interest or concern to them. Comments made by students will not be attributed.

Section two: the Developmental engagement

Section two, which consists of steps nine to 14 of the review method, describes the operational aspects of the Developmental engagement.

9 The college submits the self-evaluation for the Developmental engagement to QAA, and students submit optional students' written submission

As indicated in Step 8, colleges are requested, and the student body invited, to submit, respectively, a self-evaluation and a written submission for their HE provision. In the case of colleges, QAA requires a self-evaluation for each Developmental engagement, however, colleges are encouraged to submit a portfolio of existing material rather than write a bespoke report. The choice rests with each college, however, and the QAA acknowledges that in some cases 'off-the-shelf' documentation may require glossing or updating.

Self-evaluations should demonstrate colleges' capacity for reflection, critical review and evaluation. If QAA officers and the coordinator consider the documentation incomplete or unsuitable for the review, a QAA officer will ask the college to supplement or revise it in a sufficiently timely manner for the timetable to be met. Given the need to leave reviewers with sufficient time to analyse the self-evaluation prior to the visit, it is unlikely, in such a situation, that more than two weeks will be available for this task.

10 The college sends the self-evaluation to the reviewers

On acceptance of the self-evaluation, QAA will ask the college concerned to send a copy direct to each member of the Developmental engagement team. The self-evaluation will be analysed independently by the reviewers against the three core questions and the selected lines of enquiry; the coordinator will produce a summary of these analyses, which will be shared with the college, as a basis for the Developmental engagement preparatory meeting.

11 The coordinator meets staff and students at the college at the Developmental engagement preparatory meeting

The coordinator will agree the agenda for the Developmental engagement at the preparatory meeting with the college. It is likely to include requests for clarification of materials sent and for additional material which the reviewers need; information on the range and timing of any internal quality assurance events as this may provide evidence helpful to the Developmental engagement team; discussion about the timing and content of meetings with staff, students and other stakeholders; clarification of administrative and housekeeping arrangements; and confirmation as to the identity and availability of the Developmental engagement contact.

While it is for the college concerned to identify the personnel attending the meeting, it will be helpful if participants include the college's institutional nominee(s). If the college and its awarding body(ies) consider it appropriate, awarding body representative(s) may attend. The involvement of an awarding body representative will assist in agreeing whether, given the review theme or subject and the partnership agreement, it would be appropriate for the awarding body(ies) to be represented during the event itself.

After the preparatory meeting, the coordinator will send the college, its awarding body(ies), the reviewers and QAA a confirmatory letter and debriefing form setting out the agreed arrangements for the conduct of the visit.

12 Developmental engagement team meets at the college for the review event

The Developmental engagement will normally span two consecutive days. On the first day, after the coordinator has reminded those present of the method and protocols of review and the agreed schedule, the college will normally be invited to make a 10 minute presentation to introduce the provision concerned, describe the context in which it operates and update the review team on any developments since the self-evaluation was prepared. The college will then be asked to confirm the availability of relevant documents, including the sample of student work agreed as relevant to the Developmental engagement theme or subject; the agenda, timing and composition of meetings; and other practical arrangements.

The coordinator will liaise with the college to arrange meetings with staff, students and other stakeholders such as employers and work-based learning providers. Meetings will revolve around relevant aspects of the three core questions, the developmental theme or subject, agreed lines of enquiry and the sources of evidence; they will operate in a manner designed to enable the reviewers to gain the necessary evidence with minimal disruption to the college. This flexibility of method means that two reviews within the same college may follow different patterns. Liaison between the coordinator and the college aims to ensure clarity as to the method employed and the rationale for it.

Testing the self-evaluation and gathering evidence

The reviewers will be collectively responsible for gathering, verifying and sharing evidence as a basis for testing the self-evaluation and, if submitted, the students' written submission. Their duties will also include addressing the nature and extent of the college's engagement with the Academic Infrastructure. They will discuss the evidence, check their understanding and interpretation, refine their views and triangulate different sources of evidence to arrive at collective conclusions. They will keep notes of all meetings with staff and students, and of any emerging examples of good practice and areas for improvement; these will be circulated among the team and collated by the coordinator. Where concerns emerge in the course of the visit, an opportunity will be made available for the college to provide additional evidence to address them.

The main sources of evidence are documentation, including assessed student work, direct observation and meetings, both with members of the college and other stakeholders. Documentary evidence, is important in helping the reviewers to evaluate the college's management of the quality of learning opportunities and the delivery of academic standards. The team may also carry out direct observation of some elements of provision, for example learning resources, and, exceptionally and normally only as part of an identified Developmental engagement theme or subject, teaching. In most Developmental engagements evidence of teaching quality will be obtained from a documentary study of procedures such as the college's observation of higher education teaching, the analysis of student questionnaires and other arrangements for gathering feedback.

At the end of the visit the Developmental engagement team will formulate its conclusions and recommendations and make a short oral report to the college.

13 The Developmental engagement proceeds to conclusion, the report and action plan are finalised and distributed

Developmental engagement reports and action plans

The report production process is designed to ensure that the report is objective, accurate and helpful to the college, and its awarding body(ies). After the coordinator has collated and edited the report sections drafted by the Developmental engagement team, the draft report will be sent to the college and its awarding body(ies), requesting comments on factual errors and consequential misinterpretations. These comments will be considered by the coordinator and incorporated as appropriate into the second draft report.

The report will set out conclusions, recommendations and implications for the college and its awarding body(ies), but will contain no judgements. Recommendations will be at one of three levels in addition to identified elements of good practice for dissemination:

good practice for dissemination refers to matters which reviewers believe are worthy of being shared across the college's HE provision. They need not be innovative or sector-leading, but should be part of a process or practice which, in the context of the college, is contributing to the improvement of the management of quality, academic standards, learning and teaching or a combination of these
desirable recommendations refers to matters which reviewers believe have the potential to enhance quality, build capacity and/or further secure standards
advisable recommendations refers to matters which reviewers believe have the potential to put quality and/or standards at risk and require preventative corrective action
essential recommendations refers to important matters which reviewers believe are currently putting quality and/or standards at risk and which require urgent corrective action.

The college will then be asked, in consultation with its awarding body(ies), to formulate and send to QAA an action plan in respect of any recommendations made. QAA will provide a standard format for this plan, and will incorporate it into the final report. In the interests of capacity building, the report will be provided to the college and its awarding body, HEFCE and QAA, and future QAA review teams will draw on it.

14 The college prepares for any further Developmental engagements and Summative review

After a Developmental engagement, the college and its awarding body(ies) will wish to implement the action plan in accordance with their partnership agreement and their normal quality assurance procedures, monitoring progress and evaluating the effectiveness of their actions, in order to provide evidence of appropriate responses. During this period, liaison will be maintained between the college and a QAA representative, who will also confirm both the progress of the college's action plan and that the college is keeping its awarding body(ies) informed of its discussions.

Section three: Summative review

Section three consists of 10 steps that outline the information and activities required to conduct Summative review, which takes place at each college and involves:

liaison between the facilitator and the coordinator to prepare for the review
submission of a self-evaluation by the college and an optional students' written submission six weeks before the Summative review visit 1
analysis of the self-evaluation and (if submitted) students' written submission by the coordinator and review team
a Summative review preparatory meeting at the college involving the coordinator, college staff and students, and, if desired, by the awarding body representative(s), to plan the Summative review
a two-day Summative review visit to the college by the review team
a review team meeting off-site to draw conclusions and reach provisional judgements
a draft report containing evidence for the provisional judgements and the provisional judgements
the opportunity for colleges and their awarding bodies to comment on the factual accuracy of the report and supply additional evidence
either off-site confirmation of judgements and recommendations by the review team and if no second visit is required, a QAA officer or representative returning to the college to provide an oral report at the end of the process or if a second visit has taken place, an oral report from the reviewers before leaving the college
production of a published report containing an action plan devised by the college and agreed with its awarding body(ies)

The Summative review has been designed to draw upon evidence provided by both internal and external reviews of college HE provision. It will also provide evidence for Ofsted inspection and Learning and Skills Council (LSC) strategic area reviews. The figure below shows the information flow.

Sources of evidence colleges may use for their self-evaluation in addition to internal review and evaluation documents.

Sources of evidence colleges may use for self-evaluation

15 The college submits the self-evaluation for Summative review to QAA, and students submit their optional written submission

The college's self-evaluation and, if submitted, the student written submission form the basis of the review. The self-evaluation for Summative review covers the college's entire HEFCE-funded provision, irrespective of the HEFCE funding route. As with the Developmental engagement, colleges are encouraged to make use of existing material where possible, supplemented as appropriate, rather than preparing a bespoke submission. They may also wish to involve their awarding bodies in identifying information for the self-evaluation, including reports of awarding body periodic reviews, relevant external examiners' reports and the responses made to them. An indicative self-evaluation structure is shown in Annex B. The figure below indicates the flow of information from a range of external scrutiny to Summative review.

Information flows between a range of external scrutiny processes and Summative review

Information flows

QAA will forward the self-evaluation to the coordinator, who will advise on its suitability to support the review. If QAA, after discussion with the coordinator, considers the self-evaluation incomplete or unsuitable, QAA will contact the college to discuss amendments. As the Summative review schedule will already have been agreed, however, and given the need to leave reviewers with sufficient time to analyse the self-evaluation prior to the visit, it is unlikely, in such a situation, that more than two weeks will be available.

16 The college sends the self-evaluation to reviewers. The review team forwards its comments to the coordinator

When the self-evaluation has been agreed as a suitable basis for the review, QAA will ask the college to send copies to members of the review team. The reviewers will analyse the self-evaluation against the three IQER core questions and each reviewer will produce an initial commentary identifying any areas requiring clarification. The coordinator will collate these commentaries and will produce a summary for the Summative review preparatory meeting with the college. This summary will be shared with the college, where possible before the Summative review preparatory meeting.

17 The coordinator visits the college for the Summative review preparatory meeting

The coordinator will meet college staff and students and, usually, awarding body representatives to discuss arrangements. The agenda, which will be agreed in advance, is likely to include requests for clarification and additional materials, the timing and content of meetings, practical arrangements and agreeing a date for a second visit, should this become necessary.

After the meeting the coordinator will provide the college, its awarding bodies, reviewers and QAA with a letter of confirmation and a debriefing form setting out the agreed arrangements for the conduct of the review.

18 The Summative review team meets at the college for the Summative review visit 1

The Summative review visit 1, normally a two-day visit by the whole review team, follows broadly the same pattern as the Developmental engagement (see Step 12) but with the following specific features:

the focus is on the college's entire HEFCE-funded provision
the college nominates a Summative review facilitator to act as the single point of contact between coordinator and college
no institutional nominees are involved, the review team comprising external peer reviewers only
the focus of the event is on the college's approach to the three core questions
the review report, including the college's action plan will be published.

19 The reviewers draft the report and formulate provisional judgements.

Formulating judgements

The Summative review team's judgements may be accompanied by the identification of features of good practice and a number of recommendations. Features of good practice need not be innovative or sector-leading, but should be part of a process or practice which, in the context of the college, is contributing to improvements in the management or delivery of higher education quality, academic standards, learning and teaching, or a combination of these.

Summative review reports will include two judgements, for core questions 1 and 2.

Judgements

The judgements are confidence, limited confidence or no confidence.

Confidence

Where (i) a college is found to be managing effectively its responsibilities for the delivery of the academic standards and the quality of provision, (ii) the prospects for the future continuation of this appear good, and (iii) it has rigorous mechanisms for the management of its HEFCE-funded higher education provision in accordance with awarding bodies' requirements, it will receive a judgement of confidence. Such a judgement will be reached on the basis of evidence that the college has sound structures and procedures for assuring and enhancing quality and the delivery of standards, that it is successful in managing them and that they are applied effectively to each HEFCE-funded higher education programme. While frequently accompanied by recommendations considered advisable and/or desirable (but never essential), the overall judgement should not be seen as being qualified by such recommendations.

In short, a judgement of confidence is an expression of belief in a college's commitment and ability to identify and address any situation which potentially threatens the quality of student learning opportunities, the delivery of the standards of awards or its ability to meet its contractual obligations. This includes considering and addressing in a mature and engaged manner, and through its own procedures and those of its awarding body(ies) as specified contractually, any recommendations contained in the review report.

Limited confidence

Where significant doubts exist about aspects of a college's current or likely future management of quality and/or delivery of the standards of its higher education programmes, the college will receive a judgement of limited confidence. Such a judgement will indicate how widespread the doubts are and which aspects of the delivery of academic standards and/or quality of provision have given particular cause for concern. Such a judgement, which is likely to have implications for the college's awarding body(ies), will be accompanied by one or more recommendations considered essential and, almost certainly, others considered advisable and/or desirable.

No confidence

Where major doubts exist about significant aspects of a college's current or likely future capacity to secure and maintain quality and/or deliver standards, the college will receive a judgement of no confidence. The review report will identify the main areas of concern, discuss the means by which such a situation was able to arise and be sustained, and advise students and other stakeholders of the existence of failing or unsatisfactory academic standards, or quality of provision. It will contain one or more recommendations considered essential and others considered advisable and/or desirable.

In short, such a judgement will reflect serious procedural inadequacies or implementation failures, and be indicative of fundamental weaknesses in a college's capacity to manage its responsibilities for the delivery of academic standards or for providing higher education of an appropriate quality. Inevitably it will have serious implications for awarding bodies, which are likely to wish to take urgent action.

Judgements on these questions may be differentiated, for example, should the reviewers judge a college's management of the quality and/or standards of the programmes leading to the awards of one awarding body to be below the required threshold, a judgement of limited or no confidence would be awarded to that provision alone.

Follow-up action, which may require immediate action, will be determined on a case-by-case basis in the light of discussion between college, its awarding body(ies), HEFCE and QAA.

20 The coordinator sends the draft report to the college for comments on factual accuracy

After the coordinator has collated and edited the sections of the report submitted by members of the review team, the draft report, including provisional judgements, will be sent to the college, requesting the identification of factual errors and consequential misinterpretations.

21 The college responds to the report with additional evidence if appropriate

The college's response will be considered by the coordinator and the reviewers and incorporated as appropriate into the second draft report. Where there is no disagreement about the content of the report, colleges will be asked simply to confirm the accuracy of the report within a specified time span. Where a report and/or provisional judgements are considered inaccurate or incomplete, colleges and awarding bodies will be invited to support their view by drawing attention to evidence, normally documentary, in existence at the time of the Summative review visit 1.

22 The reviewers confirm judgements off-site or visit the college again if they have insufficient evidence to reach a definite conclusion or if there is the possibility of a limited or no confidence judgement

Once they have considered the college and awarding body response, reviewers will normally confirm their judgements off-site and progress the report to production. In this situation a QAA representative will visit the college to give oral feedback to college and awarding body representatives.

Exceptionally, where further evidence is required, or where a judgement of limited or no confidence is under consideration, the coordinator will confirm the provisionally scheduled second visit. Depending on circumstances, this visit may involve meetings with members of the college and awarding body representatives, considering additional evidence, agreeing judgements and report drafting. At the end of the return visit, the review team will make a short oral report to the college.

23 The Summative review proceeds to conclusion; the report and action plan are finalised and published

The report, with conclusions, commendations, recommendations and implications for the college and its awarding bodies will be finalised and published. The college will be asked to respond to any recommendations by formulating an action plan with its awarding body(ies), which will be published as part of the Summative review report itself.

24 The IQER cycle is complete or the college prepares for further Developmental engagement

For colleges not participating in any further Developmental engagement, other than in respect of follow-up actions, the first IQER cycle is complete. Colleges with a further Developmental engagement will continue their preparations.

Evaluation of IQER

QAA is committed to the regular monitoring and evaluation of policies, procedures and processes to ensure their continuing credibility and enhancement, and to doing so in accordance with our legal requirements and in a manner which ensures respondent anonymity. In the case of IQER, potential respondents include all participants in the process, including students.

QAA's evaluation strategy, which originated in 2001, involves the use of both questionnaires and focus groups, the former being designed to facilitate both comparison across respondent groups within one review method and across review methods. Data analysis enables QAA to monitor identified aspects of the review process, develop future policies and practices, provide feedback to reviewers and inform our reports to funding bodies.

QAA's standards

QAA has published a statement of its values and standards in its Strategic plan 2006-2011. This can be found at Annex A.

While QAA makes every effort to assure the quality of its processes and to meet its published values and standards, occasionally a college may feel it has cause to express dissatisfaction with a review. If a college is dissatisfied with the conduct of a review it may wish to make a complaint. Details of the complaints procedure can be found at QAA's website, www.qaa.ac.uk/aboutus/policy/complaints.asp

If a college wishes to challenge a no confidence judgement, it may make a representation to QAA. Details of the representation procedures can be found at QAA's website, www.qaa.ac.uk/aboutus/policy/representations.asp

Annexes

Annex A QAA's purposes, values and standards
Annex B Indicative self-evaluation structure and aide memoire for IQER teams
Annex C IQER role descriptions and person specifications
Annex D The 24 steps of IQER, proposed for the five year programme 2007-08 to 2011-12
Annex E QAA training and development policy for review team members and review facilitators
Annex F Students' written submission for the IQER pilot study

Annex A: QAA's purposes, values and standards

Purposes

To achieve its mission, QAA works in partnership with the providers and funders of higher education, the staff and students in higher education, employers and other stakeholders, to:

safeguard the student and wider public interest in the maintenance of standards of academic awards and the quality of higher education
communicate information on academic standards and quality to inform student choice and employer understanding, and to underpin public policy making
enhance the assurance and management of standards and quality in higher education and promote a wider understanding of the value of well-assured standards and quality
promote a wider understanding of the nature of standards and quality in higher education, including maintenance of common reference points, drawing on UK, other European, and international practice.

Values

QAA's work is underpinned by the following core values:

The importance of higher education

QAA values knowledge, intellectual challenge, imagination, discovery and achievement in higher education; respects the constitutional, intellectual and operational autonomy of higher education providers, and the diversity of institutional mission within the different legislative and educational contexts across the UK; acknowledges the academic calling and the importance of higher education in the personal, professional and economic lives of citizens individually and collectively; values the high international regard in which UK higher education and its awards are held; and recognises the importance of UK engagement in European and other international developments.

The entitlements of learners

Students are entitled to a higher education with academic standards that reflect national expectations and awards that meet published specifications; and to fair and equitable treatment in all aspects of their studies. QAA values the participation of students in the assurance of academic standards and quality.

The significance of the responsibilities of the providers of higher education

The providers of higher education have the primary responsibility for protecting academic standards and quality; QAA aims to work with the providers in meeting that responsibility. Effective assurance of academic standards and quality is a key indicator of professionalism in higher education and a key guarantor of the student and wider public interest. It also provides the foundation for continuous improvement and the enhancement of quality. We take as the starting point for our reviews the belief that providers wish and intend to do a good and professional job in meeting their responsibilities as guardians of their academic standards and quality. QAA is neither a regulator nor an inspectorate, and we value the insights and benefits that peer review brings to our audit and review processes. QAA depends on the valuable contribution made by the many hundreds of colleagues from the higher education sector and professional and subject bodies who work with it. We value their skills and diversity and their contribution to peer processes.

The validity of the public interest in higher education

Higher education in the UK is an important contributor to the well-being, interests and prosperity of the country. The public invests significant resources in higher education, and has a legitimate expectation that the standards of higher education qualifications will be maintained and that the quality of provision - notwithstanding diversity of institutional missions and modes of delivery - will allow learners to achieve the necessary standards.

Standards

Our standards are integrity, professionalism, accountability and openness.

QAA sets itself high standards in all its undertakings.

Integrity: We aim to show impartiality, fairness, independence and honesty in our work. Decisions and judgements - whether positive or critical, welcome or unwelcome - will be evidence-based and transparent, stated clearly, without fear or favour.

Professionalism: We aim to show high professional standards and provide a cost-effective service. We aim to get it right first time and be prompt, courteous and constructive in all our dealings. Where we get something wrong, we will acknowledge this, learn from it and improve our own work. QAA carries out a range of functions in relation to the work of the providers of higher education. In some cases our role involves the making of public judgements; in some, it involves guiding and encouraging; in some, it involves advising and commenting. In all cases, QAA will conduct itself in a way that is fitting and proportionate both to the function and to the role.

Accountability: QAA is accountable to its subscribers - the higher education institutions - as well as to a wide range of other stakeholders, partners and users of our services and the information we provide. We aim to demonstrate that we use our resources to good effect and with probity and conduct our work with integrity and impartiality; and to ensure that stakeholders are able to depend on the information and advice we provide.

Openness: We aim to be open and approachable; to be transparent in our work and methods, to build confidence and trust among stakeholders, and to provide information about QAA's work to the wider public. We aim to communicate in a clear, consistent and accessible way.

Annex B: Indicative self-evaluation structure and aide memoire for IQER teams

Context

This section should provide:

an introduction to the college and its awarding body(ies)
details of partnership agreements with the awarding body(ies)
an outline of recent changes affecting higher education provision in the college
an outline of action taken on receipt of external review or inspection reports
other relevant information.

Core questions

1 Academic standards: Does the college discharge its responsibilities effectively, as set out in its partnership agreement(s), for the management and delivery of the standards of the award(s) it offers on behalf of its awarding bodies?

Areas for consideration

How are responsibilities for managing and delivering higher education standards delegated within the management structure and what reporting arrangements are in place?
What account is taken of the Academic Infrastructure?
How does the college assure itself that it is fulfilling its obligations to ensure that the standards of higher education provision meet the requirements of validating partners and awarding bodies?
What are the college's arrangements for staff development?

To include:

the arrangements with the awarding body(ies) for maintaining appropriate academic standards
the structures and procedures for quality assurance within the college and how these intersect with those of the awarding body(ies)
the ways in which the quality assurance systems are critically appraised to ensure they are fit for purpose and fulfil their intended objectives
the use made of external examiners' reports
the use made of internal (and/or awarding body) reports on the standards and quality of the higher education the college delivers
the use the college makes of the Academic Infrastructure
the use the college makes of management statistics.

2 Quality of learning opportunities: Does the college discharge its responsibilities effectively, as set out in its partnership agreements, for the management and assurance of the quality of the learning opportunities to enable students to achieve the intended learning outcomes?

Areas for consideration

How are responsibilities for managing the quality of learning opportunities for higher education programmes delegated within the management structure and what reporting arrangements are in place?
How does the college assure itself that that it is fulfilling its obligations to its awarding body(ies) to ensure that students received appropriate learning opportunities?
How does the college assure itself that the quality of teaching and learning are being maintained and enhanced?
How does the college assure itself that students are supported effectively?
What are the college's arrangements for staff development?

To include:

the arrangements with the awarding body(ies) for delegated responsibility for quality of learning opportunities for programmes i.e. who is responsible for what?
the approach taken to assuring the sufficiency and adequacy of learning resources including staffing
the approach taken to assuring and enhancing the quality of resources for learning including staffing
the approach taken to appraising, enhancing and rewarding teaching staff
the approach taken to engaging students as partners in the management of quality in the higher education provision
the experience of students of the learning opportunities offered to them, including teaching and learning, support and learning resource.

3 Public information: Can reliance be placed on the accuracy and completeness of the information that the college is responsible for publishing about itself?

Areas for consideration

the accuracy and completeness of public information.

To include:

details of information for which college is responsible for:
- publishing on its website, and
- holding internally
the arrangements set in place to assure the accuracy and completeness of information the college has responsibility for publishing.

Annex C: IQER role descriptions and person specifications

Job title: Coordinator

Role purpose

The Coordinator coordinates and manages the Summative review and all Developmental engagements in each of the colleges to which s/he is assigned.

Key responsibilities include:

leading a programme of reviews for QAA
providing clear briefings to a wide range of college participants on the IQER method and participants' respective responsibilities
discussing and agreeing with the college, the lines of enquiry and review agenda that form the basis of the reviews
discussing and agreeing focused review activities with the college and the reviewers to ensure effective use of time
organising and coordinating review activities to ensure that conclusions, recommendations and judgements are sound and evidence based
liaising effectively with all stakeholders through face-to-face, telephone, email and written communications to ensure the smooth running of each review
providing additional training for reviewers, if necessary
making effective use of QAA's secure electronic folder system throughout the review to ensure that a full evidence base is available to reviewers and QAA staff in a timely manner and is archived promptly
respecting protocols on confidentiality
producing high-quality reports that usefully inform all stakeholders of conclusions, recommendations and judgements, where appropriate.

Person specification: Coordinator Knowledge and understanding to include:

current or recent knowledge and understanding of current issues
affecting higher education in colleges
awareness of current HE teaching methods and curricula
knowledge and understanding of the assurance of standards and quality
awareness of the role of professional statutory and regulatory bodies in programme accreditation
experience of liaison with senior management and a range of staff at other levels.

Skills include an ability to:

manage small teams (with experience either in higher or further education or in other employment)
work within tight timescales and to strict deadlines
chair meetings
communicate effectively in face-to-face interaction
train others in methods of work
produce clear and succinct reports on time
word process
communicate electronically, including emails, attachments and use of web mail
be flexible and to devise sound plans when situations change with little notice.

Role title: Reviewer

Role purpose

The reviewers contribute to the maintenance and enhancement of academic standards and the quality of higher education provision through a peer-review process. They engage in a variety of activities designed to gather and analyse evidence so that they can arrive at considered conclusions, recommendations and judgements. These outcomes help the college being reviewed to prepare an action plan to further enhance higher education provision.

Key responsibilities include:

reading, analysing and preparing written commentaries of the self-evaluation submitted by the college and any other documents sent in advance of a review
adhering to the review schedule agreed between the college and the coordinator
participating in visits to the college in order to gather, share, test and verify evidence
drawing conclusions, making recommendations and judgements on the academic standards achieved and the quality of the learning opportunities provided
recording evidence gathered from a variety of review activities and submitting this to the QAA secure folder in a timely fashion
drafting sections of the report that are referenced to evidence gathered during the review
respecting protocols on confidentiality
contributing to and commenting on the compilation of the report of the review to agreed schedules and deadlines
being available for the whole period of a review for which they have been selected and committing to complete all processes of a review once they have embarked upon it
being available for IQER in up to three colleges in 2006-07.

Person specification: Reviewer

Knowledge and understanding include:

current or recent experience, knowledge and understanding of higher education provision
knowledge and familiarity with the Academic Infrastructure and other external reference points such as professional statutory regulatory bodies
for reviews requiring subject expertise, at least five years' experience of providing higher educational-level teaching and learning delivered in colleges or higher education institutions: in the case of industrially or professionally-based reviewers, familiarity with higher education teaching and learning
understanding of programme entry requirements and the ability to interpret progression statistics including withdrawal, transfer and failure rates and destinations data
familiarity with academic support strategies and the functions of academic tutorials
experience of examining and/or verification (and preferably external examining or external verification).
familiarity with external examiners' reports and internal documents such as internal verification, second-marking and second-reading
knowledge of the quality assurance processes employed by colleges providing higher education
familiarity with the standards of higher education awards in colleges and universities in the UK.

Skills include an ability to:

conduct meetings and interviews with staff
conduct meetings with a range of current and former groups of students
write succinctly and coherently
meet tight timescales and deadlines
work effectively as a member of a team
work courteously and professionally
maintain confidentiality
communicate electronically, including emails, attachments and use of web mail.

Role title: Institutional Nominee

Role purpose

The role provides an opportunity for members of a college's staff to consider the quality and standards of higher education across the colleges and to be part of a team testing the effectiveness of the college's self-evaluation processes. Where colleges are considering revising internal review processes, participation in a Developmental engagement team may contribute ideas of ways in which processes might be revised.

Key responsibilities include:

reading, analysing and preparing written commentaries of the self-evaluation submitted by the college and any other documents sent in advance of a Developmental engagement
adhering to the Developmental engagement schedule agreed between the college and the coordinator
participating in the Developmental engagement at the college in order to gather, share, test and verify evidence
drawing conclusions and making recommendations on the college's management and delivery of the academic standards achieved and the quality of the learning opportunities provided
recording evidence gathered from a variety of review activities and submitting this to the QAA electronic review folder in a timely fashion
drafting sections of the report that are referenced to sound evidence gathered during the Developmental engagement
respecting protocols on confidentiality
contributing to and commenting on the compilation of the report of the review to agreed schedules and deadlines
helping the college to draw up its action plan for implementation of the Developmental engagement team's recommendations
playing a lead role in the implementation of the action plan within the college
being available for the whole period of a Developmental engagement for which they have been selected and committing to complete all processes of a Developmental engagement once they have embarked upon it.

Person specification: Institutional Nominee

Knowledge and understanding include:

experience, knowledge and understanding of higher education provision in general and within their own college
understanding of the college structure, policies and procedures for managing and delivering higher education programmes
familiarity with the Academic Infrastructure and other external reference points relevant to own area of expertise
understanding of programme entry requirements and the ability to interpret progression statistics including withdrawal, transfer and failure rates and destinations data
familiarity with academic support strategies and the functions of academic tutorials
experience of examining and/or verification (and preferably external examining or external verification). Familiarity with external examiners' reports and internal documents such as internal verification, second-marking and second-reading
knowledge of the quality assurance processes employed by colleges providing higher education and familiarity with higher education programmes and the standards of awards in other colleges and universities.

Skills include an ability to:

conduct meetings and interviews with staff
conduct meetings with a range of current and former groups of students
write succinctly and coherently
meet exacting timescales and deadlines
work effectively as a member of a Developmental engagement team
continue to work effectively as part of the college team after the Developmental engagement(s) has (have) been completed
communicate electronically, including emails, attachments and use of web mail
maintain confidentiality
influence colleagues within their college and take forwards the action plan.

Role title: Facilitator

Role purpose

The Facilitator ensures the smooth running of the Summative review by acting as the single point of contact between the college staff and the review team.

Key responsibilities include:

providing effective liaison between the reviewers and the college staff
ensuring that the reviewers obtain accurate, timely and comprehensive information about the educational provision and its college context
helping the reviewers to come to a clear and accurate understanding of the structures, policies, priorities and procedures of the college, and the nature of the provision under scrutiny
ensuring that reviewers are provided with appropriate evidence to allow them to reach conclusions, recommendations and judgements
bringing additional information to the attention of the reviewers and helping to correct factual inaccuracy
observing objectively, and communicating clearly with the reviewers and the subject provider
respecting protocols on confidentiality
establishing effective relationships with the coordinator and the reviewers, as well as with the college staff
participating in the college's preparations for the Summative review
attending all meetings other than those with students and employers, or where judgements are discussed
monitoring the pattern of review activities
maintaining regular telephone and/or email contact with the coordinator to ensure that reviewers are receiving the information or documents that they need, particularly for off-site analysis.

Knowledge and understanding include:

thorough knowledge of the structure, policies, priorities, procedures and practices of their college
extensive knowledge and experience of working in higher education at a senior level
extensive experience of quality assurance
knowledge and understanding of QAA's review methods
an ability to maintain confidentiality.

Skills include an ability to:

request and receive relevant and timely information from college and awarding body(ies) senior staff
provide cogent information
write succinctly and coherently
deal conscientiously with detail
make accurate records of discussions
meet exacting timescales and deadlines
work effectively with a review team
continue to work effectively as part of the college team after the IQER has been completed
communicate electronically, including emails, attachments and use of web mail
influence colleagues within their college and take forward the action plan.

Annex D: The 24 steps of IQER, proposed for the five year programme 2007-08 to 2011-12

IQER five-year college cycle planning

1 QAA briefs all colleges and their awarding bodies about the IQER method.

2 QAA confirms the number of Developmental engagements for each college.

3 Each college is given a deadline for indicating its preferences for the focus and timing of its Developmental engagement(s).

4 Colleges submit their preferred order of Developmental engagement(s) and all Developmental engagement themes (where there is no more than one Developmental engagement) to QAA.

5 QAA collates all colleges' responses into a schedule, making any necessary amendments through negotiation with colleges.

6 QAA produces a definitive schedule of the Developmental engagements and Summative review for all colleges.

(Steps 1 to 6 take place from October 2007 to December 2007)

7 QAA recruits and trains coordinators, and assigns a dedicated coordinator to each college. QAA also recruits and trains reviewers, INs and facilitators, and composes teams.

(Recruitment and training will start Spring 2007)

8 IQER five-year cycle college planning meeting takes place.

(These take place January to July 2008)

Developmental engagement process

9 The college submits the self-evaluation for its Developmental engagement to QAA, and students submit an optional written submission.

10 Once QAA has confirmed its suitability as the basis for the Developmental engagement, the college sends the self-evaluation to the reviewers.

11 The coordinator meets staff and students at the college at the Developmental engagement preparatory meeting.

12 The team meets at the college for the Developmental engagement.

13 The Developmental engagement proceeds to conclusion. The report and action plan are finalised and distributed.

14 The college prepares for any further Developmental engagement(s) and for the Summative review.

(Steps 9 to 14 take place throughout January 2008 to July 2012)

Summative review process

15 The college submits the self-evaluation for Summative review to QAA, and students submit an optional students' written submission.

16 Once QAA has confirmed that the self-evaluation forms a suitable basis for the review, the college sends the self-evaluation to the reviewers. The reviewers forward their comments on the self-evaluation to the coordinator.

17 The coordinator visits the college for the Summative review preparatory meeting.

18 The review team meets at the college for Summative review visit 1.

19 The reviewers draft the report and formulate provisional judgements.

20 The coordinator sends the draft report with the provisional judgements to the college and its awarding body(ies) for comments on factual accuracy.

21 The college responds to the report with additional evidence, if appropriate.

22 Either the reviewers confirm the judgements off-site

Or visit the college again if there is insufficient evidence to come to a definite conclusion or the possibility of a limited confidence or no confidence judgement

23 The review proceeds to its conclusion, the report and action plan are finalised and published.

24 The IQER cycle is complete or the college prepares for any further Developmental engagement of the cycle

(Steps 15 to 24 take place between October 2009 to July 2012)

Annex E: QAA training and development policy for review team members and review facilitators

Introduction

1 This policy applies to reviews in any part of the UK.

2 The QAA recognises that those selected to be review team members and review facilitators are drawn from a pool of highly qualified, experienced and well-respected personnel who already have skills in the core activities of review. In particular, they are selected for their highly developed and practised skills of written and oral communication, conduct of meetings, analysis and synthesis of a wide variety of information, and evaluation leading to sound judgement. Reviewer and facilitator training seeks to build on these skills to assist review team members and facilitators to apply them to a specific review process.

3 The training and development policy will be published.

Policy

4 The training will be designed to enable review team members and facilitators, where appropriate, to:

participate in accessible and relevant training and development which is economical in the use of their time
experience learning methods which take account of individual learning styles
participate in training which takes due account of prevailing legislation
participate fully in training activities that will be relevant to all participants irrespective of gender, age, ethnicity or disability
hone and apply core skills essential for all of the QAA's methods of review through initial training.

What can reviewers and facilitators expect of the QAA?

5 Each review team member and facilitator can expect the QAA to:

provide induction to the work of the QAA, its mission, standards and values
train him/her in specialist skills needed to carry out or facilitate review work; for review team members, this includes effective use of the electronic communications system set up to support reviews
assist him/her to develop sufficient confidence to undertake or facilitate their first review
provide training reference material to use after completion of their training
provide the QAA documents they need to conduct the reviews to which they are assigned
add them to the QAA's mailing list for receipt of relevant new QAA publications and information about the QAA's work
provide them with opportunities to contribute to the evaluation of the methods in which they have reviewed.

6 Assuming successful completion of initial training, the QAA will:

provide review team members with feedback on their performance on their first review and, where appropriate, guidance on their further development
encourage each team member to engage in the further development of his/her role as reviewer
take into account experience of prior QAA review training and experience when training review team members and facilitators to carry out QAA review methods which are new to them.

Benefits for institutions and other organisations subject to review

7 Adherence to this policy should provide the following benefits:

confidence that review team members and facilitators are properly trained to undertake review work professionally, and confidently
consistent application of each review method
consistency in the messages about the review method which the review team members and facilitators take back to their institutions.

Annex F: Students' written submission for the IQER pilot study

Background

1 Students are central to IQER, and are invited to participate in the key stages of the process. First, although this is entirely voluntary, students, through their representative body, are invited to make written submissions to Developmental engagement(s) and Summative review teams thereby making the reviewers aware of matters of particular interest or concern to them.

2 The students' representative body in the college - normally the Students' Union or equivalent¹ - has the opportunity to participate in the IQER cycle planning meeting with the IQER coordinator, the awarding body(ies) and the college during academic year 2006-07. Second, students will be invited to the preparatory meetings between the coordinator and the college to be held before each Developmental engagement and before the Summative review. Third, officers of the representative body and other students will be invited to take part in specified meetings during the teams' visits to the college. Fourth, QAA strongly encourages colleges to consult their students on the contents of their self-evaluations. Fifth there is no prescribed format or length for the written submission (though it is suggested that it should not normally exceed three pages), nor a prescribed list of contents. Within the parameters set out below, students should feel free to provide whatever information they feel is appropriate, providing that it is relevant to the focuses of the IQER, and to organise it as they choose.

3 The written submission must include a statement of how it has been compiled, its authorship and the extent to which its contents have been shared with and endorsed by the higher education student body as a whole. If, for example, the submission has been prepared entirely from the perspective of undergraduate or full-time students, this should be made clear. Under no circumstances, however, should the submission discuss the competence of individual members of staff or be used for the airing of personal grievances.

4 The submission is not an alternative college self-evaluation, nor should it take the form of a commentary on the college self-evaluation. It would be helpful, however, if students were to take account of the broad headings used by the college in constructing its self evaluation, and to consider in particular the areas that are the particular focuses of IQER. These include:

the accuracy, completeness and reliability of the information the college is responsible for publishing about its programmes (this might include the accuracy of publicity materials and the use made of programme specifications)
the information that students receive about the academic performance expected of them, their experience of the challenges posed by their programmes of study, and the ways in which their performance is assessed (this might include matters such as the usefulness of programme handbooks, aspects of assessment arrangements, and the feedback that students receive on their academic performance)
the experience of students as learners (this might include the quality of academic and personal support, and the quality and access to learning facilities)
the opportunity for students to participate in the management of higher education programmes within the college (this might include opportunities for committee representation at college and programme level, and other means of providing feedback to staff).

Style

5 The written submission should:

be balanced and relevant
focus on the college's HEFCE-funded higher education only
be concise and accessible to the IQER team
adopt a perspective which matches the subject or focus of a Developmental engagement and/or a college-wide perspective for the Summative review
be appropriately balanced between analysis and description.

Submission details

6 The written submission should be forwarded to QAA no later than four weeks before a Developmental engagement visit and no later than four weeks before a Summative review visit 1. The coordinator will confirm the date at the cycle planning meeting held in 2006-07.

Confidentiality

7 QAA strongly encourages the student body to share its written submission with the college and its awarding body(ies), and the college to share its self-evaluation with the student body. This openness is desirable because it enables the IQER team to discuss both documents freely with staff and students during the reviews, and to check the accuracy of their contents. The student body may, if it so wishes, request that its written submission be kept confidential to the QAA and its IQER team. The QAA will respect this wish, subject to prevailing legislation, but students are asked to bear in mind that the reviewers' use of a confidential submission will inevitably be restricted by the fact that its contents are unknown to the college's staff.

8 If the contents of the written submission are not to be shared with the college, this must be stated clearly on the front of the document.

¹ In colleges where no representative body exists, QAA will liaise with appropriate students (for example course committee or other representatives) to identify the best means of ensuring student involvement in a particular situation.

ISBN 1 84482 585 X

The handbook for a pilot study of an integrated quality and enhancement review

QAA 126 09/06

Contents

Part one: Background and development

Introduction

The IQER method

Background

The method of review

The IQER method

The IQER cycle in the pilot study

Reference points for IQER

The relationships of the student's learning experience to the Academic Infrastructure

HEFCE-funded consortia and lifelong learning networks

IQER personnel

Abridged method for colleges with fewer than 100 full-time equivalent students funded by HEFCE

Part Two Operating the IQER process

Section one: IQER pilot study planning cycle. Steps 1 - 8

1 QAA briefs all colleges and their awarding bodies

2 QAA confirms the number of Developmental engagements for each college

3 Each college is given a deadline for indicating its preferences for the nature and timing of its Developmental engagement(s) and the timing of its Summative review

4 Colleges submit their preferences to QAA

5 QAA collates all responses into a schedule, making any necessary amendments through negotiation with colleges

6 QAA produces a definitive schedule of all visits and Developmental engagement themes and subjects

7 QAA recruits and trains coordinators, reviewers, institutional nominees and facilitators, assigns a dedicated coordinator to each college, and composes teams

Recruitment and training

Team composition

8 IQER pilot year cycle planning meeting

Preparing for IQER

Demonstrating evidence of self-evaluation processes for IQER

Student participation

Section two: the Developmental engagement

9 The college submits the self-evaluation for the Developmental engagement to QAA, and students submit optional students' written submission

10 The college sends the self-evaluation to the reviewers

11 The coordinator meets staff and students at the college at the Developmental engagement preparatory meeting

12 Developmental engagement team meets at the college for the review event

Testing the self-evaluation and gathering evidence

13 The Developmental engagement proceeds to conclusion, the report and action plan are finalised and distributed

Developmental engagement reports and action plans

14 The college prepares for any further Developmental engagements and Summative review

Section three: Summative review

Sources of evidence colleges may use for their self-evaluation in addition to internal review and evaluation documents.

15 The college submits the self-evaluation for Summative review to QAA, and students submit their optional written submission

Information flows between a range of external scrutiny processes and Summative review

16 The college sends the self-evaluation to reviewers. The review team forwards its comments to the coordinator

17 The coordinator visits the college for the Summative review preparatory meeting

18 The Summative review team meets at the college for the Summative review visit 1

19 The reviewers draft the report and formulate provisional judgements.

Formulating judgements

Judgements

Confidence

Limited confidence

No confidence

20 The coordinator sends the draft report to the college for comments on factual accuracy

21 The college responds to the report with additional evidence if appropriate

22 The reviewers confirm judgements off-site or visit the college again if they have insufficient evidence to reach a definite conclusion or if there is the possibility of a limited or no confidence judgement

23 The Summative review proceeds to conclusion; the report and action plan are finalised and published

24 The IQER cycle is complete or the college prepares for further Developmental engagement

Evaluation of IQER

QAA's standards

Annexes

Annex A: QAA's purposes, values and standards

Purposes

Values

The importance of higher education

The entitlements of learners

The significance of the responsibilities of the providers of higher education

The validity of the public interest in higher education

Standards

Annex B: Indicative self-evaluation structure and aide memoire for IQER teams

Context

Core questions

Areas for consideration

Areas for consideration

Areas for consideration

Annex C: IQER role descriptions and person specifications

Job title: Coordinator

Role purpose

Role title: Reviewer

Role purpose

Person specification: Reviewer