Handbook for institutional audit: England and Northern Ireland

2006

Introduction

1 The Quality Assurance Agency for Higher Education's (QAA) mission is to safeguard the public interest in sound standards of higher education (HE) qualifications and to inform and encourage continuous improvement in the management of the quality of HE. To this end, QAA carries out institutional audits of higher education institutions (HEIs).

2 Institutional audit is an evidence-based process carried out through peer review. It forms part of the Quality Assurance Framework (QAF) established in 2002 following revisions to the UK's approach to external quality assurance. At the centre of the process is an emphasis on students and their learning.

3 Institutional audit balances the need for publicly credible, independent and rigorous scrutiny of institutions with the recognition that the institutions themselves are best placed to provide stakeholders with valid, reliable and up-to-date information about the academic standards of their awards and the quality of their educational provision. Institutional audit encourages institutions to be self-evaluative, and is therefore a process that, in itself, offers opportunities for enhancement of institutional management of standards and quality.

4 This Handbook describes the revised process of institutional audit that will operate in England and Northern Ireland for the cycle that extends from 2005-06 to 2010-11.

5 The revised process of institutional audit has been developed by QAA in partnership with the Higher Education Funding Council for England (HEFCE), the Standing Conference of Principals and Universities UK. It has been developed following wide consultation in the sector of the draft Operational description of the process.

Background

6 The revised institutional audit process builds upon which that operated from 2002-03 to 2005-06, known as the 'transitional arrangements' described in the Handbook for institutional audit: England, published by QAA in 2002. The separate arrangements for auditing institutions in Scotland and Wales are described in the Handbook for enhancement-led institutional review: Scotland and the Handbook for institutional review: Wales. Institutions in England and Northern Ireland that subscribe to QAA will take part in the revised process, which will operate over six years.

The aims and objectives of institutional audit

7 The aims of this revised institutional audit process are to meet the public interest in knowing that universities and colleges in England and Northern Ireland have:

effective means of ensuring that the awards and qualifications in HE are of an academic standard at least consistent with those referred to in The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ) and are, where relevant, exercising their powers as degree-awarding bodies in a proper manner
effective means of providing learning opportunities of a quality that enables students, whether on taught or research programmes, to achieve those HE awards and qualifications
effective means of enhancing the quality of their educational provision, particularly by building on information gained through monitoring, internal and external reviews, and feedback from stakeholders.

8 The objectives of the process are to:

ensure that the academic standards of UK HE awards and qualifications are maintained and securely managed
enable students and other stakeholders to have confidence in the proper management of the quality of learning opportunities offered through the programmes of study that lead to those awards
check that effective arrangements are in place to maintain appropriate academic standards and enhance the quality of postgraduate research programmes
contribute, in conjunction with other mechanisms and agencies in HE, to the promotion and enhancement of quality in teaching, learning and assessment
ensure that students, employers and others can have ready access to easily understood, reliable and meaningful public information about the extent to which the HEIs in England and Northern Ireland are individually offering programmes of study, awards and qualifications that meet national expectations in respect of academic standards and quality of provision
ensure that, if the management of academic standards or of the quality of provision is found to be weak or seriously deficient, the process forms a basis for ensuring rapid action to improve it
provide a means of securing accountability for the use of public funds received by HEIs.

The revised institutional audit process in summary

9 Institutional audits examine:

the effectiveness of an institution's internal quality assurance structures and mechanisms, in the light of the UK Academic Infrastructure and the European standards and guidelines for quality assurance in higher education (ESG) (see below, paragraph 29), and the way in which the quality of its educational provision and academic standards of its awards are regularly reviewed and resulting recommendations implemented. This provides public information on an institution's soundness as a provider of HE qualifications of national and international standing
the effectiveness of arrangements for maintaining appropriate academic standards and enhancing the quality of postgraduate research programmes
the effectiveness of an institution's approach to building systematically upon the outcomes of their internal quality assurance procedures, on the findings of reports of external reviews, and on other information and feedback from students, graduates and employers, in order to develop and implement institutional approaches to enhancing the quality of provision
the accuracy and completeness of the information that an institution publishes about the academic standards of its awards and the quality of its educational provision, including the published teaching quality information (TQI).

Collaborative provision

10 Where practicable, the institutional audit process covers provision offered by an institution in collaboration with other providers, both in the UK and overseas. However, where QAA decides that an institution's collaborative provision cannot properly be addressed as part of the institutional audit, a separate audit of the institution's management of its collaborative provision will be conducted at a time to be arranged between QAA and the institution. The decision on whether or not there will need to be a separate collaborative provision audit will be made around the time of the preliminary meeting with the institution (see below, paragraph 66). The decision will be made on the basis of the situation at the time scheduled for the institutional audit, irrespective of whether or not a separate collaborative provision audit was made under the transitional arrangements. QAA expects to continue with its existing programme of audits of specific partnership links between UK institutions and providers overseas.

Focuses

11 Institutional audit covers an institution's management of the security of the academic standards of its awards and of the quality of the learning opportunities it provides to enable students to achieve those standards. Audit teams will focus their exploration on the areas listed below.

Institutional management of academic standards
This will cover the use made of external examiners, internal and external reviews, assessment policies, the Academic Infrastructure and other reference points, management information, and other relevant topics.

Institutional management of learning opportunities
This will cover the use made of external examiners, internal and external review, students as partners in quality management, research activity to inform learning opportunities, other modes of study (such as workplace and flexible and distributed learning), the Academic Infrastructure and other reference points, management information, learning resources, admissions policies, student support, staff appraisal and support, and other relevant topics.

Institutional approach to quality enhancement
This will cover the use made of external examiners, internal and external review, the Academic Infrastructure and other reference points, students as partners in quality enhancement, management information, dissemination of good practice, staff development and reward, and other relevant topics.

Collaborative arrangements
This will cover the use made of external examiners, internal and external review of collaborative arrangements, the Academic Infrastructure and other reference points, the use made of management information including feedback, and other relevant topics.

Institutional arrangements for postgraduate research students
This will cover the use made of external examiners, internal and external review of research provision, research students as partners in quality management, the Academic Infrastructure and other reference points, management information including feedback, and other relevant topics.

Published information
This will cover the approach to ensuring the accuracy and completeness of published information, students' experience of published information and other information relating to their programmes of study, and other relevant topics, and there will be a sample of the accuracy and completeness of TQI.

12 In all of these areas, an audit team will be interested in the ways in which quality management systems and mechanisms are critically appraised by the institution to ensure that they are no greater than is necessary, fit for purpose and achieve their intended objectives economically and without 'gold-plating' (ie, bureaucratic features that may look impressive but add little value).

13 Audit teams will be looking for evidence of a careful, serious and professional engagement with these and other relevant topics on the part of the institution, with the purpose of ensuring that the academic standards of awards and the quality of provision are being managed in a manner that can engender and maintain public confidence. Teams will also wish to assure themselves that, in the approach to the provision of public information on quality and standards, institutions' policies and practices are characterised by clarity and frankness.

Information

14 An audit team will have available to it a variety of information sources, including:

a briefing paper by the institution outlining its approach to managing the security of the academic standards of its awards and the quality of its educational provision, and offering a view of the effectiveness of that approach. An index to the briefing paper will list reference to existing documents cited by the institution to illustrate its approach and provide evidence to support its view of the effectiveness of that approach
any key documents (such as a Strategic Plan, Quality Manual or equivalent) that the institution wishes to submit with its briefing paper as background or reference material
reports that can be accessed directly by QAA about the institution or its provision, such as those produced by QAA and other relevant bodies, including professional, statutory and regulatory bodies (PSRBs), within the six years preceding the audit
the information placed by the institution on the TQI site, and the additional TQI information held by the institution, as described in Annex H of HEFCE document 03/51, Information on quality and standards in higher education: final guidance
a summary, which will be shared with the institution concerned, prepared by QAA's Information Unit of the institution's TQI set, with a commentary on the completeness and currency of this information drawn from comparisons with other publicly available information.

15 It is hoped that the students of the institution will also wish to provide a briefing paper (see paragraph 62), in which case an audit team will also have a written submission prepared by representatives of students of the institution on behalf of the student body.

Judgements and comments

16 In the cycle of institutional audit that will extend from 2005-06 to 2010-11, audit teams will make judgements on:

the confidence that can reasonably be placed in the soundness of the institution's present and likely future management of the academic standards of its awards, and
the confidence that can reasonably be placed in the soundness of the institution's present and likely future management of the quality of the learning opportunities available to students.

17 An audit team's judgement is not about academic standards as such, but about the way that the institution ensures that its academic standards are secured by the work of its examiners, internal and external, judged against the reference points of the Academic Infrastructure. Nor is the team's judgement about the quality of learning opportunities as such, but rather about the way that the institution ensures that the learning opportunities available to students are of an appropriate quality, with reference to the guidance in the Academic Infrastructure. In this context, 'learning opportunities' is taken to mean the combined effect of the programmes of study and academic and personal support for students.

18 Where an audit team finds that an institution is managing the security of academic standards soundly and effectively, and where the prospects for the future continuation of this appear good, it would be expected to express its 'confidence'. Similarly, where an audit team finds that an institution is managing the quality of the student learning opportunities soundly and effectively, and where the prospects for this future continuation appear good, it would also be expected to express its 'confidence'. In each case a judgement of confidence indicates that, in the view of QAA, the institution is managing them effectively and consistently.

19 There are two areas where audit teams will find it particularly difficult to express confidence if certain elements are found to be missing. The first of these is a strong and scrupulous use of independent external examiners in summative assessment procedures. The second is a similar use of independent external participants in internal quality management procedures, with particular reference to guidance provided by the Code of practice for the assurance of academic quality and standards in higher education (Code of practice), Section 7: Programme design, approval, monitoring and review. In both cases, the emphasis is on both independence and externality being satisfied.

20 Where an audit team has substantial doubts about the current management of security of academic standards and/or of the quality of provision, and/or about the future management of either of these, it will make a judgement of 'limited confidence' and will indicate clearly the areas of concern that have given rise to such limitation of confidence. A judgement of 'limited confidence' indicates that there is evidence that the institution's capacity to manage soundly and effectively the academic standards of its awards and/or the quality of its educational provision is currently limited or is likely to become limited in future. It should be emphasised that a judgement of 'limited confidence' is not a judgement of failure. It indicates an outcome that is positive but that improvements need to be made.

21 Where an audit team has serious concerns about the current management of the security of academic standards and/or of the quality of provision, and/or about the future management of either of these, it will make a judgement of 'no confidence' and will indicate clearly the significant areas of concern that have given rise to this judgement. A judgement of 'no confidence' indicates that there is substantial evidence of serious and fundamental weaknesses in the institution's capacity to secure the academic standards of its awards and/or to maintain an appropriate quality of educational provision. Cases of failing or unsatisfactory academic standards of provision have been shown to be very rare in UK HE but, where they do occur, students and other stakeholders have a right to know.

22 The criteria for the judgements of 'confidence' are set out in Annex E. The judgements, and the criteria that determine them, are consistent with the judgements and criteria that applied in the transitional arrangements, with the judgement of 'confidence' equating to the judgement of 'broad confidence' that applied in the transitional arrangements. Consistency of judgements and their criteria is essential to support continuity of the institutional audit process and its outcomes.

23 Audit teams will also comment specifically on:

the institution's arrangements for maintaining appropriate academic standards and quality of provision of postgraduate research programmes
the institution's approach to developing and implementing institutional strategies for enhancing the quality of its educational provision, both taught and by research
the reliance that can reasonably be placed on the accuracy and completeness of the information that the institution publishes about the quality of its educational provision and the standards of its awards.

24 If the audit includes the institution's collaborative provision (see above, paragraph 10), the judgements and comments will apply to collaborative provision, and this will be made clear in the report, unless the audit team considers that any of its judgements or comments in respect of the collaborative provision differ from those in respect of the institution's 'home' provision. Any such differences will be reflected in the form of words used to express a judgement or comment. Where there is to be a separate audit of collaborative provision, it will be made clear that the judgements and comments of the institutional audit apply only to the institution's provision on its own campus(es).

25 Institutional audit reports will include recommendations for further consideration by the institution, and will identify features of good practice that the audit team considers to make a particularly positive contribution to the institution's approach to the management of the security of academic standards and of the quality of provision in the context of that institution.

26 Recommendations will be categorised in order of priority:

'essential' recommendations refer to important matters that the audit team believes are currently putting quality and/or standards at risk and which require urgent corrective action
'advisable' recommendations refer to matters that the audit team believes have the potential to put quality and/or standards at risk and require preventive or corrective action
'desirable' recommendations refer to matters that the audit team believes have the potential to improve the quality of learning opportunities and/or further secure the academic standards of awards.

27 The categorisation of recommendations and the criteria for features of good practice identified in a report are consistent with those applied in the transitional arrangements.

Particular features of the revised institutional audit process

Use of reference points

28 Audit teams will draw upon the Academic Infrastructure as a source of external reference points when considering an institution's approach to the management of the security of academic standards and the quality of its provision. They will not do so in a mechanistic way, or look for unthinking compliance with the detail of these structural underpinnings to UK HE. Teams will be looking for evidence that institutions have carefully considered the purpose and intentions of the elements of the Academic Infrastructure, have reflected on their impact on institutional practice, and have taken, or are taking, any necessary measures to best reflect in institutional practice the guidance provided by the Academic Infrastructure.

29 Audit teams will, in practice, use the UK's Academic Infrastructure as their prime point of reference, but they will be interested to know how institutions have considered the expectations of the ESG and other guidance relating to European or international practices. Teams will be asked to comment on the way that institutions are responding to the ESG and other developments relating to academic standards in the European Higher Education Area (EHEA), particularly in the context of the Bologna Process.

30 So far as the FHEQ is concerned, audit teams will wish to look at how institutions check the alignment between the academic standards of their awards and the levels referred to in the FHEQ. Teams will be able, for example, to see how this is effected in practice through the sampling audit trails (see below, paragraph 35).

31 Audit teams will explore how institutions are engaging with the Code of practice. A team will expect to see, in the institution's briefing paper, an explanation of how the institution has gone about addressing the spirit of the precepts of the Code of practice overall (but see below, paragraph 44), including references to illustrate how any resulting changes in its practices have followed, and will discuss any areas of difficulty that the institution has experienced.

32 Audit teams will also enquire into the way in which any relevant subject benchmark statements have been taken into account when establishing or reviewing programmes and awards, bearing in mind that QAA does not view subject benchmark statements as constituting definitive regulatory criteria for individual programmes or awards. They remain statements of what the relevant academic communities consider to be valid frames of reference within which a degree, normally at honours level, should be offered. They need to be used with particular care in interdisciplinary or multidisciplinary contexts, where simple or general application may be inappropriate. They do, however, provide authoritative reference points, which students and other interested parties will expect to be taken into account when programmes are designed and reviewed.

33 Programme specifications are the definitive publicly available information on the aims, intended learning outcomes and expected learner achievements of programmes of study, and audit teams will wish to explore their usefulness to students and staff, and the accuracy of the information contained in them. In particular, teams will be interested to see how programme specifications make use of other reference points in the Academic Infrastructure in order clearly to define expectations for the teaching, learning and assessment engagement between the institution and its students. It is expected that the sampling trails will provide a window through which teams can see how institutions make use of subject benchmark statements within programme specifications, and the way in which programme specifications, or elements of them, are communicated to students.

Audit trails

34 Audit teams use 'trailing' as a technique for gathering evidence in an organised way, from documents and meetings, about institutional processes and procedures, and their impact. Trails are used to examine particular institutional approaches, policies or procedures in order to test their effectiveness. Some are used specifically to sample the combined effect of an institution's policies and procedures in practice. Others may be used to help a team understand particular topics that are a feature of the institution or its mission.

Sampling trails

35 Trails that sample the combined effect of policies and procedures help the audit team understand how the institution's approach to the management of quality and standards works in practice to the benefit of students and staff. Trails of this type will be called 'sampling' trails. They follow the implementation and outcomes of institutional policies and procedures from institutional to programme level, and back; they may be likened to a sample being taken vertically through the institution's management levels. By sampling across levels, the audit team can gain a clearer view of the effectiveness of institutional management at the various levels, and be in a better position to see where policies and procedures work together economically and effectively and where, perhaps, they are more than is necessary for the job in hand - the so-called 'gold plated' approach.

36 Institutions do, of course, conduct trails of this type this for themselves, in the form of internal periodic reviews (the Code of practice, Section 7: Programme approval, monitoring and review). Sampling trails will therefore focus on an institution's internal review procedures and their outcomes, using whatever unit of internal review the institution uses, be that faculty, school, department, subject or programme, or any other grouping that is the institution's preferred unit of internal review.

37 Two trails of this type will normally be sufficient to provide a representative sampling of an institution's internal review procedures and their outcomes. Audit teams will be advised to discuss with an institution the selection of (normally) two sampling trails, with one fewer in the case of institutions with a small number of units of review, and possibly one more in the case of institutions with a large number of units of review, or where the institution feels that it cannot present a representative illustration of the effectiveness of its internal procedures through only two examples. An additional sampling trail might also be selected in the case of prior evidence, from an authoritative source, of risk to standards of awards and/or quality of provision.

38 By basing sampling trails on the institution's own periodic review procedures and its own units of review:

the only documentation required will be that which is already held, in hard copy or electronically, by the institution from internal periodic review and its follow up, and relevant external scrutiny and its follow up with the exception of a brief covering paper to help the audit team follow the trail through these documents
the institution will be able to discuss with the audit team the selection of sampling trails from the point of view of those which it considers to be representative of the scope of its units of review, the currency of review processes and illustrative of their effectiveness (although the final selection will remain with the team)
the audit team will be able to consider how policies and procedures are carried out in practice, including the way that the institution makes practical use of the outcomes of internal review to support enhancement of the learning opportunities available to students
the audit team will be able to compare publicly available summaries of external examiners' reports, the outcomes of internal periodic review and National Student Survey (NSS) data with the complete information from which those summaries have been derived; this will help the team meet the requirement that audit should examine the accuracy and completeness of published TQI
the trailing of institutional policies to see how they benefit students and staff will offer opportunities for the audit team to meet students and staff in groups that would be most coherent for the unit of internal review, thus preserving a focus on the views of 'ordinary' staff and students.

Supplementary trails

39 There may be instances where an institution has particular features of its mission or its approach to the management of quality and standards that it would wish the team to explore through the technique of trailing. For example, an institution that has a requirement for workplace-based learning across its provision may suggest that an audit trail would be a way of giving due weight to the strengths and limitations of its implementation of that requirement. Trails of this type would be additional to other trails, and will be called 'supplementary trails'. It will be for an institution to propose a supplementary trail, and for the audit team to accept or reject that proposal in discussion with the institution. Where a supplementary trail is agreed, the documentation available to the audit team would be that which already exists, in hard copy or electronically, except for brief guidance or context setting by the institution. The team might wish to support its reading of documents by meeting appropriate groups of staff and/or students.

40 There is no expectation by QAA that an institution will necessarily wish to propose a supplementary trail over and above those that are decided by the audit team, but supplementary trails are available to add to the flexibility of the audit trail methodology if required.

Overview of audit trails

41 In summary:

there will normally be two sampling audit trails, making use of existing documentation of internal periodic review procedures and their outcomes, with a brief cover note to guide the audit team through the documentation
in the case of specialist institutions with a small number of units of internal review, an audit team might consider that a single sampling trail is sufficient to provide a suitable sample size
in the case of institutions with a large number of units of internal review, or where the institution wants additional representation of its review processes, an audit team might consider that an additional sampling trail is necessary to provide a representative sample size
where there is prior evidence of risk, from an authoritative source, an audit team might consider that a further additional sampling trail is necessary to provide a representative sample of an institution's periodic review processes and their outcomes
there may be one or more supplementary trails of particular topics if these are proposed by the institution and accepted by the audit team. There is no expectation by QAA that an audit will necessarily include a supplementary trail.

42 The final selection of trails will be made by the audit team at the briefing visit following discussion with the institution. The selection will take into account the need to offer flexibility within a context of consistency. It will be made bearing in mind the value added to the audit process by undertaking the trail, and the need to balance the gathering of operational evidence against the demands placed on the institution, its staff and its students.

43 Audit trails, of whatever type, are a method of obtaining evidence and not an end in themselves. Consequently, the findings of audit trails will not normally be reported separately but will be incorporated in the audit report and its annex, as appropriate, as contributions to the evidence on which the audit team draws for its analysis, findings, judgements and comments.

Postgraduate research programmes

44 The revised institutional audit method requires audit teams to assess and report upon the extent to which institutional arrangements for securing the academic standards of awards and the quality of provision in postgraduate research degree programmes are in alignment with guidance given in the Code of practice, Section 1: Postgraduate research programmes. The introduction to this explains that the section 'is written in a firmer style than some other sections, especially the precepts, to give institutions clear guidance on the funding councils', research councils' and the Agency's [QAA's] expectations in respect of the management, quality and academic standards of research programmes'.

45 The 2005-06 special review of research degree programmes that was commissioned by the funding bodies of England, Wales and Northern Ireland made specific reference to the precepts of the Code of practice, Section 1: Postgraduate research programmes. An audit team will have access to the report on the outcomes of that special review for the institution, and the institution may wish to make reference to that report, updating the team on developments since the report was produced. An audit team's formal comment on postgraduate research programmes will form the conclusion of a section of the institutional audit report, and will contribute to the overall confidence judgements.

Enhancement

46 In the revised institutional audit method, quality enhancement is defined as the process of taking deliberate steps at institutional level to improve the quality of learning opportunities.

47 In this definition of quality enhancement, the emphasis is on how an institution seizes developmental opportunities in a manner no less systematic and no less based on clear strategic planning than quality assurance - 'taking deliberate steps'. Quality enhancement is therefore seen as an aspect of institutional quality management that is designed to secure, in the context of the constraints within which individual institutions operate, steady, reliable and demonstrable improvements in the quality of learning opportunities.

48 Audit teams will consider the ways in which institutional-level approaches to quality enhancement make systematic use of management information. Such information may come from external examiners or advisers; from external bodies such as PSRBs and the Higher Education Academy; from students, graduates and employers, from the outcomes of internal review procedures; and from internal policies, such as may be part of the institution's learning and teaching strategy. Quality enhancement as defined here has, therefore, much to do with the way in which institutions collect, analyse and use information from internal and external sources. Teams will use the sampling audit trails to find out how institutions make use of management information to support quality enhancement at programme level.

49 Enhancement of learning opportunities also takes place by staff independently generating enhancement initiatives, but such routes to enhancement are associated with good people and their good ideas rather than necessarily with good institutional approaches to quality enhancement. Such routes may be better served by engagement with organisations such as the Higher Education Academy which are able to address matters of enhancement at several levels across an institution. Therefore, while recognising that the implementation of institutional approaches to quality enhancement will be demonstrated through individual examples of good practice in teaching, learning and assessment, audit teams will be less interested in particular examples of good practice, as such, than in the way that such good practice is brought about, recognised, supported and maintained by an ethos which expects and encourages the enhancement of learning opportunities.

50 Audit teams are required to comment specifically on the institution's approach to enhancing the quality of its educational provision, both taught and by research. They are, therefore, interested to learn of the extent to which an institution has moved, and is moving, towards developing an institution-wide policy for systematically enhancing quality of provision. It is reasonable for institutions to submit, as examples of quality enhancement as a management process, planned approaches which, while they have yet to come to fruition, can nevertheless be demonstrated to have the potential to do so.

How the process works

Auditors and audit teams

51 Roles. The basic institutional audit team will normally comprise four auditors and an audit secretary. The two roles will be clearly defined, but are of equal status. In the case of institutions with an extensive or complex provision, a team may need to include additional auditors in order to ensure that sufficient coverage of the institution's portfolio of activity can be obtained to justify the judgements and comments being made. Audit secretaries and auditors will not be appointed to teams auditing their own institution.

52 QAA Assistant Director. Each audit is coordinated by a QAA Assistant Director (AD). In the period preceding the audit visit, the AD provides advice to the institution on its preparations for the audit, and works with the audit team on the initial analysis of documentation. He or she accompanies the team during the briefing visit and for the final part of the audit visit, providing advice as appropriate. It is the responsibility of the AD to test that the team's findings are supported by adequate and identifiable evidence, and that the audit report provides information in a succinct and readily accessible form, supported by reference to more detailed evidence and analysis in the annex to the report.

53 Selection. At the time of appointment, auditors will be expected to have current expertise and experience of teaching and management in HE. They will be selected by QAA from nominations made by UK institutions, and it is assumed that there will be a general willingness to offer names. QAA also assumes that institutions will nominate persons with sufficient seniority, knowledge and technical ability to ensure that audits are carried out in a competent, professional and credible way. Selection criteria for auditors will be published and every attempt will be made to ensure that the auditor cohort reflects appropriate sectoral, discipline, geographical, gender and ethnic balances.

54 Audit secretaries will be normally recruited from among senior administrative staff in institutions. They will provide administrative support and fulfil the primary coordination and liaison function during the visits to the institution.

55 Further information about recruitments and selection procedures can be found at QAA's website (www.qaa.ac.uk/aboutus/appointments/).

56 Training. Training for auditors and audit secretaries can be undertaken by QAA either directly or in collaboration with appropriate training providers. The purpose of the training will be to ensure that all team members fully understand the aims and objectives of the audit process; that they are acquainted with all the procedures involved; that they understand their own roles and tasks, QAA's expectations of them and the rules of conduct governing the process; and that they have an opportunity to explore and practise the techniques of data assimilation and analysis, the development of programmes for visits, the construction and testing of hypotheses, the conduct of meetings, the forming of judgements and statements of confidence, and the preparation of reports. The effectiveness of the training, and of the initial selection process, will continue to be the subject of formal evaluation. Existing auditors and audit secretaries who are selected to remain on the registers for institutional audits will be given training in the new process, building on their previous training and experience of earlier modes of institutional audit.

Briefing papers

Institutional briefing paper

57 Institutional audit, like its predecessor processes, is based on the expectation that a reflective institution will appraise the effectiveness of its management of standards and quality using questions of the type:

what are we trying to do?
why are we doing it?
how are we doing it?
why is that the best way to do it?
how do we know it works?
how can we improve it?

A briefing paper is an opportunity for an institution to outline the way that it goes about answering those six questions, with the first three questions focusing more on a description of its approach and the later questions taking a more reflective view.

58 The briefing paper requested for the revised institutional audit process should outline the approach taken by the institution to the management of the security of the academic standards of its awards and of the quality of its educational provision. It should inform the audit team of the way that the institution has reflected upon the effectiveness of its management processes and has acted to improve them. From the point of view of an audit team, it is more important to have a document that gives a clear picture of the institution's approach and the measures taken to reflect constructively on that approach than to have a document that is self-evaluative only in the sense of identifying strengths and weaknesses.

59 The briefing paper is an opportunity for an institution to offer a view of the effectiveness of its approach and provide references to evidence to support that view. It is important that the briefing paper has a comprehensive index giving references to the evidence that the institution wishes to cite.

60 A template for the institutional briefing paper is given in Annex D. It is aligned with the template for the report (Annex B). It addresses the main headings of the report, encourages reflective analysis of effectiveness and of any measures to improve effectiveness of the processes described under these headings, and asks for reference to documents that will illustrate for the audit team in more detail the matters outlined in these headings. QAA does not wish to specify a maximum length for the briefing paper, but would suggest that an institution might question whether a document of more than some 30 pages is likely to serve as an effective briefing document for the audit team.

61 The briefing paper should be made available both in hard copy and in electronic format. It should be a stand-alone document, capable of being understood without access to the references, although it is appreciated that an in-depth understanding of the institution's presentation will only be achieved when the reader is able to access the references. Recognising that institutions may wish to give references to documents in electronic format, it is acceptable to make references to them in the briefing paper without providing them in hard copy as long as the audit team can access them through the web or, for example, on a CD-ROM that accompanies the hard copy version of the briefing paper. Discussion of alternative ways to present the briefing paper and its references will be part of the preliminary discussion (see paragraph 66).

Students' written submission

62 Students are invited to prepare a written submission to brief the audit team. The great majority of institutional audits under the transitional arrangements received a written submission. It was clear that students welcomed the opportunity to provide their considered view to the audit team in writing, and the audit process derived great benefit from the constructive and thoughtful written submissions made by students.

63 Four particular questions proved to be useful stimuli for eliciting students' views in a way that is particularly helpful to an audit team.

How accurate is the information that the institution publishes?
Do students know what is expected of them?
What is the student experience as a learner like?
Do students have a voice in the institution, and is it listened to?

64 These were captured in QAA's booklet, Institutional audit: a guide for student representatives. They will be retained for the revised institutional audit process, and the booklet will be updated to reflect the revisions to the process. They form the basis of a recommended template for a written submission by students.

65 The written submission by students will remain voluntary. QAA would, nevertheless, give every encouragement to students to present audit teams with a written briefing. The production of a written submission takes time and effort that students might be reluctant or unable to give. QAA ADs noted in the transitional period that some Students' Union officers were apprehensive at the prospect of generating what they assumed would be major documents derived from institution-wide student surveys. QAA wishes to emphasise that a compact well-argued written submission, drawing on the views of relatively few but representative students, can be just as valuable a briefing document as an extensive paper based on wide surveys, as long as the audit team knows the provenance and scope of the document. Whatever the scale of the written submission, as long as the team knows how it was generated and by whom, it will provide a valuable context for the team's studies and discussions.

Preparation

66 The audit process will begin with a preliminary meeting between the institution and a QAA AD to discuss the structure and content of the audit as a whole. This will be arranged about 24 weeks before the audit visit. The purpose of this meeting will be to clarify the scope of the exercise; to discuss the interactions between the institution, QAA and the audit team including the relative responsibilities of all the participants; to ensure that the institution's briefing paper will be well matched to the process of audit; to emphasise that documentary evidence should be based primarily on existing material used in internal quality management, not on material prepared specially for the audit; to discuss any matters relating to the summary prepared by QAA's Information Unit of the institution's TQI set (see paragraph 14); and to consider the basis for choosing audit trails. Between the preliminary meeting and submission of the institution's briefing paper, QAA will offer such advice and guidance on the process as it can, at the request of the institution.

67 The preliminary meeting will also provide an opportunity for a separate discussion with student representatives about the written submission that it is hoped they will wish to prepare on behalf of the student body. The purpose of this meeting will be to clarify the scope and purpose of such a paper; to explore any topics, beyond the standard template for the paper, that the student representatives consider appropriate; to note any views of the student representatives that might bear upon the team's choice of audit trails; and to discuss any matters relating to the TQI set or published material in general. Thereafter, until the submission of the students' paper, if they choose to submit one, QAA will offer such advice and guidance on the process as it can, at the request of the student representatives.

68 Institutions will be invited to nominate a contact at a senior level who is able to liaise on a regular basis with QAA, normally through the AD coordinating the audit. The development of a good working relationship between QAA and the institution through such liaison should help to ensure that the institution does not go to unnecessary lengths in its preparation for the audit through any misunderstanding by the institution of QAA's expectations, or through any misunderstanding by QAA of the nature of the institution or the scope of its provision.

69 Institutions and student representatives will be requested to submit their briefing papers no later than five weeks before the briefing visit. On receipt, QAA will distribute the documents to the audit team. The team will be asked to consult (by email) on possible audit trails, for discussion with the institution and student representatives during the briefing visit.

The briefing visit

70 An audit team's visit to an institution will take place in two parts. The first part is the briefing visit, which is an opportunity for the audit team to gain a sound understanding of the institution and its approach to the strategic management of academic standards and quality of provision prior to the audit visit. During the briefing visit the audit team will explore and gain further clarification of matters outlined in the briefing papers submitted by the institution and by the students, and will consider some of the evidence offered by documents cited as references in the institution's briefing paper. The briefing visit will also offer the institution and student representatives an opportunity to bring the team up to date on developments and changes since the briefing papers were submitted, and to raise with the audit team any other matters that they consider would be particularly worthy of exploration by the team during the audit visit.

71 This briefing visit will be held five weeks before the audit visit. The audit team will be on site for three days in all, of which two days will involve meetings between the team and key representatives of the institution and its students. From these meetings and from its study of the documents made available to it by the institution, the team will consider its detailed lines of enquiry for the audit visit, and will propose a programme for that visit. The final choice of audit trails will be made by the audit team during the briefing visit, following discussion with institutional representatives.

72 The documentation available to the team during the briefing visit will normally be limited to the illustrative material identified by the institution in the index to its briefing paper; institutions may choose whether to make these documents available to the team electronically or in hard copy or in a mixture of formats, whichever is most convenient for the institution. Auditors may also indicate what additional illustrative documentation they would like to be available at the start of the audit visit, or sometimes, in the case of particular key documents, before the audit visit. Any request for additional documentation will be limited to no more than is needed to inform the specific enquiries that the team will be undertaking.

The audit visit

73 The second part is the audit visit. This provides an opportunity for the team to pursue in greater depth the lines of enquiry that it identified at the briefing visit; to extend its study of the institution's documentation relating to the management of quality and standards; to meet particular groups of staff and students; and generally to explore the focuses of audit identified in paragraph 11 above. A particular feature of the audit visit will be the team's exploration of its selected audit trails. The audit visit is also an opportunity for the institution, having reflected upon the conversations of the briefing visit, to offer to the team any further clarification or evidence that it considers might help the team to understand its approach to the management of quality and standards.

74 The audit team will ensure that its programme for the audit visit includes meetings with students so that it can gain first-hand information on students' experience as learners and on their engagement with the institution's approach to quality assurance and enhancement. It will seek opportunities to meet student representatives during the audit visit in order to discuss with them, in the context of the views expressed in any written submission, the information that it has been gathering during the visits.

75 The audit visit will normally extend over five working days (ie, Monday to Friday), of which up to four days will involve meetings between the audit team and staff and students of the institution and, if relevant, its collaborative partners. On the final day of the audit visit, the audit team considers its findings in order to make preliminary decisions and agreements on:

the levels of confidence that it believes can be placed in the institution's safeguarding of the academic standards of HE awards made in its name, or under delegated authority if it does not have degree awarding powers, and the assurance of quality of the educational provision leading to those awards
the commentaries that it wishes to make on the institution's approach to enhancing the quality of its educational provision, on the arrangements for maintaining appropriate academic standards and enhancing the quality of postgraduate research programmes, and on the accuracy and completeness of the information that the institution publishes about the quality of its educational provision and the standards of its awards
features of good practice that it wishes to highlight as making a particularly positive contribution to the institution's approach to the management of academic standards and quality of provision
recommendations for action by the institution, categorised in terms of importance and/or urgency.

76 The audit team will rehearse its preliminary findings during its private meeting at the end of the audit visit, but will continue to refine those findings through internal communications over the next few days, having had a further chance to reflect on all the information that it has gathered. Consequently, it would be premature for the team to report to the institution on the findings of the audit while on site. Instead, a 'key findings' letter will be sent to the head of the institution within two weeks of the end of the audit visit, outlining the probable main findings and likely recommendations in the draft report. This letter will be provided as a courtesy to institutions, to ensure some feedback close to the completion of the audit visit and will be provided for preliminary information only. It will not be part of the draft report, nor will it be an opportunity for discussion about the possible contents of the draft report.

77 During the final day of the audit visit, the QAA AD and some members of the audit team will offer the institutional contact a short courtesy meeting to mark the close of the team's visit, on the understanding that the team will not be in a position to give feedback on the findings of the audit at this time.

Liaison with the institution

78 During both visits to the institution a representative of the audit team will have regular scheduled liaison meetings with a named institutional contact. Such meetings will provide opportunities for the team to seek guidance or clarification outside of the formal meetings of the visits, and for the institution to raise with the team matters where it believes that further explanation or documentary evidence would help the team in its deliberations. It is important that such liaison meetings are between named representatives so as to reduce the danger of confused communications. While each auditor will lead on one or more sections of the audit report, and will chair relevant meetings during the visits, it is the audit secretary who monitors and records the gathering of information by the team as a whole. QAA considers, therefore, that there would be advantage in the audit secretary being the liaison contact on behalf of the audit team, an enhancement of that role that would be addressed in the training of auditors and audit secretaries for the revised process.

Reports

79 The revised institutional audit process makes a clear distinction between that part of the reporting process aimed at an external audience and that aimed at the institution. The report will comprise an overview of the findings of the audit, aimed at both lay and external professional audiences, and will have a 'technical' annex to carry the detail and explanatory material that is intended to be of practical use to the institution (see Annexes B and C). The report will be as concise as possible while covering enough detail for it to make sense to an external audience as a stand-alone document. It will be cross-referred to paragraphs of the annex. The annex will be as long as necessary for the audit team to explain to the institution the reasoning behind its findings. There will also be a brief summary (see Annex B) of the findings of the report, including the judgements, intended for the wider public, especially potential students. The summary and the report, without the annex, will be published in hard copy. The institution will also receive the annex in hard copy. The summary, the report and its annex will be published in full on QAA's website.

80 Apart from the introductory part of the report, each section will conclude with one of the judgements or comments identified in paragraphs 16 and 23 above, and will outline the team's analysis leading to that judgement or comment. Where appropriate (see paragraph 10), the institution's management of its collaborative provision will be addressed in a separate section of the report. The format of the report and its annex will follow a template that aligns with the template recommended for the institution's briefing paper. Its production will be coordinated by the AD coordinating the audit. The draft report and its annex will be prepared and submitted to the institution as soon as possible following the audit visit, normally within eight weeks, with a request for corrections of errors of fact.

81 The final report and its annex will be prepared in the light of the institution's response. Institutions will be invited to submit a brief (500 word) statement to be attached to the report. The purpose of the statement is to provide an opportunity for the institution to report, if it wishes, on developments since the audit visit, particularly in respect of actions taken or proposed to address the report's recommendations. It should not be taken as an opportunity to challenge the audit judgements or recommendations

Mid-cycle follow up

Purpose of the mid-cycle follow up

82 The mid-cycle follow up is an integral part of the overall institutional audit process and will support the same aims. It will serve as a short health check, for the institution and for QAA, on the institution's continuing management of academic standards and quality of provision, normally some three years after an institutional audit. It will be an opportunity to reflect upon developments made in the management of standards and quality within the institution since the previous institutional audit, and, in the context of the findings of that audit, for QAA to advise the institution of any matters that have the potential to be of particular interest to the team that conducts the institution's next audit.

83 An institution may, if it wishes, use the mid-cycle follow up to seek an external view of changes made, or proposed, in the management of standards and quality. While it is not appropriate for QAA to act as a 'consultant' in such matters, it will nevertheless be prepared to discuss changes and proposals in general terms.

Process

84 The mid-cycle follow up will take the form of a paper-based exercise conducted by two senior QAA officers. It will be based on existing institutional documentation with the intention of making as little demand as possible on institutions. The timing will be agreed with each institution as part of the institutional audit schedule.

85 The contextual material that normally will be accessed directly by QAA for the mid-cycle follow up is:

the report of the previous institutional audit
the report of any separate collaborative provision audit since the institutional audit
the reports of any other QAA reviews of the institution or its subject areas since the institutional audit
the relevant reports of other external bodies, such as PSRBs, made available to QAA through the HE Concordat
information on the TQI site, including NSS outcomes
information on any major strategic change, including that reported by the institution to HEFCE.

86 The institution is asked to comment upon:

actions taken to address the recommendations in the institutional audit report
actions taken to address any recommendations in other QAA reports since the audit
any significant changes in collaborative arrangements since the audit.

The commentary will take the form of a short briefing paper, using existing documents where possible. For example, the section of the briefing paper on actions taken to address recommendations might take the form of papers drawn from the records of key committees with some notes to help the readers follow the sequence.

87 The briefing paper should also indicate the way in which the Students' Union, or other student representation, had been engaged in preparing the institution's briefing paper, and supported the institution's comments.

88 QAA will also ask the institution to provide a sample, selected by QAA, of internal review reports for which summaries have been placed on the TQI site. The purpose of this sampling is to permit QAA to discharge its responsibility for monitoring the accuracy and completeness of information on the TQI site. The size of the sample will not exceed 20 per cent of the internal review reports whose summaries have been placed on the TQI site since the last institutional audit. The selection of the reports in the sample will be made by QAA, and will be notified to the institution eight weeks in advance of the due date for the receipt of the institution's briefing paper for the mid-cycle follow up.

89 The institution's briefing paper, the information listed in paragraph 85 above and the sample internal review reports will be read by two senior QAA officers. They will produce a short report that will comment on:

the progress that has been made against the recommendations made in the report of the institutional audit
any matters that have the potential to be of particular interest at the institution's next institutional audit
the accuracy and completeness of information on the TQI site as represented by the summaries of a sample of internal review reports.

90 A draft of QAA's report will be sent to the institution for comment. The final report will be sent to the institution and to HEFCE but will not be published. The mid-cycle follow up is not a peer review process conducted by an audit team. The report of the follow up cannot, therefore, result in a recommendation or a decision that the judgements of the previous audit team are modified or revised, but it will be used as part of the evidence base for the next engagement by QAA with the institution. The institution will not be asked to make a formal response to the report, although it may do so if it wishes.

Sign off and follow up

91 The audit will be completed when it is formally signed off. Where the audit report offers positive statements of confidence and no recommendations identifying matters of importance requiring urgent attention, the audit will be formally signed off on publication. At the mid-cycle follow up, institutions will be asked to discuss with QAA progress on developments in the management of academic standards and quality of provision since the audit. Only if QAA has cause for concern in the light of what it has seen and heard will any further activity be proposed.

92 Where an audit team makes a judgement of limited confidence or no confidence, the report will be published and there will be a programme of follow-up action. QAA will require an action plan from the institution and will request progress reports at regular intervals. The audit will not be finally signed off until the institution indicates that the action plan has been completed and implemented successfully, with a maximum time limit of 18 months. If, at that point, there remain concerns about the effectiveness of the remedial action, QAA will conduct a further visit; and if satisfactory progress has still not been made, in the case of institutions in England in receipt of HEFCE funding, HEFCE reserves the right to withdraw some or all of that funding. In the case of institutions not in receipt of public funding, QAA will use its discretion to decide whether the matter is of sufficient importance to warrant a further separate focused review. While QAA cannot act as a consultant to institutions in respect of action plans, it will be prepared to comment on an institution's proposals.

Administration of the process

93 An AD from QAA will have responsibility for the coordination and management of each audit. Every effort will be made by QAA to ensure that a close and constructive working relationship is established with institutions, and that this is actively maintained beyond the specific requirements of institutional audit. QAA already operates an optional institutional liaison system, whereby named correspondents within institutions liaise with designated staff of QAA (liaison officers) on a continuing basis. The AD who coordinates an audit will not be the Liaison Officer for that institution.

94 The audit's judgements and comments, including any recommendations and the statements about levels of confidence, will be decided by the audit team as peer reviewers. The coordinating AD will ensure that all judgements and comments are backed by adequate and identifiable evidence, and that the audit report provides information in a succinct and readily accessible form, with reference to the more detailed evidence provided in the annex to the report. To this end, QAA will retain editorial responsibility for the final text of the report and its annex.

95 QAA will continue to protect the quality of the process itself through explicit service standards (www.qaa.ac.uk).

Timetable for implementation

96 The revised institutional audit cycle will extend from 2005-06 to 2010-11, with the first visits of the revised method taking place in January 2007. In the spring of 2006, QAA will propose each institution's place in the schedule of audits. Preliminary meetings with the first institutions to be audited under the revised process will also take place in the spring of 2006. Auditors and audit secretaries will be recruited in mid-2006; their training will be provided during the summer and autumn of 2006.

Timetable for individual institutional audits

97 An institution's briefing paper and, if there is one, a students' written submission should be available to the audit team five working weeks before the briefing visit, which itself will normally be five weeks before the audit visit. The timing of the audit within the academic year may be such that a submission by the students of their document closer to the briefing visit would allow the student officers and representatives who meet the audit team to have been directly involved in the production of the students' document. In such cases QAA is willing to consider a request for later submission of the students' document. QAA continues to see merit in holding the preliminary meeting well in advance of the visits, and would suggest that the target date for a preliminary meeting should be 24 weeks before the audit visit. The timing of the visits will, of course, have to take account of the availability of the audit team and the availability of students. Any adjustments to allow for institutional or statutory holiday periods will need to be negotiated between QAA and the institution, at the institution's request.

98 The final report and its annex will be published 20 weeks after the audit visit. That schedule assumes that the draft report and its annex will be received by the institution within eight weeks of the audit visit, and that the institution will respond to the draft report within four weeks of its receipt. Publication may be later than 20 weeks if there are circumstances that delay determination of the final text of the report. An outline of the process from the preliminary meeting to publication is given in Annex A.

Complaints and representations

99 Complaints about the conduct of the audit and representations against the judgements made by the audit team are considered by QAA in accordance with the formal procedures published on its website (www.qaa.ac.uk/aboutus/policy/intro.asp).

Annex A: Outline of the institutional audit process

Audit visit minus not less than 28 weeks	QAA's Information Unit (IU) provides the QAA Assistant Director (AD) with a summary of the institution's TQI set and a commentary which will be sent to the institution.
Audit visit minus not less than24 weeks = preliminary meeting	AD visits institution to meet institutional representatives and students. AD provides briefing on the process of audit and provides guidance on the institution's briefing paper and the student submission. Institution comments on the accuracy of the IU TQI summary paper.
Audit visit minus 10 weeks	QAA receives the institutional briefing paper. QAA receives the students' written submission if applicable (see note 1 below).
Audit visit minus five weeks = briefing visit	Audit team and AD undertake the briefing visit tothe institution. Audit team holds a meeting with the head of the institution, appropriate senior staff of the institution and student representatives. Audit team identifies broad lines of enquiry for the audit visit. Audit team meets institutional representatives to agree the programme for the audit visit and audit trails. Any additional information required before or during the audit visit is identified.
Audit visit	Audit team visits the institution for up to five working days; AD joins the team for the final part. Audit team meets staff and students for discussion of lines of enquiry, including the audit trails.
Audit visit plus two weeks	Letter outlining the audit findings is agreed by the audit team and sent to the head of the institution by the AD.
Audit visit plus eight weeks	QAA sends the draft report to institution.
Audit visit plus 12 weeks	Institution responds to the draft report.
Audit visit plus 20 weeks	Report is published on the web.

Note 1: The timing of the audit within the academic year may be such that a submission by the students of their document closer to the briefing visit would allow the student officers and representatives who meet the audit team to have been directly involved in the production of the students' document. In such cases, QAA is willing to consider a request for later submission of the students' document.

Annex B: Structure of the institutional audit report

Summary

The judgements

Recommendations and features of good practice

The comments

Report

Section 1 Introduction and background

This section sets the context for the audit, and briefly reviews developments since the last audit (and mid-cycle follow up). It will focus on action taken on the outcomes of the last audit (and of the mid-cycle follow up).

Section 2 Institutional management of academic standards

The focus of this section is on the effectiveness of internal processes in making sure that academic standards are secure. The use made of external reference points and of accreditation/recognition by professional, statutory and regulatory bodies (PSRBs) and other external bodies would be a feature of this section. A large part of the team's evidence base could be expected to come from the sampling audit trails, exploring the relationship between internal processes at programme level and at institutional level.

This section of the report closes with a judgement on the confidence that can reasonably be placed in the soundness of the institution's present and likely future management of the academic standards of its awards.

In a case where the team's commentary on collaborative provision (section 5 below) would give rise to a judgement in respect of collaborative provision different from that for the institution's own provision, that difference will be reflected in the form of words used to express this judgement.

Section 3 Institutional management of learning opportunities

This is a larger section, with a focus on the combined effect of internal review in making sure that there is an appropriate quality of learning opportunities for students to be able to achieve the desired academic standards, and the support available to students to help them make good use of those opportunities. The use made of external reference points and of PSRBs and other external bodies, including the Higher Education Academy, would be a feature of this section, as would the publicly available summaries of periodic reviews. A large part of the team's evidence base could be expected to come from the sampling audit trails, exploring the relationship between internal processes at programme level and at institutional level.

The students' written submission (if there is one) is likely to be given emphasis in this section, along with the use made of information deriving from student representation and feedback, including the National Student Survey. The use made of student support through profiling/PDP and personal skills development is likely to be a feature of this section.

Section 4 Institutional approach to quality enhancement

For the purpose of institutional audit, quality enhancement is defined as the process of taking deliberate steps at institutional level to improve the quality of learning opportunities. The focus of this section is on the way that the institution takes these 'deliberate steps'. It is not a presentation of examples of enhancement of learning opportunities, although such examples might be used to illustrate the effectiveness of the institution's systematic approach to quality enhancement.

This section of the report will comment on the institution's approach to developing and implementing institutional strategies for enhancing the quality of its educational provision, both taught and by research.

Section 5 Collaborative arrangements

The focus of this section is on any significant differences in approach or implementation between the institution's management of academic standards and quality of provision in its collaborative arrangements from the management of its 'home' provision. Where there are significant differences, comment will be made on the effectiveness of the management of the collaborative arrangements, in the context of the guidance in the Code of practice for the assurance of academic quality and standards in higher education (Code of practice).

This section of the report will comment on the extent to which any of the judgements or comments in other sections of the report can be applied to the institution's management of its collaborative provision.

Where there is to be a separate collaborative provision audit, the section heading will remain for consistency, but will have a standard paragraph referring to a separate collaborative provision audit.

Section 6 Institutional arrangements for postgraduate research students

The focus of this section is on the institution's arrangements for its postgraduate research degree programmes, taking as a point of reference the Code of practice, Section 1: Postgraduate research programmes, September 2004.

This section of the report will comment on the institution's arrangements for maintaining appropriate academic standards and quality of provision of postgraduate research programmes.

Section 7 Published information

This section focuses on the institution's arrangements for ensuring that the information it publishes about its educational provision and the academic standards that it supports is accurate and complete. The institution's published teaching quality information (TQI) will feature in this section, with the audit team commenting on the basis of analysis by QAA's Information Unit and by its own sampling of TQI in the audit trails. The content of the section will also be informed by the students' written submission (if applicable) and by the team's discussions with students.

This section of the report will comment on the reliance that can reasonably be placed on the accuracy and completeness of the information that the institution publishes about the quality of its educational provision and the standards of its awards.