1 An institutional audit of the University of Bristol (the University) was undertaken during the week commencing 26 April 2004. The purpose of the audit was to provide public information on the quality of the University's programmes of study and on the discharge of its responsibility for its awards.
2 The audit was carried out using a process developed by the Quality Assurance Agency for Higher Education (the Agency) in partnership with the Higher Education Funding Council for England (HEFCE), the Standing Conference of Principals (SCOP) and Universities UK (UUK), and has been endorsed by the Department for Education and Skills (DfES). For institutions in England, it replaces the previous processes of continuation audit, undertaken by the Agency at the request of UUK and SCOP, and universal subject review, undertaken by the Agency on behalf of HEFCE, as part of the latter's statutory responsibility for assessing the quality of education that it funds.
3 The audit checked the effectiveness of the University's procedures for establishing and maintaining the standards of its academic awards; for reviewing and enhancing the quality of the programmes of study leading to those awards; and for publishing reliable information. As part of the audit process, according to protocols agreed with HEFCE, SCOP and UUK, the audit included consideration of examples of institutional processes at work at the level of the programme, through discipline audit trails (DATs), together with examples of those processes operating at the level of the institution as a whole. The scope of the audit encompassed all of the University's provision and collaborative arrangements leading to its awards.
Section 1: Introduction: the University of Bristol
The institution and its mission
4 The University, which was formed from a number of institutions including the Bristol Medical School, received its Charter in 1909. It comprises 45 departments clustered into six Faculties: Medical and Veterinary Sciences; Science; Engineering; Arts; Social Science and Law; and Medicine and Dentistry. The majority of departments are located in buildings that are within half a mile of the city centre. The Faculty of Medicine and Dentistry is located on several sites which include a number of centrally located University buildings as well as major Bristol National Health Service (NHS) Trust premises in central, south and north Bristol. The Faculty of Medical and Veterinary Sciences is split geographically: the medical sciences departments are centrally located; the Department of Clinical Veterinary Science is 14 miles away. The University has full degree-awarding powers for both taught degrees and research degrees. There is a strong research culture with 36 from 46 units of assessment rated at five or above in the 2001 Research Assessment Exercise (RAE).
5 At the time of the audit, the University had over 21,000 students on credit-bearing programmes and units, with a male:female ratio of 49:51. There were 11,416 undergraduates, most of whom were on full-time programmes and 9 per cent were overseas students. Postgraduate students totalled 5,299, with 65 per cent following taught programmes and 35 per cent engaged in research; 46 per cent of postgraduate students were part-time. Over the past five years there has been growth of 15 per cent in undergraduate numbers, 28 per cent in research postgraduates and 107 per cent in taught postgraduates. There is a Widening Participation Unit, guided by the University Strategy for Widening Participation 1999. Although the proportion of entrants from non-independent schools has risen from 55 to 62 per cent since 1999, the University recognises that it is below the benchmark set by HEFCE for this area.
6 The current Vice-Chancellor took up his appointment in September 2001 and was the catalyst for a new vision and consequent restructuring which were informed by extensive consultation across the University. A new faculty structure was introduced in August 2003, reflecting a decision to 'make faculties the budget centres of the University with a key role in policy formulation'. At the time of the audit, a number of policies and strategies in support of the University Plan were in development.
7 The mission of the University is: 'The University of Bristol is a world-class institution that contributes to society through scholarship and research, by developing creative graduates and through its cultural, social, economic and environmental activities. These ambitions are realised in a supportive and positive work environment'.
8 The University has seven educational partnerships and is a partner in two consortia, some in the UK and some overseas, involving 940 students, at both undergraduate and postgraduate level. The audit process included consideration of the University's approach to collaborative provision.
Background information
9 The published information for the audit included:
- the information on the University's web site and its prospectuses;
- the report of an audit by the Higher Education Quality Council (HEQC) in 1993;
- reports of reviews at the subject level.
10 The University initially supplied the Agency with:
- an institutional self-evaluation document (SED);
- extract from the University Plan 2003-8;
- University Learning and Teaching Strategy 2002-2005;
- framework of University plans and strategies;
- campus map;
- diagram of faculty structures;
- summary of University governance and committees, including committee structure diagram;
- diagram of support services structure;
- progress report to HEQC one year after the 1993 audit;
- diagram showing the University's quality assurance framework for learning and teaching;
- University credit framework, June 2003;
- undergraduate and taught postgraduate progression statistics 1998-99 to 2002-03, by faculty;
- Rules and Regulations for Students, September 2003;
- a discipline self-evaluation document (DSED) for the provision selected for the DATs.
11 During and after the audit visit, the audit team was given ready access to a range of the University's internal documents in hard copy and through the intranet and to documentation relating to the DATs; the latter included samples of student work.
The audit process
12 Following a preliminary meeting at the University in August 2003, the Agency confirmed that five DATs would be conducted during the visit. On the basis of the SED and supporting documentation, and the students' written submission (SWS), the audit team selected the following DATs: programmes in chemistry; dentistry; engineering mathematics; politics; and theology.
13 The Agency received the SED and supporting documentation in December 2003 and the DSEDs, accompanied by programme specifications, in March 2004. The SED and the DSEDs for dentistry, politics and theology were written specifically for the audit. The DSEDs for the chemistry and engineering mathematics DATs were based on documentation produced for the most recent departmental review in those areas.
14 The audit team visited the University on 16 and 17 March 2004 for the purpose of exploring, with the Vice-Chancellor, senior members of staff, and student representatives, matters relating to the management of quality and standards raised by the SED and other documentation. During the briefing visit the team signalled a number of themes for the audit visit. At the close of the briefing visit, a programme of meetings for the audit was developed by the team and agreed with the University.
15 At the preliminary meeting for the audit, the students of the University were invited, through their Students' Union (SU), to submit a separate document expressing views on the student experience at the University, and identifying any matters of concern or commendation with respect to the quality of programmes and the standard of awards. They were also invited to give their views on the level of representation afforded to them and on the extent to which their views were taken into account. The student body submitted to the Agency a SWS produced by the SU, drawing on information and views from a number of sources. The student body indicated that the document had been shared with staff in the University and that there were no matters within it that would require the audit team to treat it with any level of confidentiality greater than that applying to the audit process. The team is grateful to the students for preparing this valuable document to support the audit.
16 The audit visit took place from 26 to 30 April 2004 and included further meetings with staff and students of the University, both at central level and in relation to the selected DATs. The audit team comprised Dr J Barry, Dr R Davison, Professor S Dilly, Professor J Hughes, Professor A Gale and Professor T Kemp, auditors, and Ms S Welham, audit secretary. The audit was coordinated for the Agency by Mrs S Patterson, Assistant Director, Reviews Group.
Developments since the previous academic quality audit
17 The University was audited by the HEQC in 1993. The report of the audit commended the University in a number of areas, including the balance in its quality assurance processes between central guidance and local responsibility and the clear guidance on the development of new programme proposals and annual monitoring; the report noted that the University was introducing periodic assessment of departmental performance. Aspects of student support and communication with students were identified as good practice, including the quality and usefulness of publications issued to students. Elements of the University's staffing policies commended included staff development, remission of teaching for probationary staff and the use of a rapporteur in promotion procedures. At the time of the 1993 audit visit, the University was planning the introduction of a modular system from the start of the 1994-95 academic session. The audit team commended the University for its 'well-judged approach to modularisation'; modularisation is now fully embedded across the University, except in the professional undergraduate programmes in medicine, dentistry and veterinary science.
18 The report also identified a number of areas for further action for consideration by the University: extension of external input into programme development, approval and review (see paragraphs 62 to 64); assessment at interview of teaching skills and a higher take up of the staff induction programme (see paragraph 97); a more active approach to the development and dissemination of innovation in teaching (see paragraph 98); strict observation of promotion procedures (see paragraph 98); the provision of consistent information to students on assessment and classification; and the establishment of greater uniformity in approaches to student feedback and representation (see paragraphs 85 and 91).
19 Since the previous audit the University has undertaken a reorganisation, with a new faculty structure being implemented in August 2003. Quality assurance processes have been formalised and the Statutes, Ordinances and Regulations have been revised to ensure fitness for purpose and to clarify responsibilities and authority. The Education Committee, constituted in 1994, is the principal quality assurance committee for learning, teaching and assessment.
20 Scrutiny of proposals for new programmes has been strengthened through the formation of the New Programmes Group (NPG), a subgroup of the Education Committee, which considers both undergraduate and postgraduate proposals. A rolling programme of quinquennial departmental review has been established and is identified by the University as 'one of the most important aspects of [its] quality assurance procedures' (see paragraphs 58 to 60).
21 Since the 1993 audit the University has participated in nearly 40 reviews at the subject level. In all cases, the quality of the provision was approved. Several subject reports commented favourably on student progression and achievement as well as on student support and guidance. The University has introduced a number of changes in response to these reports including, changes to the processing of external examiner reports (paragraphs 65 to 69), the introduction of Annual Programme Review (APR) in the academic year 2001-02 (paragraphs 51 to 56) and support for the development of information and communications technologies (ICT) in learning, teaching and assessment (paragraphs 108 and 109) .
22 The University's collaborative arrangement with MusicSpace in Italy was the subject of an overseas audit by the Agency in 2003. The report contained a number of recommendations designed to encourage a more formalised and systematic approach to the University's collaborative provision. The University's approach to collaborative arrangements is discussed in paragraphs 128 to 135.
23 The audit team concluded that the action taken by the University in response to previous audit and subject review reports had been appropriate and effective. Further detail on the actions taken by the University may be found in the paragraphs indicated above.
Section 2: The audit investigations: institutional processes
The institution's view as expressed in the SED
24 In describing the University's framework for managing quality and standards, the SED commented that 'most of the components of the University's quality assurance and enhancement framework...[had] developed since 1993'.The institution's framework for managing quality and standards, including collaborative provision
25 The Senate, chaired by the Vice-Chancellor, is the principal academic policy-making committee of the University. Its role in quality assurance is mainly the approval of new and revised programmes of study and of Statutes, Ordinances and Regulations. The Senate receives reports from, inter alia, the Vice-Chancellor, the Registrar, the University Planning and Resources Committee (UPARC), and the faculty boards. In assimilating recommendations from various groups, Senate ensures the coherence of the regulatory framework.
26 The Teaching Support Unit (TSU) is responsible for managing the implementation of the quality assurance framework and the Education and Learning and Teaching Strategies. It provides the secretariat for Education Committee and the majority of its working groups, has formal membership of Faculty Quality Assurance Teams (FQAT) (see paragraph 30), supports the development of programme specifications, and works in collaboration with the Examination Office in relation to external examining. In its enhancement role it has membership and provides the secretariat for Faculty Learning and Teaching Development Groups (FLTDG), support for the Learning and Teaching Advisers' network, and contributes to departmental and faculty workshops and away days.
27 In December 2003 the TSU published a Handbook for Academic Quality and Standards (the Handbook), which the audit team found to be a useful compilation of all policy documents and guidelines and which is supplementary to the definitive version on web pages maintained by the TSU. The electronic version is updated whenever changes are introduced, and it is intended that the Handbook will be reissued annually.
28 UPARC, chaired by the Vice-Chancellor, is the academic and resources policy committee and reports to Senate and to Council, when appropriate; all deans and heads of services are members. Education Committee, chaired by a Pro Vice-Chancellor, is the principal quality assurance committee for teaching, learning and assessment; its terms of reference and membership were updated in October 2003 and are kept under review to ensure that they accord with the emerging Education Strategy. Education Committee reports to UPARC to ensure that proposals are properly integrated with the University's academic, administrative and financial planning processes. While Education Committee meets at least four times a year, its working groups meet more frequently, with the Working Groups of Undergraduate and Graduate Deans meeting up to six or seven times annually.
29 In meetings with senior members of the University the audit team explored the relative roles of UPARC and Education Committee and the extent to which the Senate was able to fulfil its educational functions without receiving reports directly from Education Committee. The team was reassured to hear that all papers and reports were available on the University's intranet and that any member of the Senate was free to place items on the agenda. The team was also told that the Standing Committee of Senate, while its principal role is to prepare the business of Senate, is seen as the guardian of Senate in ensuring that it is able to debate major educational issues. The team learned that the membership and terms of reference for the Senate were shortly to be reviewed but was informed that any changes would be largely to ensure appropriate representation following the reorganisation of the faculties. There was also some debate about the future of the Education Strategy Group now that it had completed its work; one possibility would be for it to assume responsibility for the University's alignment with external developments with implications for the University's approach to the management of quality and standards.
30 FQATs were established in 1994 in part in response to the report of the 1993 audit. They have three key roles: to enable faculties to take responsibility for quality assurance; to ensure quality assurance and enhancement are closely linked; and to provide a common University framework, while allowing flexibility for individual faculties' needs. The Faculties of Science, Engineering, Arts, and Social Science and Law each have an FQAT. The FQAT for Medicine covers all of medicine, dentistry and veterinary sciences while the medical sciences provision is covered by the Science FQAT. Guidelines for FQATs were introduced in 1994 and revised in 2002 and 2003. FQATs receive APRs, visit individual departments meeting both staff and students, and prepare an annual report for faculty boards, highlighting examples of good practice and noting issues of concern. A significant development was the inclusion from 2002 of a representative of the TSU as a full member of each FQAT.
31 The Plenary Group of FQAT Chairs, which reports to Education Committee, meets at least twice a year and is chaired by the Director of the TSU, a role which places the TSU at the heart of quality monitoring and review and enhancement. It acts as a forum for exchanging information about quality assurance and good practice, promoting consistency of practice as far as is 'possible and desirable', considers faculty overview reports prepared by the FQAT Chairs, receives an overview of external examiners' reports and identifies themes to be pursued in the coming year. The SED stated that in future, in addition to these tasks, the Plenary Group would deal with operational aspects of the work that was formerly carried out by the now discontinued Learning and Teaching Group. This will include matters arising from external initiatives such as guidance from the Agency and HEFCE.
32 As evidenced by the minutes of its meetings, the Plenary Group has developed, in the view of the both the University and the audit team, into a very significant component of the quality and standards structure. The SED commented that FQAT chairs had had 'a major effect in providing the necessary academic leadership to faculties a) in formalising and then developing quality assurance processes and (b) in spreading good practice'. From its meetings with FQAT chairs and its reading of the minutes of FQAT and Plenary Group meetings, the team endorses this view with respect to taught programmes. The team was unable to find a consistent approach to review of the postgraduate research student experience in departmental reviews or in FQAT reports. The team considers the work of the Plenary Group of FQAT Chairs in identifying opportunities for enhancement and matters for further consideration at taught programme level as a praiseworthy feature of the University's approach to quality assurance and enhancement.
33 The Plenary Group of FQAT Chairs and the Working Groups of Undergraduate and Graduate Deans 'work together closely', as exemplified in the production of the University Assessment Guidelines: the FQATs identified good practice, the TSU used this to draft the Guidelines for consideration by the Learning and Teaching Group, which led to the version approved by Senate in June 2000. Refinements were made both following the publication of the Code of practice for the assurance of academic quality and standards in higher education (Code of practice), Section 6: Assessment of students, published by the Agency, and in response to work of the FQATs and the Plenary Group of FQAT Chairs. The current version of the Assessment Guidelines was approved by Education Committee in June 2002 and appears in the Handbook.
34 The NPG, coordinated by the Deputy Director of the TSU, examines proposals for new programmes, being charged with ensuring that: appropriate resources are available; there are fully developed programme specifications; there is support from two external referees; and there is evidence of a student market. The Group also advises on any necessary changes to Ordinances and Regulations. The University regards NPG as exercising rigorous scrutiny, citing examples of programmes either rejected or requiring radical amendment. The SED reported that Education Committee had debated the devolution of programme approval to faculties in the interests of flexibility and speed of approval; it stated that the likely outcome of the debate would be to retain the NPG with an expanded membership.
35 Prior to the introduction of APR (see paragraphs 51 to 56), the Programme Review Group (PRG) was responsible for reviewing programmes on a rolling basis; its membership is the NPG, Chairs of FQATs and the Director of TSU. The Group is now convened only as required to undertake special reviews of particular programmes or group of programmes. Oversight of departmental review is the responsibility of UPARC; in discussion with the audit team, the University commented that reports of departmental review could, with advantage, be forwarded to the Education and Research Committees following approval by UPARC.
36 The SED stated that 'faculties and departments [were] supported by extensive University guidelines in relation to learning, teaching and assessment', with recent attention being focused particularly on the latter. All departments are 'expected to have in place clear marking criteria and stated student learning outcomes at unit and programme level'. The audit team noted that intended learning outcomes (ILOs) were included in the emerging set of programme specifications. Unit learning outcomes are first developed at the approval stage and will in due course be accessible through the University Portal. The Assessment Guidelines advise that 'marking criteria should be explicit and transparent to students'; the Working Group of Undergraduate Deans, in collaboration with the TSU, is 'producing some examples of generic marking criteria for different types of assessment to support departments in this area'. The Guidelines specify that assessment methods should test the ILOs and the SED stated that 'we advise departments that it is not good practice to rely solely on one assessment method, as this is unlikely to be able to test a variety of learning outcomes'. Its reading of a variety of programme specifications led the team to view these as acceptable. At the time of the audit, departments were not required to have in place full unit specifications for all units; the team noted that many of the existing units had specifications lacking ILOs or a statement of assessment methods (see paragraphs 75 and 76).
37 Guidance on anonymous marking is set out in the Handbook: candidates' scripts are anonymised but examination boards may discuss candidates' cases either before or after the marks have been identified with individuals, depending on the policy of the individual department. Each faculty is required to have a faculty-level examination board. The SED noted that 'there [was] some variation in faculty classification guidelines, according to faculty structure and the nature of subjects within that faculty'. This has excited comment from external examiners (see paragraph 68), which has been discussed at the Plenary Group in the context of the overview of external examiners' reports. The SED asserted 'we are continuing to consider how to implement a uniform system of degree classification, but we are satisfied that fairness to students is always the primary consideration for examination boards': the Working Group of Undergraduate Deans is working towards uniformity of practice by 'monitoring and reviewing classification procedures'. The audit team advises the University to re-examine its approach to the reliability and equity of undergraduate assessment across departments and faculties in the interests of parity and consistency in the treatment of students.
38 The Working Group of Graduate Deans, prompted in part by external examiners' comments, produced a paper on Levels and Pass Marks for Taught Postgraduate Programmes proposing a Pass mark of 50 per cent; faculties and departments can choose to have a higher, but not a lower, Pass mark than 50 per cent. The crucial role of external examiners is discussed fully elsewhere (paragraphs 65 to 69). Faculty Graduate Studies Committees act as examination boards for taught postgraduate awards and the University's Higher Degrees Examination Board, chaired by a Pro Vice-Chancellor, is the overall examination board for all research degrees.
39 The University has seven educational partnerships and is partner in two consortia, some in the UK and some overseas, involving 940 students at both undergraduate and postgraduate level. The SED stated that management of educational partnerships was overseen by Education Committee, supported by FQATs. The University has recently updated its Guidelines for Educational Partnerships in line with the Code of practice, Section 2: Collaborative provision. During 2003-04 the Programmes Review Group is undertaking a review of all partnerships and will report its findings to Education Committee with any recommendations for action. The SED stated that all collaborative arrangements were subject to the same quality assurance procedures as the home provision. See further discussion in paragraphs 128 to 135.
40 In its reading of University committee papers, the audit team formed the view that UPARC was a powerful committee with a considerable range of responsibilities not fully reflected in its title. The team found that the University had given detailed consideration to organisational change and the roles of various committees and key responsibility holders during a period of exceptional transformation. It had also generated extensive documentation in the form of a range of policies and guidelines. In so doing, the University was seeking to ensure good communication both vertically and horizontally within its quality assurance framework. A key aim was to achieve a balance between central regulation and local responsibility, with a major shift to the latter. The team observed that the role of the TSU was crucial to the effective operation of the framework. There was evidence of a sustained level of activity in relation to quality and standards issues and a continuing debate as to where to locate important functions such as programme approval. This meant that it was not always easy for the team to discern the reporting routes for committees and groups.
41 The audit team formed the view that because of a historic tension between central regulation and local autonomy, it was not always clear which University requirements were mandatory, which were advisory, and which were optional. When asked about the balance between central regulation and local autonomy senior members of the University asserted that the current balance was 'about right'. In other sections of this report the team has identified examples where the extent of the University's present flexibility of approach has created potential threats to standards and quality (see, for example, paragraphs 37, 73 and 95)
42 The audit team concluded that the committee structure was appropriate for the discharge of the University's responsibility for overseeing the quality and standards of its taught programmes, and that the FQATs and Plenary Group were functioning in an effective manner. Although the University's framework for the assurance of quality and standards was still evolving and its overall effectiveness, efficiency and economy were, therefore, difficult to gauge, the team was reassured by the quality of debate revealed in the minutes of the University's committees and the frank and open way in which matters of educational importance were discussed. The quality of debate revealed a capacity for critical analysis and reflection, which had, in the view of the team, always been apparent in the SED. Some of the discussions in meetings with the team were refreshingly frank, with staff recognising openly that the audit process had revealed challenges, which needed to be addressed. As a result, the team gained confidence that the University was fully committed to securing an effective framework for managing quality and standards. A key element in securing this confidence was the manifest vigilance of the TSU in its oversight of the management of University business and the strong collegial working relationship between TSU and the academic staff.
The institution's intentions for the enhancement of quality and standards
43 The SED devoted a substantial section to the University's approach to enhancement of quality and standards and claimed that its mechanisms for enhancement were integrated with its quality assurance processes, in particular APR, the FQATs, departmental review, and external examining. The TSU is well placed through its involvement in these processes to identify good practice and promote enhancement through dissemination. The SED also set out the enhancement benefits of a research-based education, its impact on staff appointments and on the curriculum, as well as providing advanced facilities that enrich the students' learning experiences.
44 The notion of enhancement is implicit in the University Plan and in the list of Strategic Goals of the Teaching and Learning Strategy (2002-2005). The University has produced an Education Strategy which, at the time of the audit, was in the final stages of approval. The Education Strategy will be supported by the Learning and Teaching, Participation, eLearning, and Careers Strategies, and the Admissions Principles and Procedures. The Education Strategy states that 'The Education and Research Strategies are complementary in supporting the academic aspirations of the University Plan...and are equally important to the University's development and continuing success'. The University has established the post of Faculty Education Director to provide further support for the University's plans for enhancement by identifying an individual in each of the faculties with an explicit brief and responsibility for securing enhancement and the sharing of good practice.
45 The University sees the development of skills as a source of enhancement of the student's learning experience and potential employability and has appointed a Skills Development Officer and Placements Coordinator within its Careers Advisory Service, funded through the Teaching Quality Enhancement Fund. A Lifelong Learning Strategy was informed by seeking the views of graduates and alumni. The SED also indicated that the University would continue work in other areas of longer standing, including the Teaching and Learning Strategy, support for Widening Participation, appointment of staff to support eLearning, support for the enhancement work of FLTDGs, and the establishment of a Progress Files Working Group. The University funds learning and teaching awards and the SED included examples of their impact on the development of good educational practice.
46 In December 2003, Senate approved several areas for significant future development, including rewarding staff for excellence in teaching and educational leadership, good practice in learning, teaching and assessment, implementation of progress files, liaison with Learning and Teaching Support Network centres, enhancing APR, external examining, and responding to external quality assurance initiatives.
47 Through its reading of the SED and other documentation and meetings with staff and students, the audit team found many current examples of enhancement activity and a considerable number of strategic policies designed to lead to enhancement. The team formed the view that the University's account of its commitment to enhancement was well justified. The University might wish to consider whether it might strengthen its approach by developing a means of securing an integrative overview of the various activities associated with enhancement.
Internal approval, monitoring and review processes
Programme approval
48 After the last audit, the University established an institutional approach to programme approval which involves a number of stages, with NPG as a key stage 'where detailed scrutiny of proposals takes place at University level'. Proposals progress from departmental committees to faculty undergraduate or graduate studies committees, then to faculty boards, thence to NPG. If approved by NPG they proceed to Education Committee, UPARC and Senate. The same process is used for significant changes to existing programmes and new mandatory and 'open units' which are available to students on other programmes. A template completed at department and faculty level ensures that the University format is observed, that standard questions are asked and that approval has been given at each level up to NPG. While at faculty level, a departmental representative is present to answer queries on the proposal, NPG's scrutiny is conducted without such representation. Programme specifications, with full details of how the programme addresses any relevant subject benchmark statements and where it sits within The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ), must be provided. Unit specifications with appropriate details of ILOs and associated assessment methods must accompany all proposals for new or changed units.
49 The mock institutional agreement for collaborative provision, while identifying the processes for approving an initial programme for validation, does not specify explicitly how new programmes or units will be approved within existing partnerships. The audit team established that when new or revised programmes or units are proposed by existing partners it is not compulsory that the home departments associated with the collaborative provision be involved, although in practice they are often consulted. The collaborative provider submits the proposal direct to the relevant faculty and the usual confirmation that library, computing and staffing resources are suitable is provided by the partner institution, not by University staff.
50 The SED considered that the 'approval process enable[d] serious consideration to be given to a programme proposal at every stage', ensuring 'coherence in learning, teaching and assessment methods' and that 'adequate resources [were] in place to deliver the programme'. The audit team concurred with the University's view of the process, finding extensive evidence of well-prepared proposals carefully scrutinised at all stages, with the NPG ensuring an effective institutional overview. In the case of collaborative provision, the lack of direct input from the University's home department or support services means that the University is relying on its other review processes to ensure that the proposal is properly resourced and in line with the policy and practice of the home department.
Annual monitoring
51 Since the academic year 2001-02, annual monitoring has been conducted through APR, which is subject to scrutiny by the FQATs through their annual consideration of departmental quality assurance practices. APR of the previous academic year's activity is completed by 31 December each year, with reports being sent by departments to the relevant FQAT and to the TSU. There is a standard template of issues and evidence to be considered during APR, which includes consideration of statistical information on admissions, progression and completion rates, provided by TSU; external examiners' reports; student evaluations; and the reports of any internal or external reviews, including those from professional, statutory and regulatory bodies (PSRB) from the year in question. The FQATs use the reports as the basis for their annual scrutiny, which also takes into account evidence gathered through meetings with staff and students. In addition, the FQATs consider a set of generic questions identified by the Plenary Group of FQAT Chairs, as well as any issues specific to their faculties. Although departmental practice may vary, most departments conduct APR as a single process for all their provision, or by groups of programmes, rather than singly: where departments collaborate in joint programme delivery then both departments consider the programme in their separate APRs.
52 University procedures require that collaborative provision be scrutinised through APR. Collaborative arrangments are not subject to FQAT visits but are monitored by the FQAT through the home department whose APR records include those of the collaborative partners.
53 The SED judged that 'the introduction of APR was important in strengthening the University's quality assurance framework' since several subject review reports had criticised the absence of 'formal opportunity' for departments to complete 'a written annual review' of programmes. It emphasised that 'this activity [was] currently in the early stages of development', and was subject to modification to 'improve clarity and to streamline the process', informed by the annual overviews of the process taken by TSU and Plenary Group of FQAT Chairs. FQATs were identified as a 'strong component of our internal review processes', enabling 'faculties to take responsibility for their own quality assurance', working within a 'common framework' but 'allowing each faculty sufficient flexibility for its own needs'. FQATs were described as having a dual role, both monitoring existing practice and 'encouraging the sharing of good practice', with 'a philosophy of continuous improvement underlying the FQATs' work'. Emphasis was laid on the 'involvement of practising academic staff in each faculty providing credibility and the necessary understanding of how quality assurance works at departmental level'; recent revisions, including the full membership of FQATs by a TSU representative, have reinforced the common framework of FQAT activity at the institutional level. The work of FQAT Chairs, who prepare faculty reports on their own FQAT activity and then meet as the Plenary Group of FQAT Chairs to consider these and other procedures, was commended by the SED.
54 The University recognises that it has some way to go before best practice is embedded throughout the University. By way of example, the audit team observed that the FQAT annual reports for the academic year 2002-03, while reporting considerable variability in practice in several areas of assessment, did not in general make related recommendations for action in these areas. The FQAT to the medical provision appeared to the team to be noteworthy in its very firm, and occasionally critical, commentary on variability in practice.
55 The audit team concluded that the APR process had established itself as a core feature of annual monitoring. Departments had been careful to collect and consider the required information and data, in particular external examiners' reports and the results of student evaluations. There was variation in the use made of the statistical data supplied by TSU: some departments conducted very thorough analyses of the material, but others were more cursory in their approach. The TSU data were broken down by programme, but as most departments run only unit and not annual programme questionnaires, student evaluation considered in APR was at the departmental rather than the programme level in most cases, as were external examiners' reports. The process thus feeds effectively into both FQAT and periodic review of departments, but further consideration might be given to whether the approach to the specific experiences of students on joint degree programmes might be better served if the APR process required explicit consideration of the particular features of each programme, as well as those common across the department.
56 The APR process only covers taught programmes and not research degree programmes. The FQAT process can include meetings with research students and, in the academic year 2003-04, some of the generic issues identified related to research students, but research degrees did not form a major part of FQAT activity in previous years. In the areas covered by the DATS, only one FQAT met research students routinely, and prior to the academic year 2003-04, the generic topics identified were all specific to taught students. Even though questions about both research and taught postgraduate provision were included in all FQAT briefs for the academic year 2003-04, the majority of FQATs did not meet research students during their visits. The FQAT in one faculty did not consider research provision at all, on the grounds that the FQAT had focused on the postgraduate experience, although only for taught postgraduates, in the previous year. A change in approach will be necessary if FQATs are intended to monitor the quality of research postgraduate provision. In the evidence available to it, the audit team could not discern any systematic processes at departmental, faculty or institutional level for scrutiny of the collective experience of research students, either in terms of statistics or evaluation. Currently, the only material relevant to this area is the individual student's annual report form, which is considered by the head of department and the faculty's graduate dean, and reports on the upgrading, withdrawal or degree results of individual students. There is no annual process through which these data are generalised to monitor trends or identify issues for departmental or faculty consideration. The team advises the University to identify a means, through the APR and FQAT processes, to extend the annual monitoring process to research student provision in departments.
57 The audit team concluded that the longstanding work of the FQATs was effective both in monitoring and in identifying and sharing good practice in the University's taught programmes. There was clear evidence of close scrutiny of APR and other documentation by FQATs, checked against the evidence of their meetings with staff and students, and providing full reports to both faculty and institutional bodies. There were differences between FQATs in how they interpreted their role and how well their reports were structured in addressing the core questions, with some variation in how far they felt bound to pursue the generic questions: in the academic year 2002-03, one faculty decided to focus entirely on postgraduate taught provision and ignore the generic agenda. There was also a tension between the role of FQATS in monitoring departmental observance of institutional procedures and in promoting an enhancement agenda, but the team concurred with the University's view that, on balance, the collegial basis of the FQAT process was an effective one in ensuring gradual adherence to University expectations in quality assurance, while making sure that specific faculty or departmental needs were properly addressed. The work of the FQAT chairs in analysing common trends and issues across the faculties, feeding these into institutional level discussions through their reports to Education Committee and other groups, ensured a strong institutional overview of the outcomes of APR, strengthened by the practice of FQAT teams in revisiting, the following year, the outcomes at departmental level of issues raised in the previous round.
Periodic review
58 Periodic review is undertaken through departmental reviews. When the process was first established, it was intended that departments be subject to review every five years, but in practice it has taken 10 years to cover all departments in the first round which was nearing completion at the time of the audit. A quinquennial cycle of review has been agreed for the future by UPARC, which has responsibility for the operation of the reviews and the monitoring of the outcomes. Departments prepare a SED which is checked by the TSU for its evaluative and analytical qualities before it is accepted as the basis of the review. Data in support of the review include those available from APR processes supplemented by, for example, data on research students which departments provide. The review panel includes at least two external reviewers, and they are asked to ensure that the provision meets national standards and expectations in the subject. The review process itself includes visits to the department and meetings with a range of staff and students, including research students as well as those on taught programmes. The review report is relatively brief but includes an evaluation of all aspects of the department's work and recommendations, divided into those aimed at the department, faculty and university. UPARC receives the report directly and meets the head of department to consider the recommendations, before writing to the head to confirm formally the actions to be taken. Progress reports on actions taken are required after a year from all those to whom recommendations are made. Departmental review papers are available to Senate and can be discussed there.
59 The SED identified departmental review as 'one of the most important aspects of the University's internal quality assurance processes', and staff whom the audit team met confirmed that it had acquired additional significance with the cessation of external subject reviews. Apart from 'the benefits of self-reflection and analysis' for the department preparing for the review, 'tangible and beneficial changes' arising from reviews include improvements in 'learning, teaching and assessment practice' and in the infrastructure, and adjustments to staffing profiles. The SED explained that UPARC's control of the process was appropriate because 'recommendations normally cover[ed] academic and resource issues and some propose[d] action at faculty or University level'. Annual monitoring of the review process itself by the Pro Vice-Chancellors and TSU staff has led to incremental improvements in review practice, focussing recently on the need to improve follow-up processes and to assist departments in implementing recommendations successfully.
60 The audit team concluded that the departmental review process was conducted with thoroughness. The team viewed a substantial number of SEDs prepared for departmental review and, although they differed in length and format, each provided a serious and comprehensive review of all aspects of a department's activity, including a proper coverage of teaching and learning issues in general, and quality assurance issues in particular. Review reports indicated that the panels had looked carefully at these aspects of the department and in most cases a substantial number of recommendations were made in these areas, as well as in research, staffing and other topics. The team noted that departmental review was intended to cover the experiences and needs of research students but also that the data provided by the departments on their research students were often rudimentary: one example only reported progression rates for cohorts over the last three years, too short a period to identify completion rate issues. UPARC itself clearly takes the review process very seriously and its direct involvement gives the process a major significance. It appeared to the team that UPARC's discussion of review reports focused on resource and staffing issues, rather than recommendations relating to teaching and learning or quality assurance, and that not all the follow-up reports addressed the full range of recommendations, concentrating on those for which UPARC had required specific action. The team therefore supports the University in its intention to improve the follow-up processes and invites it to consider whether a body other than UPARC might take specific responsibility for ensuring that the review process has successfully addressed issues directly related to teaching and learning and quality assurance.
61 Overall, the audit team concluded that the University's systems for approval, monitoring and review were in alignment with the relevant section of the Code of practice and were fit for purpose and operating as intended.
External participation in internal review processes
62 External participation in programme approval is obtained through the requirement for comment on the proposal from two external advisers, only one of whom may be an external examiner. There is no standard template for the comments from the external advisers; departments are required to respond to the comments. There is no direct external input into annual monitoring, except for the consideration during APR and by FQATs of external examiners' reports and reports of any external reviews during the year in question. The panel for periodic departmental review includes at least two external reviewers, normally academic experts from another institution but sometimes from appropriate professional bodies, with expertise that meets a set of guidelines, including both research and teaching expertise in the discipline. The external members of the review panel receive a comprehensive set of guidance notes, which asks them to confirm that the provision meets national standards and expectations in the subject.
63 The SED did not evaluate the effectiveness of such external participation. It did note the 'useful input' from industrial advisory boards in some science and engineering departments but the boards are not routinely involved in internal review.
64 The audit team concluded that, while external input was required for programme approval, it was not intrinsic to the process: the absence of a template for external responses allowed for relatively brief and informal comments. When the requisite external comments had not been obtained prior to consideration by the faculty or NPG, proposals had still been approved provisionally, subject to the necessary external advice being obtained. Even though the team was assured that, in such cases, the relevant Chair would ensure that any issues raised were addressed, it concluded that external approval was seen as a necessary condition for programme approval, but not a major source of advice or external reference in programme design. External participation is much more embedded in periodic departmental review, where strong and scrupulous use is made of the external advisors.
External examiners and their reports
65 The SED stated that 'external examiners [were] fundamental to enable the University to benchmark and maintain its academic standards'. It further commented that its external examining system '[had] been under constant scrutiny during the past ten years' and that 'the emergence of Teaching Quality Information (TQI) requirements [had] made it even more important that we are clear about how the system operates and that it [was] transparent to all involved'. Appointment of external examiners must conform to the University Guidelines for External Examining of Taught Programmes which were revised following the publication of the Code of practice, Section 4: External examining and came into effect from the academic year 2002-03. A new version of the external examiner's report form, containing a section intended for publication in line with TQI requirements, had been approved immediately prior to the audit. Departmental nominations, supported by the nominees' curricula vitae or equivalent information, are submitted to faculty boards for approval. The external examiner receives a pack of information from the Examinations Office, which includes the Guidelines and a copy of the outgoing external examiner's most recent report. External examiners are reappointed annually. The external examiner attends the Board of Examiners' meeting and gives an oral report at that stage, which is followed by a written, annual report addressed to the Vice-Chancellor.
66 Reports then go to the Pro Vice-Chancellor (Education) and the TSU, which distributes them to 'all interested parties' as defined in the Guidelines. Follow up of recommendations in the reports is the normal responsibility of individuals in academic departments, such as programme directors, and the head of department writes annually to individual external examiners to inform them about progress in implementing their recommendations, a process said in the SED to have been 'made formal in the last two years'. The audit team confirmed that such follow-up does occur and is monitored by the TSU.
67 External examiners' reports form part of the material required for APR (see paragraphs 51 to 55). FQATs are responsible for ensuring that reports 'are processed and responded to appropriately in a timely manner'. The audit team observed that comment on any delay or failure in the submission of reports is noted in FQAT annual reports. Guidance to departments and FQATs on the role and handling of external examiners' reports appears in the Handbook.
68 The Annual Reviews of External Examiners' Reports for the academic years 2000-01, 2001-02 and 2002-03 were undertaken by the Plenary Group of FQAT Chairs. The report for the academic year 2002-03 noted that external examiners' reports had been received for all undergraduate programmes 'with a few identified exceptions', and that others outstanding related to taught postgraduate programmes, which finished three months later in the academic year. The report further noted that 'the vast majority of the comments were extremely positive and where constructive criticisms [were] given, these [were] against a background of a secure learning environment for students'. The audit team can confirm that the Annual Review for the academic year 2002-03 was thorough and analytical, covering concerns raised by external examiners and a range of identified good practice. Lists of 'continuing' and 'new' issues were included, with University-level responses: these issues comprised assessment methods; degree classification, where the Working Group of Undergraduate Deans recommended a move to common classification bandings; the purpose and use of the oral examination timetable; staffing; and the new external examiner's forms. The team was satisfied that appropriate action had been, or was being taken, on all these items, and viewed favourably the rigour, comprehensiveness and general effectiveness of the 'Annual Overview' process; while some issues raised in the academic years 2000-01 and 2001-02 persisted, for example, degree classification and oral examinations, others, for example, pass marks in masters examinations and the use of wider ranges of marks had been largely resolved.
69 The audit team read a number of external examiners' reports representing the range of the University's provision, and can confirm that the majority were complimentary about the assessment process, the curriculum, teaching methods, the standards of the awards and the level of student achievement; the latter two judgements now appear in the section of the external examiner reports destined for publication, according to the TQI requirements. Its reading of the reports of the Plenary Group of FQAT Chairs, and the reports of FQAT visits to departments, led the team to conclude that the formulation of an overview of the external examination process by the FQAT Chairs represented an example of good practice.
External reference points
70 According to the SED, the University's 'approach to the Code of Practice has been to consider each section in turn, as soon after it is published as possible'. The process of mapping the University's policy and practice against the precepts of the Code of practice had, until its dissolution in November 2003, been undertaken by the Learning and Teaching Group. This task has now been assigned to the Plenary Group of FQAT Chairs 'because of the FQATs' close involvement with faculty practice and University policies'. Prior to the Plenary Group's consideration, the TSU conducts a 'gap analysis', of which a documented example in relation to the section of the Code on programme approval, monitoring and review was provided for the audit team.
71 The SED provided a comprehensive table summarising how the University had addressed or was in the process of addressing issues raised by the gap analysis. Reading of the responses listed in the table indicated that the University had in place documented policies relating to most of the provisions of the Code of practice, some of long standing, in relation to external examining, for example, and that varying degrees of revision and updating had been required to achieve alignment with the Code, some of which had been completed recently.
72 While modifications to the University's policies and procedures have achieved broad adherence to the Code of practice, the audit team found instances where practice deviated from the precepts and guidance of the Code. The University is not yet in alignment with the sections of the Code on collaborative provision (see paragraphs 128 to 135) and postgraduate research programmes (see paragraphs 88, 89, 91, 95, 119, 120 and 123). Although the Guidelines on the quality assurance of distance learning (the Guidelines) are not part of the Code, the team also noted in this context that the University had not yet developed specific guidance on distance learning, despite its having a well-established programme delivered by distance learning (see paragraphs 104 to 107).
73 The University regards the FHEQ as fitting well with its modular programmes at undergraduate and postgraduate level; it has recently completed modularisation of taught postgraduate programmes and 'used the FHEQ to guide the transition to modular awards'. The SED continued 'it has helped to iron out inconsistencies and, together with the modular framework and the introduction of programme specifications, has enabled the University to check that award titles match the level of the qualification and the programme structure'. The levels specified in the FHEQ have replaced the University's former nomenclature. The audit team was satisfied that the University, through the TSU, was taking a positive and dynamic stance towards the FHEQ in the design of its programmes. At the time of the audit, the University was introducing a credit framework on which it had 'consulted widely internally'; Senate approved the framework in June 2003 with the proviso that it would be tested during the academic year 2003-04 with difficulties highlighted to Education Committee for any necessary action. The team noted that certain programmes in the Faculty of Science breached the University's credit framework to an extent prompting, according to FQAT visit reports, requests from the departments concerned that the programmes concerned stand outside the framework. Of particular note was FM11 (Chemistry and Law), which was described in the programme specification as a BSc (Honours) degree. This is based on two years of chemistry, followed by two years in the Law School, the final year of which contains 80 credits of H-level Law. However, no H-level chemistry is experienced by the student and the audit team considers therefore that this programme diverges from the FHEQ. Accordingly, while noting ongoing discussion in the University on this matter, the team recommends that the University review its approach to qualifications titles for programmes of study involving more than one discipline in the context of the guidance in the FHEQ to ensure that such titles represent accurately the balance of the components in both level and volume of study.
74 Programme approval and review processes were identified by the University as the areas where it principally tested the compatibility of its programmes with subject benchmarks, although the SED also noted that these were 'occasionally... referred to by external examiners'. It was anticipated that benchmarks would be used more widely as programmes and the benchmarks themselves were updated. The SED also commented that 'some of the benchmarks [were] more useful than others', the less helpful ones being described as lacking subject-specificity. External examiners are required to certify that the programmes which they cover are of a standard defined by the benchmarks.
75 By contrast, the University finds the development of programme specifications 'useful' in providing a summary of information about a programme 'that might be available in different places but has not previously been collated'. Departments are supported in formulating programme specifications by means of 'several templates', notes of guidance and direct assistance from the TSU. Programme specifications are required for all proposals for new or revised taught programmes and must be formally approved by the relevant faculty board. The University anticipates that all programme specifications will be completed by the end of the academic year 2003-04. The SED highlighted questions that had arisen during the preparation of programme specifications, such as whether the aims and learning outcomes of core and mandatory units attached to each programme supported the programme aims and learning outcomes, and how could programme specifications be drafted in order to be appreciated for their various audiences. There was also debate about the approach to producing programme specifications for joint programmes. The University stated that such questions would be addressed once all programme specifications were in place.
76 The audit team read a number of programme specifications and found that those which had been completed were in line with national guidelines. As the University completes programme specifications for all its programmes, it should ensure that the system for awarding University credits takes appropriate account of the interrelationship between unit ILOs and programme ILOs. In the case of unit specifications, it was clear that new or revised unit specifications were required to be produced according to an appropriate template but no timetable for completion had been established. It was also clear that many of the existing unit specifications lacked statements of ILOs and/or assessment methods. The team would, therefore, recommend that the University completec full unit specifications for existing units as soon as is practicable, ensuring that both learning outcomes and the associated assessment methods are specified.
77 In summary, the audit team views the engagement of the University with both the Code of practice and FHEQ as satisfactory but requiring attention in certain aspects, particularly in relation to the sections of the Code on collaborative provision and postgraduate research students. The University is making progress towards having programme specifications in place for all of its taught programmes but as the work is still at a developmental stage the team is only able to confirm that the University's overall approach in this area is appropriate.
Programme-level review and accreditation by external agencies
78 The period since the previous audit saw the complete cycle of Agency subject reviews. The SED offered a thoughtful analysis of the reasons for some aspects having acquired lower averages than others, noting that the University had identified the areas of curriculum design, content and organisation and quality management and enhancement as areas that needed to be strengthened. Weakness in quality management and enhancement was attributed to a failure to reflect institutional-level quality assurance procedures at departmental level, to which the University had responded through FQATs and the TSU 'raising awareness of University policy and working with departments in implementing it'. In relation to the curricula, the SED stated that the University had judged that the small number of lower scores were 'partly as a result of continuing to take a traditional approach to teaching and learning in some subjects (which [it did] not necessarily think [was] detrimental to the education provided) and in some cases a failure to ensure that the curriculum [was] designed to achieve the intended learning outcomes'. The SED also identified changes introduced at the institutional level as a result of subject review visits, highlighting the introduction of APR 'for more formal tracking of programmes annually'. Subject review reports were considered by FQATs within faculties and by the Plenary Group of FQAT Chairs institutionally, and also fed into departmental review: such reports are considered both in APR and periodic departmental review, along with any reports or recommendations from professional bodies.
79 The audit team reviewed the responses to subject review, both institutionally and in the DATs. It concluded that both the institution and the specific disciplines had responded appropriately to the areas of criticism, although institutional progress had been relatively slow in the area of assuring departmental adherence to University policy.
80 The SED included a listing of the University's programmes that are regulated by PSRBs. Accreditation visits are 'managed at faculty and department/school level with input from the TSU as requested'. The audit team saw ample evidence of appropriate and timely responses to reports from PSRBs at departmental level, both directly and in how these were monitored by APR and FQAT processes. It was less sure that the University had systematic mechanisms at the institutional level for ensuring that generic themes from such reports were identified and addressed, since University involvement with PSRBs, while it might involve the Vice-Chancellor or Pro Vice-Chancellors, was not the specific responsibility of the Plenary Group of FQAT Chairs nor of Education Committee or any of its subcommittees. As in many other areas, the TSU acted as the main body maintaining an institutional overview of all these processes.
Student representation at operational and institutional level
81 According to the SED, students are represented at all three levels of decision making within the University: at University level, on a range of committees including Council, Senate, and Education Committee; at faculty level, on faculty boards and undergraduate and graduate studies committees and on education committees in the Faculties of Medicine and Dentistry and Medical and Veterinary Sciences; and at departmental level on the student liaison committees, through attendance at departmental meetings and through the programme representative system. At the departmental and faculty levels, programme representatives are elected at the beginning of the year to sit on the student liaison committee or other departmental committees. At University level, membership is usually taken up by sabbatical officers of the SU.
82 The SED did not comment on the effectiveness of these arrangements beyond stating that, together with the SU, the University tried 'to ensure that students [had] ample opportunities to make their views known and influence developments at various levels in the University'. It went on to say that 'departments and faculties, in collaboration with the Union, make every effort to ensure that the course representatives are elected at the beginning of each academic year'.
83 The SWS confirmed the effectiveness of representation at the University level, but drew attention to difficulties with the course representative system which, in the view of the SU, was almost defunct in the academic year 2002-03 but had been revived for the academic year 2003-04. The SED acknowledged the contribution of the training programme for course representatives offered by the SU to the restitution of course representation. SU Officers considered that, at the institutional level, the Joint Committee of Sabbatical Officers of the Union and members of the Council was a particularly effective arena for discussion of areas of interest to the student body.
84 In meetings with students at the institutional and discipline levels, the audit team heard of considerable variety in the arrangements for student representation at departmental level. Not all departments operate Student/Staff Liaison Committees (SSLC), with some relying on student societies instead as a mechanism for student representation; in some departments only undergraduate students are represented on the SSLC while in others the SSLC is the only forum for student representation for all levels. In other departments, for example Chemistry, postgraduate students, while excluded from the SSLC, are represented on a range of other departmental committees and can bring forward issues to the Chemistry Graduate School Committee (GSC). At the time of the audit, there were representatives from Dentistry on the Dental Education Committee, the FQAT and Senate as well as the departmental SSLC. In Engineering Mathematics there is not only an established SSLC but also an annual Forum for each year group with an agenda set by student representatives. This was greatly appreciated by the students and deemed by the team to be a feature worthy of note.
85 As in other areas, the University's emphasis on guidance rather than regulation and avoidance of central prescription gives rise to diversity of practice. Staff whom the audit team met stated that the University's guidelines for establishing and operating student liaison committees were very brief, the most important guideline being that staff membership should not exceed student membership. The extent to which SSLCs were student-driven also varied between departments. There was a range of methods of disseminating committee minutes including email and posting on notice-boards. The team concluded that this diversity of approach towards student representation seemed to work in practice, with students reporting that they were able to express their views, which were taken seriously by the University, and that the arrangements allowed them to have an input into decision-making processes at the operational and institutional level.
Feedback from students, graduates and employers
86 The SED stated that, although departments were free to determine for themselves the most appropriate way of obtaining student feedback 'it [was] a requirement that this occur[ed] and [was] documented, monitored through APR and FQAT activities'. Strategies for securing feedback must draw on advice provided in the University Guidelines for Direct Feedback from Students. In addition to the feedback gained through the student representative system, the most common mechanism for eliciting student feedback was through end of unit questionnaires, the results of which were fed into APR and supplemented at the FQAT visit stage by direct input through meeting a representative sample of students across years and programmes without a member of staff being present. Students also meet panels during internal departmental reviews. Feedback to students of the results that emerge from analysis of questionnaires is communicated through a variety of means, including lectures, web sites, the virtual learning environment (VLE) and through the student representation system.
87 The SED did not include direct comment on the overall effectiveness of arrangements for collecting feedback from students, but did emphasise that the University had guidelines for obtaining direct feedback from students since 1994, indicating that this demonstrated that the University had taken the need for such feedback seriously for at least a decade. The SED also highlighted the participation of the University in a HEFCE-funded project on 'Collecting and Using Student Feedback on Quality and Standards of Learning in Teaching and Higher Education' which resulted in details of the University's Guidelines for Direct Feedback from Students being included in the resulting publication 'Collecting and Using Student Feedback-a guide to good practice'. From meetings with students and staff, the team concluded that the practice of obtaining student feedback from taught students through unit questionnaires was widespread. Furthermore, from its scrutiny of documents and discussions with taught students, the audit team concluded that there was a high degree of satisfaction with teaching; this corroborated a recent survey by the SU which found that 88 per cent of undergraduate students were satisfied, or very satisfied, with the teaching they received.
88 The University became aware through a report that it commissioned on the postgraduate experience at the University that less than 60 per cent of the research students who responded to the survey felt 'there [was] an appropriate and accessible channel via which to make comments regarding academic support and receive feedback'. The report made some 211 recommendations covering academic support; information and induction; skills training and career development; language and learning culture; postgraduates as teachers and demonstrators, facilities support and information services; residential accommodation; and representation and feedback mechanisms. The latter, in particular, prompted recommendations in the report on the need for the University to undertake regular research into the needs and wants of the postgraduate student body, the range of feedback mechanisms, the publication of responses to feedback, the setting up of focus groups, the establishment of anonymous feedback mechanisms as a means to research postgraduate student satisfaction and the provision to students of a summary of the results of any questionnaire they complete. Such concerns of the research postgraduate students were echoed to the audit team in its meeting with a group of such students, during which students also reported a need for better induction at both University and departmental level. In addition, the team heard of wide variation in both the existence and effectiveness of research student representation at departmental level.
89 The audit team acknowledges that the University commissioned the report on its postgraduate provision and welcomes the fact that the findings of the report are in the process of being considered at its major committees. The findings of the report concur with the team's view of the postgraduate research student experience at the University, which draws on the evidence of meetings with the postgraduate research students and its reading of documentation. Accordingly, the team advises the University, as a matter of priority, to review its arrangements for supporting postgraduate research students, establishing a systematic approach to annual review of the research postgraduate learning experience and maintaining an appropriate central overview of the collective learning experience and progress of research students.
90 The University's main contact with graduates and employers is through a range of activities organised by the Careers Advisory Service, including workshops for graduates, employer visits, special projects and through industrial advisory boards. The Careers Advisory Service uses employer feedback to inform its programme of workshops and other activities. In addition, departments with vocationally-orientated programmes maintain an ongoing dialogue with PSRBs. The University has also obtained detailed feedback from graduates through a Life Long Learning project, involving graduates and alumni, carried out by the Careers Advisory Service. According to the SED this project has provided the University with 'a rich data source' that will inform its Education Strategy.
91 The audit team concluded that the account given in the SED of the approach to gathering feedback from taught students, graduates and employers was accurate. The account of support offered to postgraduate research students was less representative of the situation as perceived by the team and evidenced through documentation and meetings. Student feedback gained through APR, FQAT visits and departmental review offers a direct means for students to make a positive contribution to the assurance of quality and standards across the University's provision.
Progression and completion statistics
92 The Student Systems and Information Office is responsible for producing some management statistics from the University's student records system. In addition, the TSU produces summary intake, progression and completion statistics for all departments in preparation for APR and departmental review. The University has clear statements of the statistics to be considered at the various review events.
93 The statistical set to be considered at APR comprises intake, progression and exit data for the current and previous academic years. Intake data include details of gender, school background (state or independent), ethnicity and GCE A-level points score. These data cover both undergraduate and taught postgraduate provision. The data set to be considered at departmental review is an extension to that considered by the APR. In addition to the APR data set, consideration is also required of first destination survey statistics and intake, and progression and exit data for research postgraduates. Admissions data, including the ratio of offers to applications, are also considered. The University did not reflect in the SED on the effectiveness of the considerations given to these data at APR and departmental review. The SED did comment that the completion profile for undergraduates reflected the care with which it selected, supported and assessed students.
94 The audit team reviewed a range of APR reports which demonstrated variation in the thoroughness of debate on the data set: in some cases there was no record of consideration of the data; but other reports included both the required data sets and a record of thorough consideration of the information. Departmental review reports that the team examined showed a similar variation: some contained no references to the data set or information supplied while others showed clear consideration of this information. The team concluded that the receipt and consideration of the required data sets were not yet universally embedded in the APR and departmental review processes.
95 In meetings with staff, the audit team learnt that the University's central records system is not currently capable of monitoring the progression of postgraduate research students and that the timeliness of recording completion data could be improved. The team was informed that central collection of progression and completion data on research students was generally problematic and that each individual faculty held only records of the current registration status of such students. It appeared to the team, therefore, that the University had no centrally administered overview of progression and completion rates of research students across the institution. The team recommends that the University give urgent consideration to the establishment of a system which would allow it to maintain a central overview of the progress of its research students in support of its assurance of quality and standards in this area.
Assurance of quality of teaching staff, appointment, appraisal and reward
96 The University has a wide-ranging, established People Strategy (2002), which is complemented by relevant sections from the evolving Education Strategy and the current review of the University's Reward Strategy. The University's Research Strategy (2002) confirms the intention only to appoint academics who have achieved or have the potential to achieve an international level in research, but it is intended that this is balanced by 'establishing dedicated career opportunities for staff who choose to focus principally on the education function...' Committees relevant to staff appointment, appraisal and reward are the Staff Committee, the Equal Opportunities Advisory Group and the faculty and departmental Promotions Committees.
97 The staff appointment system is described in the University's Guidelines on the Recruitment and Selection of Staff and is backed up by training courses for staff involved in the process, which includes compulsory training for all members of interview panels. The recruitment, selection and interview process complies with recognised good practice and the Guidelines. It includes a requirement for all short-listed candidates for permanent teaching posts, including professorships, to give a presentation, but allows faculties to devise criteria for judging the presentations that are relevant to their specialties. The Teaching and Learning Guidelines (1995) also emphasise that candidates should be evaluated both on their teaching skills and their research performance. Appraisal of all academic staff is through an annual staff review and development (SRD) process which is being extended to include administrative staff in academic departments and staff in support services departments. In meetings with staff, the audit team was told that more than 14 support services departments and two associate academic departments had achieved Investors in People accreditation.
98 The Education Strategy, which at the time of the audit was in the final stages of approval, has 10 objectives, the third of which is 'to reward staff for excellence in teaching and for supporting the learning experience of a diverse student group'. This has two action points: to explore the possibility of introducing a new, senior role equivalent to professor for educational leadership, pedagogic research and scholarship; and to acknowledge achievement in teaching through University teaching prizes. Currently, there is a range of Learning and Teaching Awards, which cover the categories of individual Fellowships, support for departmental activities recommended in departmental review, project support and new learning and teaching initiatives. There are also University Teaching and Learning Prizes funded through HEFCE Teaching Quality Enhancement Funds.
99 Meetings with staff at institutional and subject level confirmed that the arrangements for recruitment, selection, appointment and promotion were perceived as being sound. Staff confirmed their involvement in and the value of the SRD process. It was widely recognised that the University was intending to recognise fully the contribution of teaching excellence in the promotion process. The audit team considered assurance of the quality of teaching staff (through the appointment, appraisal and reward system) was appropriate and highlighted as good practice the resolve of the senior staff in the University to recognise and reward teaching excellence in a research-intensive institution.
Assurance of quality of teaching through staff support and development
100 Staff support and development encompass a wide range of activities that aim to address the complex needs of the academic and administrative staff at different stages of their careers. The programme of activities is described in the institution-wide 'Course Diary' and is informed by reports from heads of departments summarising the outcomes from the staff reviews. The elements most relevant to the quality of teaching are the Teaching and Learning in Higher Education (TLHE) programme, based in the Graduate School of Education, and a series of workshops run by the Learning Technology Support Service.
101 The TLHE programme, established in 1994, is funded through Staff Development and is accountable to the Staff Development Committee and to the Education Committee. Successful completion of the programme leads to the award of Advanced Certificate in Education. The TLHE programme is compulsory for all new appointees without prior experience or accreditation. The programme is subject to annual monitoring and is a requirement for confirmation of satisfactory completion of the probationary period. The probationer is supported by a mentor for the programme and will also have a probationary adviser who can be the same person as the mentor. The University stated that it required postgraduate students involved in teaching and related duties to undertake some form of training provided by the University, faculty or department; the TLHE programme is not compulsory for students and contract research staff undertaking such duties. Some of the research students whom the audit team met had received little structured training and others who were involved in marking had received only limited guidance on the task. The Education Committee receives an annual report on the enrolment, progression and completion of new staff, subdivided into permanent and contract staff but with no comment on postgraduate students. Some faculties and departments also offer their own training programmes and other forms of staff development. University training programmes are mandatory for new admissions tutors and existing tutors every three years. It is being considered whether a similar compulsory training programme should be introduced for personal tutors.
102 Peer observation of teaching is not compulsory, although it is encouraged, and this has led to diversity of approach between departments. The Teaching and Learning Guidelines (1995) advise that regular peer review should be encouraged and monitored by heads of department with the recommendation that this should be a mutual peer observation between pairs of staff. The University identified 'the nature of the peer observation process within the departments' as the highest priority theme for the 2003-04 schedule of the Plenary Group of FQAT Chairs. The DATs illustrated the variation of practice around peer observation and review. The audit team formed the view that the system in dentistry was an example of good practice that should be highlighted as it involved thorough and systematic peer review, which operated on a three-year cycle which allows clinical, tutorial and lecture teaching sessions to be assessed in sequential years. This is conducted against a structured template and the completed forms are seen by the unit organiser and head of teaching division and are available at unit review and annual programme review. A similar system is in place in the Politics Department. The Department of Theology and Religious Studies adopted a 'Quality Circle' process in which the observation was informal and the feedback was oral and confidential, Engineering Mathematics Department have a mandatory system but with informal, confidential feedback; there is no system for peer observation in the School of Chemistry.
103 The audit team recognised the value of the TLHE programme in supporting staff without previous teaching experience. The University may wish to consider making some form of staff development in teaching and learning training compulsory for postgraduate students involved in teaching, demonstrating and marking undergraduate assessment. The team endorsed the priority given to reviewing the peer observation process in the activities of the Plenary Group of FQAT Chairs. Overall, the team concluded that the University's approach to assurance of the quality of teaching staff through staff support and development was appropriate and operating as intended.
Assurance of quality of teaching delivered through distributed and distance methods
104 In January 2004, the University published the terms of reference for its newly established eLearning Systems Project Board which reports to the Information Services and Systems Committee. The document noted that many departments in the University had already purchased a variety of relevant software and that many continuing professional development (CPD) courses were already delivered through a VLE. One aim of the Board is to develop a strategy and make recommendations which will rationalise and optimise provision, particularly as further collaborative projects regionally and internationally are being planned. The documentation identifies e-Learning as having the potential to enhance student learning consistent with the Education Strategy's strategic goals. Among the issues overseen by the new Board will be VLEs, on-line assessment and tutorial authoring software. The terms of reference for the Board and its authority are primarily concerned with functional, technical and financial issues and while it is responsible for ensuring that its recommendations are consistent with University policies it does not have an explicit remit for quality assurance issues relating to approval, monitoring and review of delivery through forms of e-Learning. The audit team was informed that the intention was that e-Learning be seen as a support tool that ideally could be embedded in all programmes and across a wide range of teaching contexts.
105 At the time of audit, the University had only one distance-learning programme, the MSc in Reproduction and Development. The University envisages developing its own guidelines on distance learning when the Guidelines and the Code of practice, Section 2: Collaborative provision have been revised and approved, at which time the University will also revise its existing Guidelines for Educational Partnerships.
106 The MSc in Reproduction and Development was offered for the first time in 2001 and is delivered to on-line learning students around the world in countries including China, India and the Philippines; at the time of the audit some 34 students were registered for the programme. The audit team was able to examine relevant documentation which demonstrated that the programme had been approved following all the University formal requirements, passing through a series of committees from department through to Senate, and that appropriate monitoring procedures had been applied, including student evaluation arising from both on-line feedback and a residential workshop. There was a detailed formal proposal, input from external advisers, a programme specification, a set of regulations and, more recently, an external examiner report and an extensive APR which had been considered at relevant committees. The operation of the programme represents a model of good practice in several respects.
107 It was not evident from the documentation whether the University had undertaken an appraisal of its practice against the Guidelines at the time of programme development. The audit team endorses the University's intention to prepare its own guidance on distance learning, and suggests that this be given priority as soon as the updated Agency guidance for this area is received. In so doing, the University might wish to consider securing information about all distance-learning activities across the University to allow it to draw upon the range of experience of e-Learning in defining its policies and procedures for teaching delivered through distributed and distance-learning methods.
Learning support resources
108 The Information Services Division is responsible for providing resources and support for learning and is also the locus for developing the relevant policies and plans. There is a broad range of initiatives including new strategies for e-Learning and ICT. Learning support resources include the libraries, the e-Learning resources and the associated staff in these areas and the Computer Centre, Learning Technology Support Service, Institute for Learning and Research Technology. Each academic department has a designated subject librarian and each faculty has an e-Learning adviser. Assessment of learning resource needs is a formal part of the process for approving new programmes and the changing needs of existing programmes are picked up in APR and fed in to the FQAT and departmental periodic review processes. The HEFCE Teaching Quality Enhancement Funds are used for a variety of initiatives that improve learning resources for students, including software developments and supporting staff time in the Graduate School of Education, the Careers Advisory Service and Learning Technology Support Service.
109 A Library and Information Resource Provision Strategy is currently under development and the University recognises the shift towards electronically-based library resources, as evidenced by the increase in computer-assisted learning and the development of a VLE which is used in about 500 units. The University provided a statistical comparison of library expenditure and book stock per full-time equivalent (FTE) student in a range of higher education (HE) institutions which demonstrated that the University was above average on both parameters in 2000-01 and 2001-02. The SED reported financial supplements to the periodicals and books budget in the academic years 2002-03 (£224,000) and 2003-04 (£125,000), which equated to an increase of 39 per cent between 1999 and 2003.
110 In documentation and meetings with students, the attention of the audit team was drawn to some issues associated with the quality of academic and non-academic support and facilities. Students whom the team met acknowledged that there were two major concerns but otherwise they were largely satisfied with the provision. Notwithstanding the financial supplements reported above (paragraph 109), there were concerns about the adequacy of the provision of copies of books in the Arts and Social Science Library and the computer availability and quality of software in one of the 24-hour computer rooms. Students appreciated the efforts of departmental staff in trying to address such deficiencies through personal loans of books and production of study packs. Several groups raised the inadequacy of the photocopying facilities. The team heard that a Library Student Liaison Group was formed in November 2003 to try to resolve some of the problems identified. Students whom the team met considered laboratory and information technology (IT) facilities to be good. Students appreciated the wide range of optional skills courses that were available and the work of the support services departments, including counselling and careers guidance.
111 The audit team considered that the support services' departments appeared to provide effective and appropriate services, which are appreciated by their clients. Monitoring of these departments is achieved through annual written reports that are received by University committees. There are no plans for having formal Service Level Agreements, although several of the departments have some form of engagement with external agencies, which is perceived as improving quality. The team suggests that the University consider the contribution that a systematic approach to reviewing support services' departments, comparable with the reviews of academic departments might make to the enhancement of provision on this area.
112 At the time of the audit, the University was undergoing many changes supported by a range of new strategies either recently produced or in progress; these included the Education Strategy, e-Learning Strategy, ICT Strategy, Library and Information Resources Provision Strategy and review of the Reward Strategy. Until recently there was no systematic approach to assessing educational infrastructure needs at departmental, faculty or institutional level but this is being addressed in the implementation of the University's Education Strategy which makes provision for a more structured approach to planning associated with learning resource and other educational infrastrucure needs. The general quality of library and IT facilities is satisfactory although, as noted, the Arts and Social Science Library attracted criticism from both staff and students. There are close links between the departments and their subject librarians and a range of local-level mechanisms for establishing library usage and user satisfaction.
113 The audit team recommends that, as planned through the implementation of its Education Strategy, the University pay particular attention to the systematic identification of learning resource and educational infrastructure needs. This should include a procedure for identifying and prioritising educational infrastructure needs and assuring itself that all students and teachers have access to appropriate computing facilities. Parity of access to learning resources for students in different disciplines should be key to the implementation of these actions, drawing on the provisions of the range of emerging strategies.
Academic guidance, support and supervision
114 The University's Student Recruitment and Admissions Office produces a range of publications designed to help students in the application process. Departments supplement these publications with programme-specific information. Senate has recently approved new Admissions Principles and Procedures for home and overseas undergraduates. The SED stated that the 'main aim of the University's admissions procedures [was] to select applicants with high academic potential and create a student body that [was] balanced and diverse in terms of background and experience...'. Academic support and guidance begin during the introductory week of the academic year, when all incoming students are provided with an information pack 'covering all aspects of their entry into the University'. There is a formal programme of activities at both departmental and institutional level in the introductory week.
115 Academic staff are the main source of guidance to students both in their role as teachers, and as personal tutors. As teachers, they have office hours, during which they are available for consultation by students who attend their lectures and seminar classes. It is up to students whether they avail themselves of this opportunity. In addition, all undergraduate and taught postgraduate students are allocated a personal tutor who is a member of the academic staff and acts as the first point of contact between the student and 'the University'. Joint honours students have a personal tutor in both departments.
116 The Notes of Guidance for Personal Tutors and their Tutees state that the role of the personal tutor is to offer both academic advice and pastoral support, which might involve referring students to other specialist support services. The Notes of Guidance, which are distributed to all staff and are in the Handbook, include a list of useful contacts for both academic staff and students. In addition to reviewing a tutee's academic progress and providing feedback and advice, particularly in reference to the choice of option units and units in other departments, the personal tutor is expected to attend meetings at which the academic progress of tutees is discussed.
117 Tutors are expected to see their tutees at least three times a year, more for first-year students, and on other occasions at the student's request. From discussions with students, it was clear that the frequency with which students saw their tutors was variable. Nonetheless, students find their tutors accessible; they make themselves available to see students as and when requested. The audit team concluded that lack of frequency of formal meetings did not necessarily entail lack of access. A survey conducted by the SU found that, although the frequency of formal meetings sometimes fell short of the guidance, 66 per cent of undergraduate students were satisfied with the frequency of meetings, while the corresponding proportion for taught postgraduates was 63 per cent.
118 On the basis of its discussions with students and its reading of documentation, including the SWS, the audit team concluded that there was a need to bring more consistency to bear on the personal tutor system; this accords with the University's own conclusion from student feedback that there was diversity of practice in approaches to personal tutoring. There were indications that a number of tutorial arrangments did not meet the requirement for three meetings a year with the onus being on students to take the initiative. There is a risk in reliance on such an informal approach that some students might not receive adequate academic guidance and support.
119 Research students have supervisors rather than personal tutors: discussions with research students established that there was variation in the frequency of meetings between student and supervisor and the provision of feedback despite specific regulatory requirements in these areas. The onus was often on students to arrange supervisory meetings and to maintain contact with their supervisory team.
120 From meetings with students at institutional and subject level the audit team concluded that the majority of students found staff to be both accessible and helpful. The diversity of approach to supervisory arrangements for research students reinforces the team's overall concern about the University's arrangements for supporting postgraduate research students (paragraphs 88, 89, 95, 119, 120, and 123). The University's formal requirements in the area of academic guidance, support and supervision observe the provisions of the relevant sections of the Code of practice, except in the provision of support for research students. The University may wish to consider how it might bring more consistency into the application of these formal requirements for academic guidance, support and supervision for all of its students in the interests of parity of approach across the institution.
Personal support and guidance
121 Personal support and guidance also begins at induction during the introductory week. All students receive the booklet Reception and Registration of New Undergraduates, which explains all the processes, involved in reception and registration, for example, student cards, email addresses, the student health service and accommodation. In addition, all students receive the student services booklet, which provides information on the range of services available, including Health Services, Access for Deaf and Disabled Students, Counselling, Accommodation, Advice Centre, Careers Advisory Service and International Student Advisory Service. The audit team heard of effective support given to students with disabilities.
122 The International Students Advisory Service, which has two international student welfare officers, organises a 'meet and greet' service for international students at major arrival points and, during the introductory week, mounts a two to three day programme for all international students. There is a 'drop in' welcome lounge for international students during the first two weeks. Thereafter the Service supports students throughout their time at the University. There is a social programme throughout the year, including a group for the spouses of international students.
123 The induction process for postgraduate research students was less well structured. Some research students whom the audit team met reported that they had not received an induction; this was particularly common for students who began their studies at a time other than the start of the academic year. A significant percentage of postgraduate research students met by the team had not received any formal induction by the end of their first week at the University.
124 Diagnostic tests during the introductory week identify students in need of additional English language support. Language Centre staff work closely with academic departments in the design and delivery of the remedial English language programme which runs throughout the academic year. Overseas postgraduate students would have appreciated more information on the English language support available to them.
125 As noted earlier, in their role as tutors academic staff have a pastoral as well as academic role and work closely with staff from the student services department to ensure that appropriate support is available to all students, including part-time students. The Counselling Service is available to all students on a self-referral basis, although academic staff may also refer students should they consider it appropriate. Tutors maintain contact with students through the VLE, through email and in discussion groups. The University is developing progress files to be accessible to students through the University Portal as a supplement to personal tutorial arrangments.
126 The University has adopted an innovative approach to the resolution of student complaints and appeals. It has established the post of Student Complaints Officer (SCO) and introduced a Mediation Service. The brief of the SCO is to listen to any student with a complaint or grievance and advise on the procedures that are open to those who are intent on taking their complaints further. Those who wish to do so can avail themselves of the Mediation Service, which was set up specifically to help resolve student disputes, and is staffed by four senior members of the academic staff trained in mediation. Since its introduction, the Mediation Service has had a marked impact in reducing the number of appeals to Council; the audit team considers the role of the Mediation Service in the prompt and effective resolution of a significant number of student complaints and appeals to be a particularly noteworthy feature of the University's approach in this area.
127 All units producing support services to students report annually to Senate and Council. From its discussions with students and staff, and its reading of the documentation made available to it, the audit team concluded that the personal support and guidance that is available to students is of a high standard and responsive to student needs. Provision in this area is in alignment with the guidance of the relevant sections of the Code of practice.
Collaborative provision
128 The University has seven educational partnerships, some at undergraduate and some at postgraduate level, involving 940 students. There are also two consortium arrangements to which the University contributes and awards a small number of masters level and taught doctoral programmes. Among the collaborative arrangments there are three overseas partnerships, one in Hong Kong and two in Italy (in Bologna and Florence). At the time of the audit, the University had withdrawn from a link with a further overseas partner because it was unable to secure appropriate quality assurance; a revised and limited Agreement had been issued to enable the University to offer the former partner consultancy advice on curricular and assessment matters.
129 The University first published its Guidelines for Educational Partnerships in 1994 and they have been updated on two occasions in part to ensure that the arrangements were in line with the Code of practice, Section 2: Collaborative provision. In December 2003 the Senate approved revised Guidelines which are generally consistent with the Code. The University has also developed a mock institutional agreement to govern the operation of educational partnerships. The SED stated that arrangements for securing quality and standards for partnerships were the same as for on-campus programmes. Management of collaborative provision is overseen by Education Committee, supported by FQATs. As indicated in paragraphs 133 and 135 actual practice is only recently catching up with University expectations.
130 The University's collaborative arrangement with MusicSpace in Bologna was the subject of an overseas audit by the Agency in 2003. The audit report identified a number of areas where the University's practice was inconsistent with the Code of practice and the University's own Guidelines. These included: a failure to apply the University's requirements for approval and validation of programmes offered in collaboration; deficiencies in the formal Agreement and arrangements for the stewardship of quality and standards, including monitoring and review of the operation of the programme; and irregularities in the appointment of an external examiner.
131 The SED stated that the report '[had] drawn our attention to the need to be more rigorous in monitoring collaborative arrangements'. In an appendix to the report, the University described the remedial actions taken in response to the findings of the report. It was clear from meetings with staff that the University had taken account of the findings of the report and taken action towards remedying the deficiencies identified. Education Committee convened the PRG to undertake a review of all the University's partnerships to ensure that all such arrangements conformed to the University's Guidelines. As a first step in this review, faculties were asked to ascertain whether there were in existence further collaborative arrangements of which the Education Committee was hitherto unaware. The University provided the audit team with a list of its partnerships with a further list of student numbers. The University is encouraged to maintain actively the register of collaborative partnerships and to find a means of combining and updating details of student numbers within the same list. The Review Group will report to Education Committee towards the end of the 2003-04 academic year.
132 From scrutiny of committee papers, the audit team noted that the NPG had identified irregularities in a collaborative arrangement for the delivery of an MA in Cultural Performance for which an agreement was signed before the arrangement had been considered by the Education Committee and the Senate. In relation to a further arrangement, in February 2004, the PRG minutes reported that 'there seemed to be some confusion about the partnership arrangements with all the partners involved, this required clarification and the programme needed clear guidance on how to move forward'.
133 The audit team met programme directors for the majority of the University's current collaborative arrangements. Discussion in that meeting established that the University now had a more strategic approach to the management of its collaborative provision. A policy to engage in partnerships only when they are of mutual benefit, for example, the addition of new or complementary strengths to the University, has been established. It was confirmed that all partnerships were now subject to the University's standard procedures for monitoring and review. It was clear that while the review of all partnerships was yet to be completed, the audit of the partnership with MusicSpace had been a learning experience for the University and that the University's own Guidelines were now fully understood and implemented. It was also noted that while there was now a model Agreement, adaptations could be made for specific circumstances. Programme directors were also alert to the risks inherent in over reliance on individuals in the operation of collaborative arrangments and aware of the need for succession arrangements.
134 The audit team considered that the mock institutional agreement was generally consistent with the advice provided in the Code of practice, Section 2: Collaborative provision. The team would wish to draw the University's attention to some areas where greater specificity would provide additional safeguards for the operation of its partnerships: the language to be used for assessment, including oral examinations should be specified; external examining arrangements should ensure that examiners have sufficient competence in the language of instruction and assessment; the agreement should include a statement on applicable law; and arrangements to protect the students' interests in the event of termination should be stipulated.
135 The audit team was able to conclude that the University was conscious of the need to have in place appropriate arrangements for approving and monitoring its collaborative partnerships. It has a set of recently updated guidelines for this purpose and a model Agreement. The report on the University's partnership with MusicSpace, Italy was a stimulus to a thorough review of all partnership arrangements. When the review of its collaborative arrangements is complete and appropriate recommendations are received and implemented by the Education Committee the University should be able to justify fully its claim in the SED that all programmes '...delivered through such partnerships are subject to the same quality assurance procedures as other programmes'. The University plans to update its Guidelines in the light of the forthcoming publication of a revised Code of practice, Section 2: Collaborative provision. The University is advised to review its Guidelines and the content of Agreements to ensure that responsibilities for the management of quality and standards in its collaborative provision are clearly defined, particularly where they apply to overseas partners.
Section 3: The audit investigations: discipline audit trails
Discipline audit trails
136 In each of the selected DATs, appropriate members of the audit team met staff and students to discuss the programmes, studied a sample of assessed student work, saw examples of learning resource materials, and studied annual module and programme reports and periodic school reviews relating to the programmes. Their findings in respect of the academic standards of awards are as follows.
Chemistry
137 The DAT covered the following programmes:
- F100 BSc Honours Chemistry (a three-year programme);
- F103 MSci Honours Chemistry (a four-year programme for those studying chemistry in more depth and to a level appropriate for entry to a PhD programme);
- F104 MSci Honours Chemistry with Study in Continental Europe (a four-year programme in which the third year is spent studying chemistry at a Continental European university);
- F105 MSci Honours Chemistry with Industrial Experience (a four-year programme in which the third year is spent in industry);
- F107 MSci Honours Chemistry with Study in North America (a four-year programme in which the third year is spent studying chemistry at a North American university);
- F108 BSc Honours Chemistry with a Preliminary Year of Study (a four-year programme including the preliminary year). The F108 programme was started in 1995 with the specific aim of enabling access to HE for mature students and students who had taken GCE A-levels inappropriate for direct entry to the other programmes. The first students to take the programme were registered in 1997;
- two exit BSc programmes for students who do not, at the end of their second year, reach the standard required to continue on the F104 and F105 programmes respectively. Both of these programmes were in their final year of operation;
- F101 BSc Honours Chemistry with Study in Continental Europe (a four-year programme in which the third year is spent studying chemistry at a Continental European university);
- F102 BSc Honours Chemistry with Industrial Experience (a four-year programme in which the third year is spent in industry).
138 The DAT also covered the chemistry element of the following programmes offered jointly with other departments:
- three chemical physics programmes (with the Department of Physics);
- F320 BSc Honours Chemical Physics (a three-year programme);
- F331/322 MSci Honours Chemical Physics (a four-year programme for those studying chemical physics in more depth and to a level appropriate for entry to a PhD Programme);
- F336/323 MSci Honours Chemical Physics with Industrial Experience (a four-year programme in which the third year is spent in industry);
- FM11 BSc Joint Honours Chemistry and Law (a joint four-year programme in which the first two years are spent studying chemistry and the second two years are spent studying law).
139 All BSc (Hons) Degree programmes are recognised by the Royal Society of Chemistry (RSC). The F103, F104, and F105 programmes are accredited by the RSC and, as part of a five-year revision, have been submitted to the RSC for reaccreditation. The F107 programme has been submitted for accreditation for the first time.
140 The DSED was based on documentation produced for the Chemistry Departmental Review of May 2002, but restructured and updated for the purpose of the audit. It provided contextual information about the School of Chemistry and sections devoted to undergraduate education and postgraduate education. In a short conclusion, the DSED claimed that the School provided 'appropriate learning and teaching opportunities to enable its students to develop the skills and knowledge required on completion of the degree'; it also claimed to be 'working to improve the student experience through effective quality enhancement mechanisms'.
141 The School of Chemistry represents a large sector of the science provision at the University. The School was rated 5*(B) in the 2001 RAE and received a grading of 'Excellent' in the 1993 HEFCE Teaching Quality Assessment. External input to the School's provision is provided by the Teaching Advisory Board (TAB), and the Industrial Advisory Board (IAB). The audit team learned that the TAB had advised on the restructuring of the first-year laboratory course to reflect the experience of the intake more appropriately. The team considered that the use of input from both boards to the curriculum was a noteworthy feature. As the School had undergone departmental review in May 2002, it was of interest to the team to explore matters raised in that review and the responses thereto. It was also of interest to learn how the University's policies for the management of quality and standards were enacted at School level for a large and successful academic unit.
142 The audit team was provided with a set of programme specifications for all of the degree programmes which drew on the FHEQ and made reference to the Subject benchmark statement for chemistry. They are not included in the student yearbooks; the School may wish to consider either including the specifications in the yearbooks or directions to the TSU web site where they also appear. The team noted that, while full unit specifications are a University requirement for new or revised modules, the specifications for a substantial number of units were published in the relevant yearbook in a much simplified version lacking ILOs and methods of assessment. The team recommends that full unit specifications be developed to ensure that students have complete information about the units which they are studying.
143 The DSED did not include any information about progression and completion data. A set of statistical data provided in the course of the DAT indicated very high levels of both retention and achievement; such data are considered at a departmental meeting in November. The School has appointed a School Progress Officer who is charged with receiving and acting upon feedback from teachers on absenteeism or late work; dealing with day-to-day problems of undergraduates; and acting as the channel through which progress of all chemistry students is transmitted to the Faculty of Science Progress Committee. The students met by the audit team were aware of the role of the Progress Officer as a complement to the personal tutorial system. The team regarded the role of the Progress Officer as an example of good practice both in enhancing student support and monitoring quality and standards.
144 Internal monitoring of both degree programmes and module units is continuous throughout the year by the Teaching and Planning Committees, by means of student questionnaires and informal feedback, and through the SSLC and the GSC. End-of-year review of programmes and units occurs through APR. APR in the School operates in accordance with standard University procedures, taking into account student and staff feedback and comments from external examiners. The audit team read the APR reports for 2002-03 and can confirm that they cover the following: a very detailed analysis of student feedback questionnaires; detailed analysis of the examination results for all four years of the programmes; the marking of the final year research projects; review of programme specifications; and consideration of, and responses to, external examiners' reports for 2002-03.
145 The Science FQAT visits the School annually following receipt of the APR and provides a means whereby the University monitors the quality of the School's teaching programmes. The audit team read the report of the FQAT visit to the School in April 2004. The FQAT team had identified a series of both University-wide generic issues and faculty-specific issues for discussion. The report covered progress in response to issues raised in the 2002-03 visit and identified the appointment of a Teaching Laboratory Manager as good practice in the provision of technical support to students, an opinion with which the audit team would agree. The team noted that departmental responses to matters raised were full and well-documented in the FQAT visit report. The team concluded that the FQAT visit to chemistry exemplified best practice as a highly effective way for the University to maintain an overview of the student experience both at the undergraduate and, in the case of chemistry, at the postgraduate research levels. The Departmental Review of chemistry of May 2002 was also a rigorous exercise which covered research and management issues as well as teaching and learning. Issues noted by the Review team included the variability in levels of training of postgraduate students involved in teaching support.
146 Three of the School's programmes were identified by the School to the FQAT as lying outside the University's credit framework. Of particular note was FM11 (chemistry and law), which was described in the programme specification as a BSc (Honours) degree. This is based on two years of chemistry, followed by two years in the Law School, the final year of which contains 80 credits of H-level law. However, no H-level chemistry is experienced by the student; and the audit team considers therefore that this programme diverges from the FHEQ. While noting ongoing discussion in the University on this matter, the audit team recommends that the University revire its approach to qualifications titles for programmes of study involving more than one discipline in the context of the guidance in the FHEQ to ensure that such titles represent accurately the balance of the components in both level and volume of study (see paragraph 73).
147 The audit team read the external examiners' reports for the years 2001-02 and 2002-03. These were highly complimentary, confirming the high standards of the programme (which is research-led) and the quality of the student experience as corroborated in the team's meetings with students. The team noted that the recently revised external examiners form was in two sections, the first requiring confirmation that the programmes were in alignment with subject benchmarks, the FHEQ and the Code of practice: the examiners confirmed this in every case. The team also noted that University policy required the School to respond to every issue raised by the external examiners and that this had taken place, responses being made both to the TSU and the individual external examiners. The team viewed the operation of the external examiner system as entirely satisfactory with added reassurance from the FQAT investigating follow-up action.
148 The DSED stated that assessment within the School was both continuous and carried out through written examinations, and followed the University Assessment Guidelines as set out in the Handbook. Assessment procedures and details of how marks are allocated for the various degree programmes, together with expected learning outcomes, are 'given in the programme specifications and the student yearbooks'. The School Student Progress Officer has both a monitoring and pastoral role which the audit team considered an example of best practice in support for student welfare and progress. The team confirmed that the numerical breakdown of the credit allocated between and within years was provided, but could not locate any grading criteria or descriptors, for instance, descriptions of the qualities associated with a particular grading level that are required to achieve that level, in the yearbook, although it understood that such criteria were published in the Faculty Handbook and the TSU web site. The School may wish to consider the inclusion of grading criteria in the chemistry yearbooks.
149 The audit team explored other issues relating to assessment: first, the assessment of students while on their one-year placement in Europe, which counts 20 per cent towards their final degree mark. The team learnt from reading minutes of examination boards and its meetings with staff and students that the learning in certain European universities had not been assessed in Europe in other than a fragmentary way and that translation of the resulting marks to the University had, therefore, not been fully secure, despite the best endeavours of the School to devise means of translation in such cases. Secondly, the assessment of the work performed on the industrial placement programme had raised concerns with the external examiners but the team was reassured that a revised assessment procedure had been put in place for the 2003-04 session. Thirdly, the lack of double-marking or any other form of internal moderation of examination scripts which counted towards degree classification. The team was informed that a secretarial check was made of additions and completeness of marking, but that no internal moderation, even by sampling, was practised in the School. The team therefore recommends: first, that the School review its relationships with those European universities that fail to assess adequately substantial amounts of lecture material; secondly, that it keep under review the moderation of the industrial placement year; and thirdly, that it introduce some form of internal moderation, in line with the University's Assessment Guidelines, of examination scripts counting towards degree credit.
150 The audit team reviewed a range of assessed student work from all levels of the programmes. It was satisfied that the nature of the assessment and the standard of student achievement were appropriate to the titles of the relevant awards and their location within the FHEQ.
151 Student yearbooks are provided for the various programmes and years and are well structured and informative. As noted above, students are provided with a short précis of the curriculum, learning outcomes and means of assessment rather than a set of programme specifications.
152 The School is fortunate in the sizeable Worsley Chemical Library, which is part of the University library system: it contains 10,000 books, 3,000 theses, 19,000 bound journal volumes, and 60 current titles, and is appropriately staffed. It has 130 study places for students, is staffed for between 50 and 60 hours per week in term time, and offers 24-hour access for postgraduate students. The School also contains over 600 networked computers. Laboratory provision is appropriate as regards spaces, equipment and technical support. The principal means of monitoring learning support is through the SSLC and student feedback on laboratory courses. Students whom the audit team met were fully satisfied with library and IT provision and with the level of equipment in the undergraduate laboratories, but less satisfied with the condition of the undergraduate teaching laboratories, a theme noted in departmental review in 2002. The SSLC is chaired by the Head of Department and comprises one staff member from each of the sections, undergraduate members from all years and degree programmes, the Administrative Officer and the Director of Undergraduate Studies. It meets four times a year to receive student comment on teaching and learning. The records of discussion are forwarded to the Teaching Committee and/or section heads for discussion and action. SSLC minutes are circulated to academic staff and published on student notice-boards. The students met by the team were satisfied with the operation of the SSLC, and the team noted the comprehensive and critical SSLC minutes. Despite the lack of any student role in chairing or acting as secretary to the SSLC, the team concluded that the systems for gathering undergraduate student opinion were effective. The team was informed that there was no student representation on either the Staff Meeting, or the Teaching Committee: it also noted that one of the conclusions of the recent FQAT visit was that the School should 'consider ways in which students might have a greater involvement in quality assurance processes', a view with which the team concurred.
153 Postgraduate students are not represented on the SSLC but have representation on the GSC which comprises two academic staff members, two postgraduate students from each of the three sections, and a postdoctoral researcher; the GSC meets five times a year. At its meeting with staff, the audit team was informed that the GSC also functions as a forum for discussion of issues raised by postgraduates. The Graduate School monitors the progress of all postgraduate students and ensures that they have opportunities to gain subject-specialist training, advanced subject knowledge, communication skills and an understanding of professional matters. Students are encouraged to monitor their own acquisition of transferable skills using the RSC/DfES-sponsored Professional Skills Record, the development of which had involved the School.
154 The DSED described the process of postgraduate recruitment, induction and monitoring of progress: the latter involves production by the student of an annual report which is discussed in an end-of-year interview between the student and two members of the academic staff: in the second and third years neither of these is the supervisor. Detailed feedback to the student is given both orally and through a written report. The students have the opportunity to comment about the overall training programme and all partners sign the form. All such forms are read by the Director of the Graduate School, and any concerns expressed are acted upon. In the event of unsatisfactory progress, a further interview is held after three months. The forms are then submitted to the Graduate Dean of the Faculty of Science for approval. Successful completion of the first year leads to the process of transfer of student registration from MSc to PhD. The audit team shared the view expressed in the departmental review that the system for monitoring PhD students in chemistry was very good. It also noted that the same review had commented that not all students involved in demonstrating duties had attended the University-run training course. However, training is given to research postgraduates acting as demonstrators at the beginning of each session by academic staff in the School and attendance at these sessions is mandatory. The team recommends that participation in appropriate University-level training be made a requirement for postgraduate students acting as demonstrators.
155 All staff at the School had been subject to SRD in the previous 12 months. There is no peer observation in the School, a situation noted in the report of the most recent FQAT visit. While congratulating the School on the universal participation in SRD the audit team also recommends that the School give early consideration to the introduction of peer observation.
156 On the basis of the written evidence it examined and its meetings with staff and students, the audit team was satisfied that the quality of the learning opportunities available to students was suitable for programmes of study leading to the named awards.
Dentistry
157 The DAT covered the following programmes:
- the undergraduate Bachelor of Dental Surgery (BDS) programme;
- the Doctor of Dental Surgery (DDS) programme, which is a taught doctorate.
158 The BDS is a five-year programme with a total of 253 full-time students. The DDS programme comprises one year of full-time study followed by two years of part-time study combined with research and practice, there are 10 students currently enrolled. The pre-clinical components of the BDS programme (predominant parts of years one and two) are taught mainly in the Faculty of Medical and Veterinary Sciences. The clinical part of the BDS and all of the DDS are delivered within Bristol Dental School and Hospital. This involves academics in the Department of Oral and Dental Science and colleagues in the NHS. The Department of Oral and Dental Science is one of the five departments in the Faculty of Medicine and Dentistry.
159 The DSED was written for the purpose of the audit; it was accompanied by: the confidential draft report of the visit by the General Dental Council in December 2003; programme specifications for both programmes; student handbooks and School of Dentistry Codes of Practice. The programme specifications are adequate but, in common with all dental and medical programme specifications, suffer from the difficulty of trying to make non-standard, vocational programmes fit a standard academic template. Discussion with staff confirmed due account had been taken of subject benchmark statements and General Dental Council guidelines (The First Five Years) for the BDS. The DDS programme specification had been produced in discussion with the TSU and with external dental colleagues and was considered by them to represent a template that might be used for other professional taught doctorates, in that it managed to meet the requirements of the FHEQ and the professional CPD arrangements.
160 The School's approach to the management of progression and completion data for the BDS was set out in the Code of Practice on Student Progression which included details of record keeping, requirements for successful progression, monitoring student progress, managing underachieving students, fitness for practice and the functions of the Student Progress Committee. The minutes of the Pre-clinical and Clinical Dental Progress Committees complied with the provisions of the Code of Practice. Material for the DDS programme covering admissions, formative assessment, summative assessment and progression was also in keeping with the stated procedures.
161 Arrangements for monitoring and review of the provision were described in the Code of Practice for Quality Enhancement, which also included an overview of external review of the BDS programme, the University and faculty level mechanisms for quality management and enhancement, the departmental level mechanisms and staff quality management. The departmental level mechanisms covered element and unit review, the data collection necessary at that level for external, University and faculty review, the purpose of the APR and the Dental Education Committee's role. It also included the use of external examiners' reports and student feedback.
162 Documentation seen by the audit team demonstrated excellent compliance with the APR process with complete records of the most recent APR including element review material, external examiners' reports for the last four years and details of student feedback. The conduct of the APR was documented in the minutes of the relevant Dental Education Committee and the formal response from the FQAT. There was clear evidence that there had been consideration of external examiners' comments, student feedback, and changes to examination format and handbook quality. In addition, the minutes of the Board of Dental Studies for the last two years demonstrated that there was rigorous review of the minutes of its subcommittees and coordination of important themes around quality. The Board of Dental Studies reports to the Faculty Board of the Faculty of Medicine and Dentistry.
163 Student feedback for the BDS and DDS/MSc programmes was provided to the audit team. This operated at all levels from unit to programme and was clearly taken into account by the programme organisers. This was confirmed in the meetings with the students where examples were given of the impact of student feedback such as journal club changes for the DDS students and the wide range of committees at programme, departmental, faculty and university level on which the students were represented. The team considered that School's use of innovative ways of gaining, analysing and sharing student feedback through the use of the VLE was a feature of good practice.
164 The assessment strategies and policies were clearly described in the Code of Practice on Assessment, which included marking criteria for a wide range of assessments including professional behaviour. It defines the roles and responsibilities of the academic staff who must be involved in designing, implementing and monitoring assessments of all types. Absolute compliance with all elements of the University's Guidelines on Assessment was not expected because the Code stated explicitly in its introduction that some University guidelines applied to all faculties except that of Medicine, as follows: 'particularly where mention is made of class of degrees and Boards of Examiners. It should also be noted that all examinations in the Faculty are professional examinations and that, alone among the Faculties, programmes in the Medical Faculty are not modularised, each year representing 120 degree points'. The status of the Code was clear as summarised in the phrases 'This Code of Practice describes the conduct of assessments in the BDS programme...This document is to advise on good practice in assessments...The University Regulations are mandatory'. The Code of Practice includes excellent sections on the Philosophy of Assessment in the BDS programme, the wide range of assessment methods used in the programme and the scoring and judgement of professional attitudes.
165 The student handbooks provide a broad range of practical information about facilities, academic and personal conduct, student support schemes and contact details. The assessment timetable and details of each assessment were laid out clearly in the Clinical BDS Handbook. The assessments for the DDS and their relationship to The Royal College examinations were adequately summarised. In discussion with the students, they were complimentary about the quality of information provided both before starting and throughout the programme. There was a standardised format for the Handbooks for the programme which was observed notwithstanding the involvement of two separate faculties.
166 The approach to ensuring appropriate learning resources and support was contained in the Code of Practice on Learning Resources which takes account of the University Learning and Teaching Strategy, the Faculty Planning and Research Committee's guidance and the Board of Dental Studies Mission Statement for Learning Resources. It covers teaching material and equipment, library services, IT, teaching and social accommodation, and technical and support staff. The learning resources for years one and two of the BDS programme are coordinated by discussion at the Board of Medical Studies. The mechanism for addressing learning resources is through the unit and element organisers reviewing their needs in time for the APR so that the Dental Education Committee can consider the matter; the Chair of the Committee then makes recommendations to the relevant budget holder. It was apparent that the current level of resource, in terms of space, equipment and staff, was well documented. There was evidence that the system was operating well, an example being resources for human disease and pathology teaching with the topic raised at unit level and then pursued by the Dental Education Committee and the Dean and clinical colleagues.
167 In discussion with the students, there were high levels of satisfaction with the quality of the facilities, library provision, IT availability and specialist equipment. There was also strong positive feedback about the contact with staff at all stages of the five year programme and the postgraduate programme. The development of a VLE and introduction of associated software were particularly valued.
168 Staff development and peer observation was working well. Peer observation involves a three-year cycle, which allows clinical, tutorial and lecture teaching sessions to be assessed in sequential years. This is conducted against a structured template and the completed forms are seen by the unit organiser and head of teaching division and are available at unit review and APR. The approach to peer observation was considered by the team to be excellent practice.
169 The audit team reviewed a range of assessed student work and was satisfied that the nature of the assessment and standard of student achievement were appropriate to the titles of the awards and their location within the FHEQ. The team was also satisfied that the quality of the learning opportunities was suitable for programmes of study leading to the named awards.
Engineering mathematics
170 The DAT covered the following programmes:
- MEng Engineering Mathematics (four-year programme);
- MEng Engineering Mathematics with Study Abroad (four-year programme with year three spent studying abroad);
- BEng Engineering Mathematics (three-year programme);
- MEng Knowledge Engineering (four-year programme);
- MEng Knowledge Engineering with study abroad (four-year programme with year three spent studying abroad);
- BEng Knowledge Engineering (three-year programme).
171 All programmes are recognised by the Institute of Mathematics and its Applications.
172 The Department has significant input into the following programmes:
- MEng/BEng Engineering Design;
- MEng/BEng Computer Systems Engineering;
- MEng/BEng Avionic Systems;
- MSc Industrial and Environmental Modelling.
173 The Department was subject to an internal Departmental Review in February 2004, shortly before the audit visit. The Department submitted the SED prepared for that internal review as the DSED, along with programme specifications. Annexes referred to in the document were made available to the audit team. All of the programme specifications were in alignment with the FHEQ and informed by relevant subject benchmark statements.
174 The Department of Engineering Mathematics is situated within the Faculty of Engineering and has 15 academic staff. The Department is research active, having gained a 5* A rating in the 2001 RAE, and has been awarded significant research grants. The Department participated in subject review in 1998 jointly with the School of Mathematics: the provision was approved with a grade of 23.
175 The University specifies the data sets which must be considered at APR and departmental review. The audit team read the Engineering Mathematics APR report for the academic year 2002-03. It comprised a brief account of the delivery of each unit together with some commentary of the level of performance, usually in the form of average examination marks. Clearly actions were taken in some units as a result of consideration of marks. For example, in one unit it was reported that 'the marks from the previous year had been low and the unit content had been reduced as a consequence, which had produced better results in 02-03'. The data sets as required by the University and provided by the TSU to be considered in APR were not included and there was no systematic consideration of progression or classification data. The APR focused on individual units and not on the programme and the programmes to which the review related were not mentioned specifically. The report did not follow the format specified in the 'Notes of Guidance for Programme Review'. The APR functioned as a review of units with a focus on examination performance. The team noted that the APR reports contained no reference to external examiners' comments; APR is not used by the Department as a means of monitoring issues raised by external examiners as the review meeting takes place prior to receipt of their reports. External examiner reports are considered at the Departmental Education Committee and in meetings with the FQAT. The Departmental Education Committee keeps programmes under review at its regular meetings.
176 One of the annexes to the departmental review document made available during the audit contained the required data sets. Although the data provided were comprehensive, the review document contained no discussion or analysis thereof. The audit team learnt that progression data were considered routinely after examination board meetings. At the time of the audit, the report of the Departmental Review had not been issued and hence the team was therefore, unable to judge whether issues had been identified and follow-up action initiated.
177 The external examiners' reports read by the audit team were very positive with few issues being identified for attention. The team saw the responses to the reports, which explained how the issues raised were being addressed. A pro forma detailing 'Issues' and 'Responses' was completed and sent to the TSU so that a central overview could be maintained.
178 Assessment strategies were specified in the unit descriptions and student handbook both of which are on the Departmental web site, and also available to students in paper form. Assessments typically involve formal written examinations, short tests and coursework. Progression rules are also included in the Student Handbook. Any mark obtained in a supplementary examination (resit) is capped at 40 per cent in line with University regulations. The students whom the audit team met were clear about what was expected of them in assessments. The programme specifications correlated well with the types of assessment exemplified.
179 The audit team reviewed a range of assessed student work, by level and unit. The team was satisfied that the nature of the assessment and standard of student achievement were appropriate to the titles of the relevant awards and their location within the FHEQ.
180 Students whom the audit team met reported that they found programme and unit handbooks helpful and accessible. Handbooks were available on the Department's web site; students were provided with free printing should they wish to print a paper copy. Students were clear about the ILOs and assessment of units. Information provided to them at induction was comprehensive and useful. The Department's web site also provided information about option choices which was clear and helpful.
181 The Department has dedicated computer laboratories. Students whom the audit team met reported that they had ready access to computers with good technical support. They considered the library facilities to be more than adequate.
182 Some programmes have a year where the student studies abroad. From discussions with staff and students, the audit team concluded that the year abroad is supported appropriately, with clear points of contact for the student. Staff from the University made two visits during the year to students studying abroad.
183 Without exception students praised the friendliness and availability of all staff, whom they found to be knowledgeable and helpful. The accepted normal means of communicating with staff is by email; students found that this worked well, with prompt replies being received from staff. There were weekly timetabled meetings with personal tutors in year one and fortnightly timetabled meetings in year two. In the final year the project supervisor takes on the role of personal tutor. Students found these meetings with personal tutors both helpful and supportive. Students also appreciated the assistance given in obtaining summer work relevant to the programme with the Department's industrial contacts.
184 Staff reported that all full-time teaching staff engaged in a peer observation scheme in which observation was non-reciprocal. All staff new to teaching must attend a teaching skills course. Attendance at this course is strongly encouraged but not mandatory for postgraduate students who teach.
185 The Department has a variety of student feedback forms: some ask for a response on a five-point scale, others invite more open-ended commentary. All units employed an end of unit questionnaire. Summaries of the questionnaire results were provided to students either on the web site or in lectures. Students found staff receptive and responsive to their requests for change.
186 The Department operates a SSLC for student representatives. This meets once a year, with the agenda being set by the student members. Additionally, students also set the agenda for a student forum held each year.
187 Students had recently drawn attention to two main issues: the timing of assessments and the return of work. Students reported some cases where several assignments were due within a short time period, causing undue pressure. Staff acknowledged the problem, which in part was due to the requirements for submission of work set by other departments within the Faculty. Students reported variability in practice in the return of work. Some students reported submitting work which was not returned; others reported delays of months. The quality of feedback also varied. In discussions with staff and students, it was clear that staff were aware of the problem and were seeking solutions. The amount of coursework set to students was, in some cases, disproportionate to the credit value of the unit; this contributed to delays in the return of work with a high volume of coursework. Steps were being taken to address this through the issuing of guidelines to indicate volumes of coursework appropriate to units of various credit sizes. The audit team would encourage the Department in this endeavour.
188 The audit team was satisfied that the quality of learning opportunities was suitable for the programmes of study leading to the named awards.
Politics
189 The DAT covered the following programmes:
- BSc Politics;
- the politics element of the following joint programmes:
- BSc Economics and Politics
- BSc Philosophy and Politics
- BSc Politics and Sociology
- BSc Social Policy and Politics
- BA Politics and French
- BA Politics and German
- BA Politics and Russian
- BA Politics and Italian
- BA Politics and Spanish
- BA Politics and Portuguese
- BA Theology and Politics
- MSc/Dip Development Administration and Planning;
- MSc/Dip International Relations;
- MSc/Dip Gender and International Relations;
- MSc/Dip International Security;
- MSc/Dip Development and Security;
- MSc/Dip Politics and International Relations;
- MSc/Dip/Cert in Research Methods and the MPhil/PhD.
190 As the last internal review of politics was in 1994 the DSED was prepared specifically for the audit visit. For undergraduate programmes it contained very detailed data on entrance qualifications, progression and degrees awarded. Programme specifications were provided and indicated clearly the content and structure of the programmes, setting out appropriate educational aims and learning outcomes for both undergraduate and postgraduate awards. All the specifications were referenced to the FHEQ and the University's own credit framework; the specifications for undergraduate programmes were also referenced to the Subject benchmark statement for politics and international relations.
191 The Department of Politics has 18 full-time academic staff and offers a range of undergraduate awards, the main one being the BSc Politics. There are also four joint honours programmes in the Faculty of Social Sciences and Law and seven in the Faculty of Arts. Total student enrolment on these programmes during the academic year 2003-04 was 262 FTEs. In addition, 257 students on other University degrees and study abroad programmes were registered for politics units. At postgraduate level there are six taught MSc/Diploma programmes accounting for 130 FTEs as well as another 40 students registered for MPhil/PhD degrees. The Department was rated 5(A) in the 2001 Research Assessment Exercise. MSc degrees last for 12 months, the final three months being devoted to writing a dissertation of 10,000 to 15,000 words; the Diplomas last for nine months and involve the same coursework and assessment as the MSc degrees except for the dissertation.
192 Monitoring of progression data is exemplary: an example of change initiated as a result of monitoring was the introduction of a new diagnostic test in English language for international students. Degree results over the last six years have been excellent with over 90 per cent of students graduating with a First or Upper Second class honours degree.
193 The DSED identified FQAT visits and APR as the 'cornerstone of Departmental Quality Assurance' and stated that 'effective internal quality enhancement mechanisms' were essential if the student experience was to be successful. Internal monitoring and review has a number of elements including student feedback, programme reviews, staff reviews and peer observation of teaching. Students complete three questionnaires for each unit: one relating to lectures, one to seminar teaching and one to the unit as a whole. The owners of units then prepare a self-assessment report which includes a summary of student feedback and is made available to students, although those students seen by the audit team had not seen these reports. The self-assessment goes to the Chair of the Academic Committee with the original questionnaires for monitoring. Although student evaluations of staff teaching were generally high, the team was informed that a score of less than 3 - on a range of 1 to 5 - would automatically lead to peer observation of that individual's teaching. Students complete end of programme questionnaires which are considered by the departmental Academic Committee.
194 Students have an input into APR and the subsequent FQAT visits. When an APR recommends action, the Head of Department and the Chair of Academic Committee are responsible for ensuring that this is carried out. Following receipt of the APR the FQAT visits the Department and consults students as well as staff; the FQAT report is submitted to the Faculty.
195 There are satisfactory arrangements for staff review and peer review of teaching. The audit team was assured that if either of these processes or student feedback suggested that a particular member of staff should be offered staff development then the Head of Department was responsible for assuring that appropriate action was taken. The Department is responsive to the development needs of its staff. New members of staff without prior teaching experience are required to attend the TLHE programme. A noteable feature in support of staff development is that the Department keeps a 'best practice' file to which can be added experiences gleaned nationally as well as locally; the contents of the file are discussed at Departmental away days.
196 External examining arrangments conform to the University's guidelines. External examiners have commented favourably both on the politics programmes and the examination process itself. Reports from external examiners are received by the Head of Department who, after internal consultation within the Department, replies to them. These responses are then considered during the APR and the subsequent FQAT visit as well as at the following year's examination board. The Head of Department is responsible for ensuring that any follow-up action is taken.
197 Although assessment strategies are consistent with the Code, there is little formative assessment, particularly on taught postgraduate programmes where students are not required to undertake any formative assessments before submitting the summative assessments on which their degree results are based. At undergraduate level, final-year students have the benefit of feedback from mock examinations.
198 The audit team reviewed a range of assessed student work; it was satisfied that the nature of the assessment and the standard of student achievement were appropriate to the title of the awards and their location within the FHEQ.
199 The Department produces separate handbooks for undergraduate and postgraduate students. While, at the time of the audit, these did not include either programme or unit specifications they did outline the structure of programmes and provide brief unit descriptions. They also contain much other useful information and guidance on approaches to assessment and on marking and marking criteria. Unit information is available to students through the departmental intranet, the VLE and in the departmental office.
200 IT facilities were deemed to be satisfactory by students but they did have some reservations about the library, echoing the findings reported in the SWS, namely that a lack of multiple copies put pressure on the short loan collection. Difficulties with the library stock were also identified in the minutes of the Politics Staff meeting of October 2003, which noted that the book budget '[did] not correlate with the increased number of students'. The matter was kept under review by the SSLC, improvements were made to the short loans service and the Department attempted to ameliorate the situation by introducing course packs for some of their units. This action undoubtedly helped to ease the situation and was appreciated by students.
201 As noted above, student feedback on individual units is through questionnaires. On wider issues students feedback informally through their tutors and teachers and, more formally, through the SSLC. At undergraduate level two students from each year are elected to the Committee which meets regularly and circulates its minutes to all students by email. There is also a staff-student committee for taught postgraduates and another, established more recently in November 2003, for research postgraduates. These committees, which are student driven, appear to be working well and there are no major areas of student discontent. In addition to the library, recent examples of issues raised on these liaison committees and acted upon include course packs, the replacement of examinations with formative and summative essays at level 1 and the slowness of computers.
202 From its reading of documentation and assessed work, and from its discussions with staff and students, the audit team was satisfied that the quality of learning opportunities available to students were suitable for programmes of study leading to the named awards.
Theology
203 The DAT covered the following taught programmes in theology and religious studies, together with postgraduate research study:
- BA Hons Theology and Religious Studies;
- BA Hons Theology and Politics, Theology and Sociology, Philosophy and Theology;
- MA in Buddhist Studies;
- MA in Christianity and Culture.
204 As a substantial proportion of the University's collaborative provision, programmes offered by Wesley and Trinity Colleges, together with Bristol Baptist College, was overseen by the Department, this was also considered. The DSED was prepared specifically for the audit, and provided a brief factual guide to the aims and learning outcomes of the provision, curricula and assessment, quality of learning opportunities and maintenance and enhancement of standards and quality, without any evaluative comment. Programme specifications were appended, along with programme handbooks, but no supporting data, statistical or otherwise, was supplied. During the audit team's visit it became clear that the Department was self-evaluative and made effective internal use of such data.
205 The programme specifications made clear links to the FHEQ and to the subject benchmark statements. The audit team was told that the production of the specifications had led the Department to identify the close fit between their provision and the subject benchmarks, issued in 2000, for the first time.
206 Progression and completion data, as supplied by TSU for the APR process, were duly considered by the Department during that process and demonstrated high levels of progression on taught programmes: data on research student cohorts were not available. Generally, both APR and FQAT processes were conducted with thoroughness and linked appropriately to a series of other internal mechanisms for monitoring and enhancing teaching quality, such as an effective Teaching Committee. The APR process extended to the theological colleges and was reported through the department, although the annual monitoring of the collaborative provision was not undertaken by the FQAT team but by annual review meetings conducted by the Dean of Arts.
207 External examiners' reports on the taught programmes of the Department and the theological colleges were thorough and supportive and followed University policy in their format and how they were processed, although there was a one-year time lag in the adoption by the colleges of the new template for such reports, introduced in the Department in the academic year 2002-03. Issues identified were rigorously debated internally and outcomes reported back to the external examiners and also through APR and FQAT visits.
208 The assessment strategies and policies of the Department were derived from those of the Faculty of Arts, and were clearly laid out in the student handbooks. The audit team considered as noteworthy practice the provision of clear and consistent statements of assessment criteria for different grades and of how these criteria were weighted differently for work at different levels. Methods of assessment, especially after level C, were relatively traditional, with learning outcomes for generic skills such as oral presentations not assessed formally after that level. The team noted that there were examples of overlap between coursework and exam questions on some modules, affecting some answers, and suggests that the Department may wish to consider whether its current procedures are sufficient to ensure that this does not result in inequity for students. Until recently, joint degree programmes were expected to foster interdisciplinary skills and interconnection between the two subjects, but this was less apparent in the DSED documentation: in meetings it was explained that the personal tutor for such students guided them to choose modules which facilitated such development, but it was not assessed directly.
209 The audit team reviewed a range of assessed work; it was satisfied that the nature of the assessment and standard of achievement were appropriate to the titles of the awards and their location within the FHEQ.
210 The student handbooks were generally accurate and informative although, like the departmental web site, they only contained abbreviated versions of unit specifications and hence limited information on assessment procedures and learning outcomes. Student rights and responsibilities were generally clearly stated, although there was no guidance either on complaints and appeals procedures or on arrangements for resits: instead students are advised to consult the Rules and Regulations for Students booklet which provides information about University complaints and appeals procedures.
211 Learning resources and support within the Department were appropriate and sufficient, but both staff and students confirmed persistent concerns that the Arts and Social Sciences Library was insufficiently funded for taught programme provision, particularly in provision of books. The Department's efforts to mitigate the problem through use of study packs and other measures, and by supplementing the central budget from departmental funds, was much appreciated by students, but it appeared that there was no clear University mechanism for relating library spending to student needs. The personal tutor system was regarded as working well, although largely on a responsive basis, with students able to access both personal tutors and all staff whenever required. The use of the system to support personal development was informal at present.
212 There was clear evidence of the impact of student feedback on units through the evaluation questionnaires. These were considered by unit staff who produced a self-evaluation at the end of each year responding to the comments, which was discussed by the Department, while the overall evaluation results were thoroughly considered within the APR process. There was no programme level evaluation at levels C and I, but at level H students met the Senior Tutor to complete a general programme questionnaire and also to discuss supplementary questions, for example, on the working of the personal tutor system, and this was also analysed and followed up thoroughly. Postgraduate research students could comment on their supervision and other matters on the annual report form which went to the Head of Department and Graduate Dean. Research students are also advised in the departmental postgraduate handbook that they can discuss any matter relating to their progress with the department's Postgraduate Officer. Students are represented at the department meeting and on its committees as well as through the SSLC, an active body in which students feel able to raise a range of issues and receive timely and generally supportive responses on issues within the Department's control.
213 The audit team noted an evident staff culture of concern for the quality of learning and the fostering of staff development, for example, through an annual meeting to discuss good practice, based on a good practice file available for staff consultation. The team concluded that the current undergraduate programmes were ideally matched to the interests of the staff and the types of student recruited, but that, unlike the postgraduate programmes or the programmes offered by the theological colleges, which appealed to wider audiences and to vocational and public concerns, they appeared to be based on a relatively narrow academic framework, which might explain the relatively homogenous student intake.
214 The audit team concluded that the quality of learning opportunities was suitable for the programmes of study leading to the named awards.
Thematic enquiries
215 The audit team did not select any areas for thematic enquiry.Section 4: The audit investigations: published information
The students' experience of published information and other information available to them
216 Printed information sources available to the audit team included undergraduate and postgraduate prospectuses, the Student Services booklet, general regulations for doctoral and masters degrees, the University calendar, Rules and Regulations for Students, programme and unit guides. The team also viewed the University web site which presented a wide range of information about the University, its academic provision, arrangements for student support and facilities. The team discussed these sources of information with a representative group of undergraduate and taught postgraduate students, a group of research students and students in the five DAT areas.
217 The University has an appropriate system for assuring the quality and accuracy of the prospectus. Departments are responsible for checking the accuracy of relevant sections, with the Academic Registry managing the process and having final responsibility for its production. The principle that the University has adopted for other publications and for internet-based material is that the originator of any material is responsible for its accuracy and quality. The checking of any publicity material on the web site of a partner college is the responsibility of the University department where the programme resides. The University has a corporate webmaster responsible for corporate items and for producing templates for use by departments to ensure consistency.
218 The students' experience of the information that they receive was largely positive. Taught students, both undergraduate and postgraduate, whom the audit team met, were complimentary about information that they received from all sources. This was also true of the students met during the DATs. Pre-arrival information, induction and handbooks, whether internet-based or paper-based, were all found to be useful and clear. The views expressed by students whom the team met concurred with comments in the SWS.
219 There are no University guidelines on the format or content of student handbooks, leading to a lack of consistency across the institution as was illustrated by the handbooks available to the audit team through the DATS. The conformity and quality of handbooks could be enhanced by the sharing of best practice; the audit team recommends that the University consider specifying minimum requirements for the contents and quality of student handbooks with particular reference to regulations and assessment criteria.
220 Postgraduate research students reported variable experience of provision of information. Although pre-arrival information was generally found to be useful, the students reported a lack of information about information services and accommodation. With these exceptions for postgraduate research students noted, the audit team considered that information for students was accurate, thorough and complete.
Reliability, accuracy and completeness of published information
221 The University is fully aware of requirements for the collation and publication of TQI and the relevant deadlines for publication. The preparations for the publication of the qualitative information as required by HEFCE's document 03/51, Information on quality and standards in higher education: Final guidance are being managed by the TSU. External examiner report pro formas have now been amended in line with the final guidance from HEFCE. The University guidelines on departmental review have now been amended to state that summaries will have to be published. Information on employer links is currently being prepared by the Careers Service. Many programme specifications are currently on the University web site; others need faculty approval before being added. The University plans to have a link from the TQI web site to the programme specifications on the University web site. The summary of the University Learning and Teaching Strategy will be prepared by the TSU.
222 Staff met by the audit team expressed confidence that all information would be loaded onto the TQI web site within the required time. The team shared this confidence and was satisfied that the University would meet the requirements of document HEFCE 03/51 within the timescale specified.
223 On the basis of documentary evidence and meetings with staff and students, the audit team formed the view that reliance could be placed in the accuracy, integrity, completeness and frankness of the University's published information.
ISBN 1 84482 152 8
next >> Findings
Top
QAA updates
QAAtweets
QAAtube