1 An institutional audit of the University of Surrey, Roehampton (Roehampton) was undertaken during the week commencing 1 December 2003. The purpose of the audit was to provide public information on the quality of Roehampton's programmes of study and on the discharge of its responsibility for its awards.
2 The audit was carried out using a process developed by the Quality Assurance Agency for Higher Education (the Agency) in partnership with the Higher Education Funding Council for England (HEFCE), the Standing Conference of Principals (SCOP) and Universities UK (UUK), and has been endorsed by the Department for Education and Skills. For institutions in England, it replaces the previous processes of continuation audit, undertaken by the Agency at the request of UUK and SCOP, and universal subject review, undertaken by the Agency on behalf of HEFCE, as part of the latter's statutory responsibility for assessing the quality of education that it funds.
3 The audit checked the effectiveness of Roehampton's procedures for establishing and maintaining the standards of its academic awards; for reviewing and enhancing the quality of the programmes of study leading to those awards; and for publishing reliable information. As part of the audit process, according to protocols agreed with HEFCE, SCOP and UUK, the audit included consideration of examples of institutional processes at work at the level of the programme, through discipline audit trails (DATs). The scope of the audit encompassed all of Roehampton's provision and collaborative arrangements leading to its awards.
Section 1: Introduction: the University of Surrey, Roehampton
The institution and its mission
4 Roehampton has its origins in the former Roehampton Institute of Higher Education (latterly Roehampton Institute London), formed in 1975. In January 2000, the Institute entered into a federal arrangement with the University of Surrey (UniS) and at that point adopted its current name. The Federal University of Surrey is 'an academic federation based on broad complementarity of academic provision, and mutual support and co-operation in the region'. Under the formal Deed of Federation, the two constituent institutions remain separately funded, separately responsible for the employment of staff, have separate control of their property, and retain separate responsibility for the recruitment and registration of their students. While Roehampton holds its own powers to award taught and research degrees, it has voluntarily held those powers in abeyance and its students receive degrees of UniS. Both institutions have their own senates, and there is also a Federal Senate, which is ultimately responsible for safeguarding the standards of UniS awards.
5 Roehampton is organised into four Colleges - Digby Stuart, Froebel, Southlands and Whitelands - each with its own site. Each College has a providing body, which owns the buildings and estate, and a governing body, which has no academic role but exists to safeguard the interests of the providing body. Almost all of the property used by Roehampton (including buildings, furnishings and library resources) is owned by the Colleges. 'Deeds of adherence' signed in 1977 devote this property to the term-time use of Roehampton for the furtherance of its objectives, which were agreed to be consonant with those of the providing bodies. The Colleges are responsible for the pastoral care of students and for the management of residences, catering and conference-booking. In other respects, Roehampton is a unitary institution.
6 Academically, Roehampton is organised into eight schools: Arts; Business and Social Sciences; Education Studies; English and Modern Languages; Humanities and Cultural Studies; Initial Teacher Education; Life and Sport Sciences; and Psychology and Therapeutic Studies. On 1 December 2003, it had 6,274 full-time students (5,437 undergraduate, 474 postgraduate certificate in education (PGCE), 247 taught postgraduate, 74 research postgraduate and 42 others) and 1,157 part-time students. The biggest subject area, education, had over 1,800 students; the majority of other students were in the arts, humanities and social sciences. All of Roehampton's taught provision is delivered through a Credit Acccumulation and Transfer-rated modular scheme introduced in 1991. For undergraduates, the scheme permits two-subject study 'where possible', which leads to awards of BA (Hons) and BSc (Hons).
7 At the time of the audit, Roehampton was considering the results of an analysis of the risks of a possible application to the Privy Council for permission to adopt the title 'Roehampton University', and of exercising its own degree-awarding powers, while continuing to maintain and develop its close links with UniS. The audit team was informed that this possibility was under consideration in the light of the Government's White Paper - The Future of Higher Education - and the 'rapidly changing national and global environment for higher education (HE)', and that Roehampton wished to work towards a collaborative alliance between higher education institutions (HEIs) in its sub-region, defined as the west/south-west region surrounding London. A decision to proceed with the application was made by Roehampton's Council at its meeting on 8 December 2003.
Mission statement
8 Roehampton's vision, expressed in its Strategic Plan, 2001-06, is 'to be a university with an international reputation for teaching and research, a secure national presence and a distinctive local identity'. Its mission is as follows:
'Roehampton is committed to the pursuit of pure and applied academic enquiry and providing its stakeholders with an academic service of the highest quality. We will offer to as wide a community as possible a portfolio of teaching and learning opportunities across a broad but coherent field of disciplines. We will conduct research of importance and disseminate the results. We will meet the information, developement training and career development needs, both of Roehampton and of a range of external communities'.
Collaborative provision
9 Under the Deed of Federation, Roehampton may develop and maintain relationships with other institutions. Currently, Roehampton has a relatively small amount of collaborative provision, but some expansion is planned, particularly in relation to the delivery of Foundation degrees. Partner institutions offering UniS-validated degree programmes are termed 'associated institutions' of the Federal University.
Background information
10 The published information available for this audit included:
- Roehampton's undergraduate and postgraduate prospectuses, and the information on its web site;
- the report of a quality audit of Roehampton Institute by the Higher Education Quality Council (HEQC), published in 1996;
- the eight reports of HEFCE and Agency reviews of provision at subject level published in the six years preceding the audit, including two reports on developmental engagements.
11 Roehampton provided the audit team with:
- an institutional self-evaluation document (SED);
- a range of supporting documentation, including the Strategic Plan, 2001-06, and the July 2003 editions of the Quality and Standards Handbooks for Taught Programmes and Research Degree Programmes;
- the report of the Agency's 2002 review of the quality assurance arrangements of the Federal University;
- discipline self-evaluation documents (DSEDs) for the provision selected for DATs.
12 During the briefing and audit visits, the audit team was given ready access to Roehampton's internal documents and to the intranet. It was also provided with a range of documentation relating to the selected DATs, including examples of assessed student work.
The audit process
13 Following a preliminary meeting at Roehampton in March 2003, the Agency confirmed that three DATs would be conducted during the audit visit. The audit team's final selection of DATs was Business Studies (focusing on the BA/BSc Business Studies and the MSc/Postgraduate Diploma in Voluntary Action Management); Drama (BA/BSc Drama, Theatre and Performance Studies); and Education (BA/BSc Early Childhood Studies and the Professional Development Programme).
14 The Agency received the institutional SED and supporting documentation in July 2003 and the DSEDs, accompanied by programme specifications, in October 2003. Both the SED and the DSED for Education were written specifically for the audit. The DSED for Drama included a document written for the quinquennial review in April 2002. The DSED for Business Studies included a self-evaluation review document for the BA/BSc Business Studies programme written for quinquennial review in March 2003.
15 The audit team visited Roehampton on 29 to 30 October 2003 for the purpose of exploring, with the Rector, senior members of staff and student representatives, matters relating to the management of quality and standards raised by the SED and other documentation provided for the team. During this briefing visit, the team signalled a number of themes for the audit visit, and decided that it did not wish to pursue a thematic enquiry. At the close of the briefing visit, a programme of meetings for the audit visit was developed by the team and agreed with Roehampton.
16 At the preliminary meeting for the audit, the students of Roehampton were invited, through their Students' Union (SU), to submit a separate document expressing views on the student experience at Roehampton, and identifying any matters of concern or commendation with respect to the quality of programmes and the standard of awards. They were also invited to give their views on the level of representation afforded to them, and on the extent to which their views were taken into account. In July 2003, the student body submitted a document to the Agency. The document, which was not confidential, had been prepared by officers of the SU and included an introduction explaining that it had been written out of term time, when there were few students available to contribute. As a result, it drew upon documents completed by the SU earlier in the year for the Agency's Developmental Engagements in English and History. The audit team is grateful to the SU for preparing this document to support the audit.
17 The audit visit took place from 1 to 5 December 2003 and included further meetings with staff and students of Roehampton, both at central level and in relation to the selected DATs. The audit team comprised Dr M T Atlay, Professor G M Kennedy, Professor D J Lockton and Mr J H Napier, auditors, and Ms W J Appleby, audit secretary. The audit was coordinated for the Agency by Ms P Le Rolland, Assistant Director, Reviews Group.
Developments since the previous academic quality audit
18 The previous academic quality audit, of the then Roehampton Institute, was undertaken by HEQC in June 1995, and the report was published in January 1996. The report commended several aspects of the Institute's practice, including its development of comprehensive, effectively monitored and well-supported arrangements for quality assurance; the thoroughness and effectiveness of its validation and review arrangements; and the work of its Educational Development Office in relation to staff development and good practice in teaching and learning. The report also set out a number of recommendations for the Institute to consider, including reflecting further on how it assured the overall comparability of its patterns of degree classification with those of other HEIs; reflecting on the balance between central and devolved scrutiny; reviewing the information provided to students at programme level, to ensure consistency of good practice across the Institute; exploring further the possibility of additional mechanisms for the coordination of academic support; and reviewing existing student evaluation documentation, with a view to developing a consistent protocol for student evaluation of modules and programmes.
19 The SED prepared for the current audit summarised the ways in which Roehampton had responded to the 1996 report. It indicated that the recommendations and suggestions in the report had 'all been integrated into the systems and practices of Roehampton'. It also drew attention to the introduction of Quality and Standards Advisers (QASAs) to 'form a communication and mediation link between academic staff and central University quality and standards requirements', and to the enhancement of the role of the Educational Development Office (now the Roehampton Educational Development (RED) Centre). The current audit team was satisfied that Roehampton had responded effectively to the recommendations of the 1996 report. More detailed comment can be found elsewhere in this report (see below, paragraphs 27, 32 to 35, 64 and 71).
20 Since 1997, Roehampton has participated in eight HEFCE and Agency reviews of provision at subject level. Its performance in these reviews has been generally strong. The SED provided an overview of performance in relation to each aspect of assessment and outlined the modifications that had been made in response to the general findings.
21 The SED also outlined a range of key institutional developments since the previous audit, including the installation of a new Rector in 1999; academic restructurings in 2000 and 2002, resulting in the dissolution of faculties and the creation of the current eight-school structure; the establishment of the Federal University in 2000; the publication of the Strategic Plan, 2001-06 in 2001; a comprehensive revision of University committee structure in 2002; and the start of a review of academic support departments in 2003. The audit team explored aspects of some of these developments during the course of the audit.
Section 2: The audit investigations: institutional processes
The institution's view as expressed in the SED
22 The SED stated that 'Roehampton's processes are informed by a developmental approach to quality assurance and enhancement. Roehampton aims to promote a robust, self-critical and supportive environment in which systems and procedures are characterised by structural coherence, explicitness, effective communication and monitoring, and wide ownership of processes by the academic community. Through use of external reference points, Roehampton ensures that its systems remain appropriately positioned within the national framework of academic standards; and through increased involvement of the student body, that the student experience remains a prominent measure of their fitness for purpose'.
23 In discussing this statement with the audit team, Roehampton explained that its approach was developmental in that it acknowledged that developments could come from a number of sources, both internal and external, and that different processes might be appropriate for different areas. Staff who the team met reported that a developmental approach allowed staff the time to reflect, communicate and discuss matters, and that an emphasis was placed on self-evaluation.
24 The SED also stated that, as a constituent partner in the UniS Federation, Roehampton subscribed to a federal Policy Statement on quality and standards, which set out a number of principles, including: 'that educational provision and supporting systems and procedures should be subject to monitoring and evaluation in a commitment to continuous improvement; that, whenever appropriate and practicable, procedures and educational provision should be informed by the views of its students and other 'stakeholders' and by independent peer review; that learning and teaching activities and the commitment to quality assurance and enhancement should be supported by relevant and comprehensive provision for educational and staff development; that full recognisance is taken of quality external reference points...; and that each institution (in the Federation) will discharge their responsibilities for the standards of awards...by ensuring that regulatory frameworks, underlying principles, systems and procedures to safeguard and enhance standards and to assure and enhance quality are consistent (if not necessarily identical)'.
The institution's framework for managing quality and standards, including collaborative provision
25 The SED reported that responsibility for the assurance of quality and standards at institutional level rested with Roehampton's Senate, but was 'managed, strategically and operationally, except in matters of principle, through a committee structure; a group of bespoke academic posts; academic schools; and the RED Centre'.
26 In terms of committees, the major body with operational authority in respect of quality and standards is the Academic Management Committee (AMC), chaired by the Pro-Rector (Academic). There are five policy groups which report into AMC: the Curriculum Development Policy Group (CDPG), also chaired by the Pro-Rector (Academic), which is responsible for the initial approval of new programmes; the Quality and Standards Policy Group (QSPG), chaired by the Dean of Taught Programmes, which has responsibility for the formulation, development and monitoring of policies relating to quality and standards; the Learning and Teaching Policy Group (LTPG), chaired by the Dean of Learning Enhancement, which is responsible for matters relating to the enhancement of learning and teaching; the Collaborative Provision Policy Group (CPPG), chaired by the Dean of Collaborative Provision, which liaises with the associated institutions and is responsible for this aspect of Roehampton's work; and the Research Policy Group (RPG), chaired by the Director of Research, and which has responsibility for the management of research policy.
27 In addition to the academic posts responsible for chairing these committees and groups, Roehampton has appointed three QASAs who 'work closely with cognate groups of schools...and provide a conduit between University management and academic areas'. These postholders play a 'key role in quality assurance and enhancement at Roehampton'. There are also four University Academic Advisers who 'provide comprehensive academic advice at University level to students'.
28 The central policy groups are broadly replicated within each of the eight academic schools. All schools share an agreed core committee structure comprising a School Board, chaired by the Head of School, with four subcommittees: Planning and Resources; Marketing, Recruitment and Admissions (MRA); Learning and Teaching; and Research. Chairs of the School Learning and Teaching Sub-Committees and Research Sub-Committees sit on LTPG and RPG. School Learning and Teaching Committees have responsibility for both monitoring and developmental activities and their membership includes a member of the RED Centre. In addition, the QASAs are members of relevant central policy groups and are ex-officio members of school boards.
29 The establishment of the current school and associated committee structures took place in 2002, a year prior to the audit. The SED indicated that the devolution of 'significant responsibility to schools' through the common committee structure had strengthened 'structural coherence across Roehampton', and that the committee cross memberships had resulted in 'the enhancement of effective communication and information flow' between the centre and the schools. The evidence available to the audit team indicated that these claims were justified: the committee structures appeared to be working well and to be allowing a flow of information from schools to the centre and vice versa. The team concluded that Roehampton had established an appropriate institutional framework for the effective discharge of its responsibilities for managing quality and standards.
30 Roehampton's formal quality assurance procedures are documented in the Quality and Standards Handbook for Taught Programmes (the Handbook), produced by the RED Centre and available both in hard copy and on the internet. The Handbook includes a range of useful information, including clear documentation on the role of the QASAs, although the audit team did observe one instance, relating to assessment, where its procedures had not been observed (see below, paragraph 44).
The institution's intentions for the enhancement of quality and standards
31 The SED stated that Roehampton intended 'to review the effectiveness of its committee restructuring, particularly in its significance for the enhancement of quality and standards, and to evaluate the continuing effectiveness of its systems, after a year in which Schools have exercised greater responsibilities for quality assurance following the dissolution of faculties'. It also reported that the RED Centre and 'the operational effectiveness of the QASAs were scheduled for review in 2003-04'. In the view of the audit team, Roehampton's plans were appropriate, given the recent restructuring of both academic and committee structures.
Internal approval, monitoring and review processes
Programme approval
32 Roehampton's procedures for new programme approval are set out clearly in the Handbook, which also provides guidance on writing programme specifications, including referencing to The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ), subject benchmark statements and level descriptors. Annual school plans identify new programmes intended for development in the forthcoming four years, and the process starts with scrutiny of proposals at school level. Thereafter, a new programme approval form is signed by the programme convenor, Head of School, Director of Information Services and QASA, and presented for approval to CDPG. There are two approval methods: 'Method A', which is designed for a programme starting from first principles, and the 'abbreviated' 'Method B', which is used when the proposal is for a variation of an existing programme (including a change of site or delivery mode) and for undergraduate non-degree programmes.
33 Under Method A, the documentation undergoes two scrutiny processes before the programme is allowed to proceed to validation, first by a preliminary scrutiny panel, comprising the QASA and an academic from within the school but outside the discipline area, and then by an integrated approval panel, comprising the chair of Roehampton Approval Panel (UAP), the QASA and the programme convenor designate, with a member of the RED Centre acting as secretary. The purpose is to ensure that the documentation meets University requirements. The process includes discussion with all sections of Information Services via the relevant Academic Liaison Officer (see below, paragraph 75) to ensure that appropriate academic resources are available. When approval has been given, a validation is conducted by the UAP, the membership of which includes the QASA and two external subject specialists. The validation report is presented to AMC, which makes a formal recommendation to Senate.
34 While the audit team noted the rigorous check on documentation that formed part of Method A, it queried the nature of the role of the QASA within the process. Staff are encouraged to consult the relevant QASA before submitting a new proposal and the QASA also gives advice about appropriate preparation for the subsequent phases of the approval process. In addition, the QASA chairs the preliminary scrutiny panel and then sits on the UAP. It appeared to the team that the QASA had both a developmental and judgemental role within the process that could, potentially, jeopardise the independence of the UAP. Roehampton may wish to consider whether some further clarification of the role of the QASA in the programme approval process is desirable, in order to maintain the independence of UAPs.
35 Method B is less onerous, with no system of formal prior checking of documentation by panels. A UAP with one external adviser reports to AMC, which makes a formal recommendation to Senate. Permission for Method B approval can only be granted by CDPG or CPPG on behalf of AMC.
Annual monitoring
36 Roehampton's arrangements for annual monitoring are based on the production of a Programme Annual Review (PAR) by each programme convenor. PARs are approved by the relevant programme board, examined by a scrutiny committee of the school board, and then approved by the head of school for forward transmission to the QASA. The PARs are considered by QSPG, which presents a report to AMC. The Handbook provides a detailed template for PARs and all must include, as appendices, external examiners' reports, statistics, programme board minutes, a list of modifications approved during the year and a summary of module feedback.
Periodic review
37 All programmes are subject to periodic review every five years, unless AMC has given specific approval for a different period. The process of periodic review is identical in all respects to Method A programme approval, including the participation of two external peers, except that the programme team produces a SED for the event. The Handbook gives detailed guidance on the required content of the SED and supporting documentation, and the review report is presented to AMC, which then makes a recommendation to Senate.
38 In the case of both programme approval and periodic review activities, AMC receives composite overview reports, which identify all conditions and recommendations and any institutional action to be taken. These reports form the basis of the annual monitoring of the two processes. The audit team noted that these arrangements were consistent with the expectations of the section of the Code of practice for the assurance of academic quality and standards in higher education (Code of practice), published by the Agency relating to Section 7: programme approval, monitoring and review, and that Roehampton had formally checked its adherence to this section of the Code of practice on several occasions. In particular, it had considered whether Senate received enough information to enable it to monitor the outcomes of approval and review, and was satisfied that its arrangements were sufficient, given that AMC, acting under delegated authority from Senate, received the composite reports and that the Rector, who chaired Senate, signed off all programme documents.
External participation in internal review processes
39 Roehampton makes use of external participation in its internal procedures for programme approval (see above, paragraph 33) and programme review (see above, paragraph 37). The criteria for the appointment of external advisers are clearly documented in the Handbook. In the case of Method A programme approval and the periodic review of programmes, UAPs include two external panel members. In respect of Method B programme approval, the UAP includes one external member, who may, where the proposal is for a change of site or delivery mode of an existing programme, be a current external examiner. Dialogue with a Method B external adviser is conducted by correspondence, and the Handbook specifies the areas upon which they should be asked to comment. The adviser is required to submit a report to the Head of Quality and Standards Development, and the report is forwarded to the programme convenor designate, who is expected to provide a detailed response. From the evidence available to it, both at institutional level and through the DATs, the audit team was satisfied that Roehampton was making strong and scrupulous use of external participation in its internal processes.
External examiners and their reports
40 Roehampton has produced a Guide to Good Practice (the Guide) for external examiners for taught programmes based on, and broadly aligned with, the relevant section of the Code of practice. The Guide is included in the Handbook and made widely available to staff. It states that Roehampton 'considers its arrangements for external examining to be an integral part of its quality assurance system' and expresses its commitment 'to ensuring the presence of at least one independent and appropriately qualified external voice on the examining team for each University award'. Roehampton requires its external examiners 'to assist in ensuring and enhancing the academic integrity, inherent fairness and public currency of its awards' and 'seeks to facilitate this process through a coherent framework enabling external examiners to comment freely...on all major issues associated with setting and maintaining the academic standards of its awards, and with the assessment processes which determine who shall receive them'.
41 The Guide contains clear details of the process for the selection and appointment of external examiners and the criteria against which their suitability will be assessed. Nominations are initiated by the relevant programme board, scrutinised and approved within the appropriate school, and finally checked against the criteria by the Dean of Taught Programmes, before an appointment is confirmed by the Pro-Rector (Academic) on behalf of the Federal Senate. Nominations are noted by AMC and sent to Senate for ratification. Every taught programme is required to have at least one external examiner who serves for three years, with the automatic option of a fourth.
42 External examiners are required to report to Roehampton on a standard form, which is returned to the RED Centre. Roehampton is seeking to encourage external examiners to complete the form electronically to facilitate compliance with requirements as set out in HEFCE's document 03/51, Information on quality and standards in higher eduaction: Final guidance. The reports are considered at programme, school and university level and form part of the documentation prepared for programme approval and review. Responses to the reports by programme boards are also included in PARs (see above, paragraph 36), copies of which are sent to the external examiners. All reports are scrutinised by the Dean of Taught Programmes and the Head of Quality and Standards Development in the RED Centre, who prepare a composite report for consideration by QSPG and AMC before submission to Senate and the Federal Senate. The composite report comprises five sections: a commentary on the effectiveness of administrative processes; a brief overview of matters raised in each section; examples of good practice; a summary of matters raised, with suggested action points for Roehampton; and Roehampton's register of external examiners for the year.
43 The audit team explored the robustness of Roehampton's external examining arrangements in three main ways: by comparing the Guide with the relevant sections of the Code of practice; by assessing the depth of understanding of the system among staff; and by considering the effectiveness of responses to external examiners' reports. In so doing, the team noted that QSPG was currently considering Roehampton's continuing adherence to the Code of practice in relation to the information sent to external examiners and in relation to the award of Certificates of Higher Education in cases where external examiner scrutiny of level 1 work was not currently required. In general, the team's analysis of the evidence available indicated that staff had an appropriate understanding of the procedures, coupled with a clear willingness to engage constructively with comments made in external examiners' reports, and that timely action was taken in response.
44 In discussion, the audit team learnt of one instance where the assessment and external examining procedures set out in the Handbook had not been observed and students had taken an examination that had not been internally scrutinised nor approved by the external examiner. The team was informed that, while the procedures stated that draft level 2, level 3 and M (masters) level examination papers should be sent to external examiners where appropriate, a draft examination paper would not be sent for external scrutiny if the external examiner did not wish to see it. Roehampton is advised to take steps to ensure that its assessment procedures in respect of the internal scrutiny of examination papers are always observed and that there is always prior external scrutiny of examination papers leading to an award.
45 In addition to its external examining arrangements, Roehampton operates a system of external observation of examination boards, through which an Observer is appointed to the final undergraduate and postgraduate Awards and Progression Boards 'in an advisory capacity'. The remit of these Boards is to agree degree classifications and distinctions, confirm results, and approve or decline condonations and programme terminations. Observers attend only the main Awards Board each year and are asked 'to observe procedures' and to advise on their appropriateness, through an annual report for AMC.
46 The audit team was provided with the detailed reports from the Observers for the last two years and the associated University responses. It noted the positive comments of the Observers on the care and attention with which the Boards approached their duties, but also the range of concerns raised, particularly at undergraduate level. While it was evident to the team that there had been a range of specific factors which had affected the operation of the final undergraduate Board during the preceding year, the team considered that there was currently a lack of clarity about the role of Observers and particularly whether they were present solely to observe and to advise in retrospect, as the terms of reference suggested, or were able to satisfy themselves on behalf of Roehampton and its external examiners that Roehampton's rules and regulations were being appropriately applied by the Board. In its discussions with senior staff, the team was informed that the Observer had full speaking rights and their views would be invited by the Chair in the event of any contentious issue.
47 The audit team was also concerned that Awards Boards were taking place twice a year with no external observer present, albeit with fewer students under consideration. In the view of the team, this practice was inconsistent with the expectation of the Code of practice that external examiners should be required to report on the extent to which processes for the determination of awards are sound and have been fairly conducted. Roehampton is advised to ensure that there is always external representation on Awards Boards, and to review the role of Observer, considering in particular whether they should explicitly have the ability to seek clarification during the Board, when decisions are being made.
External reference points
The Code of practice
48 In the SED, Roehampton stated it had reviewed its practice against each section of the Code of practice as they were made available. Written assessments were considered by the then Academic Standards Committee (now AMC) in spring 2001 and the 'perceived gaps' between the precepts of the Code of practice and Roehampton's current practice were then addressed. At the time of the audit, Roehampton was reviewing systematically the progress made since 2001 to determine whether there were matters outstanding and was keeping abreast of other developments which had an impact on areas covered by the Code of practice. The SED provided a detailed account of Roehampton's response to each section of the Code of practice, indicating where work was in progress.
The FHEQ
49 The SED reported that Roehampton had debated the FHEQ in 2001, and had agreed to make a number of modifications to its awards structure and regulations, setting a timetable of action for broad adherence by September 2003. The modifications related to differentiation between levels 2 and 3 (previously Roehampton had not sought to distinguish between learning outcomes at these levels) and revising the nomenclature of some awards. As a result, a revised set of regulations for programmes had been developed and implemented for students starting their studies from September 2002. Thus, at the time of the audit, Roehampton was operating to two sets of regulations for some programmes, but with guidance set down to differentiate between affected students. The SED identified two areas for further enhancement: the development of learning outcomes for intermediate exit points on some awards; and the harmonisation of all existing MA learning outcomes to ensure that they conformed to the language of the qualification descriptors. Work in these areas was in progress at the time of the audit.
Subject benchmark statements
50 The SED stated that the use of subject benchmark statements was 'firmly established'. Programme specifications include a statement about the relationship of learning outcomes to the relevant benchmark and the statement is discussed at validation and review. A recent survey had asked heads of school how the introduction of subject benchmark statements had informed planning and the responses had suggested that Roehampton's expectations about the use of the statements was 'well understood and appropriately addressed within programmes'.
51 The audit team pursued Roehampton's application of these aspects of the academic infrastructure through its consideration of relevant policies and procedures, its scrutiny of documentation, and in discussion with staff at all levels. For each of the DATs, the team was provided with a commentary on how the relevant programmes reflected the expectations of the Code of practice, the FHEQ and subject benchmark statements. The team was satisfied that Roehampton was taking appropriate and timely action in relation to these matters, in respect of both its internal and its collaborative provision, and that staff had an understanding of the academic infrastructure appropriate to their roles and experience. The team noted areas where further attention to the precepts of the Code of practice was required (see above, paragraphs 44 and 47).
Programme-level review and accreditation by external agencies
52 In the six years preceding the audit, Roehampton participated in eight HEFCE and Agency reviews of provision at subject level. Its performance in these reviews has been generally strong. At central level, the reports are received by AMC, with QSPG undertaking the substantive analysis and keeping AMC informed through its minutes. At programme level, each programme board is expected to address subject-specific matters, identify appropriate action and report on progress through the PAR. School boards also consider reports and programme board responses.
53 The SED provided an overview of performance in relation to each aspect of provision and gave examples of where Roehampton had taken action in respect of recommendations contained in the reports. It reported, for instance, that Roehampton had been concerned about a pattern of 'dropped points' in the 'Teaching, Learning and Assessment' aspect and had sought to enhance its provision through staff development, the issuing of initial guidance on the production of assessment criteria, and close monitoring of the provision of student feedback.
54 Since 1997, Roehampton has had four accreditation visits from professional, statutory and regulatory bodies. The SED indicated that none of the visits had reported any areas for concern or imposed any requirements. It stated that until 2001-02, such visits were arranged and conducted at a local level by the schools, but that a note of guidance had subsequently been approved by AMC 'to ensure that Roehampton was appropriately involved in these visits by approving documentation, participating in the visit at senior University level, and formally receiving and approving reports and responses'. Outcomes of accreditation visits are reported to QSPG and through the minutes of QSPG to AMC.
Student representation at operational and institutional level
55 The SED described as 'a central feature of student life at Roehampton' the opportunity provided 'at all levels for students to express their views about the nature of their academic and academic-related activities, and in particular to evaluate the quality of their learning experiences in a regular and systematic way'. This opportunity is currently provided through a system of student representation on committees and through 'written and oral evaluation procedures'.
56 Students are represented on institutional level committees by the SU, and SU officers who met the audit team confirmed that they felt fully involved in committee debates, giving examples to support this view. At school and programme levels, students are represented on programme and school boards, where the matters raised may be addressed directly or communicated to another part of Roehampton, and are fed into the PAR. The SED reported that a Programme Representatives Scheme was introduced by the SU in 2002 and piloted during 2002-03, with the aim of improving the effectiveness of student representatives. Those who met the team confirmed that the processes of representation were working as intended, although some had only recently been approached by the SU to be trained.
57 Roehampton is satisfied that its current arrangements for student representation are effective and provide a good base for future planned advancement, a view that the audit team was able to confirm in its discussions with students, in both institutional-level meetings and during the DATs. The team was provided with a number of examples of changes that had occurred as a result of student feedback. These included changes to modules and module selection, the pattern and nature of assessment, access to book stock and computer resources. It was clear that the students valued all of the means available to them, formal and informal, for expressing their views, and through which they could make a positive contribution to Roehampton.
Feedback from students, graduates and employers
58 As a constituent partner in the UniS Federation, Roehampton subscribes to the principle 'that, whenever appropriate and practicable, procedures and educational provision should be informed by the views of its students and other 'stakeholders''. The SED outlined a range of both formal and informal approaches to the collection of feedback from students. Formally, taught modules are evaluated by students during teaching, usually through the completion of evaluation forms which are analysed after the examination process has been completed. The exact nature and content of such forms is the responsibility of each programme team, as is the way in which the matters raised are taken forward. Research students are asked to provide 'written feedback on their progress and experience' as part of the annual reporting system. In addition, meetings with current and former students are a key part of the periodic review process: 'the agenda for such meetings is open and review panel members may ask students anything they wish'. More informally, the RED Centre, with the support of the schools and normally in relation to specific issues, convenes 'occasional focus group meetings...in order to obtain a more interactive and qualitatively richer understanding of student perspectives'.
59 In respect of feedback from employers, Roehampton's position reflects in its development and growth in recent years. The audit team was provided with examples of good links in certain subject areas, particularly education, where the links are long-established, and those areas where alignment of the curriculum and learning resources is a requisite for professional accreditation of programmes. The team heard that for some other areas it was not easy to identify a typical employer because of the diverse opportunities open to graduates. As a result, achieving any meaningful interaction with employers was viewed as somewhat problematic. It would be desirable for Roehampton to give further consideration to the means by which it might ensure the systematic involvement of employers across the institution drawing on the good practice that already exists in some subject areas.
Progression and completion statistics
60 The SED presented an analysis of its admission, progression and award statistics, based on Roehampton's own data and that provided by external sources such as UCAS and HEFCE. The SED stated that Roehampton had kept track of its completion and continuation rates relative to its published benchmark, and that this analysis had shown that it was consistently at or close to the benchmark for institutions of its type.
61 Roehampton has recently introduced guidance on the use of statistical information in PARs, covering admissions data, entry qualifications, student demography, progression, attainment and award classifications. First destination statistics are not included, although they are made available to programme convenors. Statistical information is readily available on Roehampton's intranet. Staff who met the audit team spoke positively about the recent improvement in the quality of the statistical data available to them and the support and guidance on its interpretation provided by the Head of Planning. The team's scrutiny of the PARs available to it suggested that practice in relation to the effective use of statistics at programme level was somewhat variable, but developing (see below, paragraph 97).
62 In response to the recommendations of the 1996 audit report (see above, paragraph 19) Roehampton reviews annually the comparative statistics for degree award profiles across the institutions of the Federal University. The audit team heard from staff that Roehampton was also beginning to monitor the comparative progress of international students through its International Student Steering Group. While noting this developing good practice, the team saw no additional evidence of institutional level monitoring of other key statistical information by the principal quality committees. In particular, there appeared to be little consideration of comparative performance data relating to Roehampton's own students across different subject areas or between single and combined honours awards. It would be desirable for Roehampton to consider how it might further develop its mechanisms for keeping an oversight of comparative progression, retention and attainment statistics across the institution.
Assurance of the quality of teaching staff, appointment, appraisal and reward
63 The SED provided a comprehensive commentary on Roehampton's arrangements for staff appointment, appraisal and reward. For all but temporary positions, the process of staff appointment is overseen by the Human Resources Department. Roehampton's policy is to advertise all posts externally (unless there are clear special circumstances), in order 'to attract the best and most diverse field of potential candidates for all substantive positions'. There are guidelines for the conduct of both short listing and interviewing, and training is provided for those involved in the selection process, in particular the chairs of appointment panels. Once appointed, most staff have a probationary period of one year, at the end of which a written report is produced.
64 Induction arrangements for new staff are clearly documented and involve induction at both institutional and school/department levels. Staff informed the audit team that Roehampton had a well-developed mentoring scheme at school level, available to both full and part-time staff, and a number of staff reported that they had found the support of their mentors valuable. Staff without teaching experience are expected to take the Institute for Learning and Teaching in Higher Education Postgraduate Certificate in Learning and Teaching in Higher Education, delivered through the RED Centre. The Centre also provides specific information and support for visiting lecturers, following some pilot activity in 2002-03.
65 Appraisal schemes for academic and related staff are well-established. The audit team was informed that Roehampton was in the process of moving to an annual cycle of appraisal, from a biennial appraisal with a mid-period review. This development was felt to be necessary to align appraisal with the development of annual planning cycles at both institutional and school levels. Staff who met the team confirmed that appraisal was a live and working system, and provided several examples of its positive effects.
66 The SED indicated that staff reward was a matter of concern for Roehampton, because of its location in an area where the cost of living was high; funding constraints meant that it needed 'both to press for additional resources to address pay levels and to be imaginative in its use of non monetary reward by further development of conditions of service and benefits'. The SED reported that Roehampton was using funds made available under the HEFCE 01/16 initiative, Rewarding and developing staff in higher education, to increase training and development budgets and extend fee waivers for staff taking qualifications. It also planned to provide benefits for staff by extending existing schemes relating to, for example, special leave and flexible working.
67 The SED was clear that Roehampton intended to keep its policies for recruitment and retention under active review, to ensure the quality of selection decisions. As a result of its discussions with staff, the audit team was satisfied that Roehampton's current arrangements for appointment, appraisal and reward were comprehensive, and were applied rigorously and fairly, with a number of enhancements planned.
Assurance of the quality of teaching through staff support and development
68 The SED stated that each year Roehampton set aside top-sliced funds to support the cost of staff training and development. Responsibility for overseeing matters relating to staff development rests with the Staff Development and Training Committee. The Committee has specific responsibility for the development and planned implementation of policy, for establishing guidelines and setting priorities, and for agreeing and monitoring Roehampton's annual training and development plan.
69 Activities for academic and educational development focus on the opportunities for staff through a programme of professional development. The programme includes the Postgraduate Certificate for new academic staff, the provision of support for visiting lecturers (see above, paragraph 64), and other workshops and master classes. The RED Centre and the Research Office are together piloting a development programme for those new or relatively new to the supervision of research students. Principal budget holders are expected to make available sufficient funds to meet the costs of activities identified in training and development plans for their area of responsibility; such activities include awaydays, conference attendance, and sabbaticals or study leave. Heads of school are required to draw together evaluation data to inform their school plans and to report annually to the Staff Development and Training Committee. This information contributes to the Committee's next round of annual planning.
70 Roehampton has for a number of years been using the Investors in People (IiP) Standard 'as a framework to improve its approach to the development of all its staff in line with institutional objectives'. The SED provided examples of actions taken as a result of work towards IiP, including staff involvement in developing the Strategic Plan; developments in staff induction and appraisal; increasing the staff development budget; and review of internal communication, resulting in the re-launch of the staff newsletter and development of a 'Staffzone' web site. Most parts of Roehampton have now achieved IiP status, and the target for the remaining areas is 2004.
71 Through its enquiries and its discussions with staff at all levels, the audit team found evidence of active engagement with staff development. In particular, the team found that the contribution made by the RED Centre, and its positive approach to providing support to staff and academic teams, was a feature of good practice. As it takes forward its work in relation to staff development, Roehampton may wish to consider the merits of establishing its arrangements for the peer observation of teaching on a more formal footing. It would also be desirable for Roehampton to ensure that all postgraduate students teaching on its programmes are appropriately prepared for their role, given that such students are not currently required to participate in the training programme it provides.
Assurance of the quality of teaching delivered through distributed and distance methods
72 The SED stated that internet-based learning is in its infancy and at present mainly located on one School. The SED made reference to the Agency's Guidelines on the quality assurance of distance learning, indicating that they were not wholly relevant to Roehampton's own distance-learning provision, which is 'at present not franchised and uses only Roehampton staff'. The SED reported that Roehampton had sought to assure quality by piloting modules within distance degrees and using an external expert to evaluate the pilot. Once established, distance degrees are subject to the normal periodic review process, which includes consideration of 'registration procedures, learning materials, student support mechanisms and assessment'. The audit team noted, however, that at the initial university approval event for a change of mode to delivery to distance learning, student support mechanisms were not specifically included in the Handbook as an area of consideration for UAPs, nor was there a requirement that UAP members had experience in the delivery of distance-learning provision. Given the particular support needs of students studying at a distance, Roehampton is advised to review the appropriateness of the current programme approval/review procedures for a change in mode of delivery to distance learning.
73 One of the posts in the RED Centre is a Head of Distributed Learning Development, who reports to the Director of Educational Development. The SED reported that distributed learning through the use of the virtual learning environment WebCT was reviewed the same way as other learning methods, through PARs and periodic review. Distributed provision is developed and monitored by the Learning and Teaching Technology Working Group, which reports to LTPG and to the Communication and Information Technology Sub Committee of the Strategic Planning and Resources Committee (SPARC). Roehampton and UniS have recently agreed to jointly purchase the next generation product, WebCT Vista, a development that is to be 'monitored closely in 2003-05' and a 'substantial report...presented to both institutions to inform developments'.
Learning support resources
74 Oversight of Roehampton's learning resources is maintained by SPARC, chaired by the Rector and with a direct reporting line to Senate. The SED summarised the range of resources available and the ways in which Roehampton sought to ensure that they were of appropriate quality. It also outlined Roehampton's plans for expanding the physical environment, through the provision of additional halls of residence, a learning and teaching building which will include a dedicated Postgraduate Centre, and a specialist dance/physical education facility.
75 Roehampton's library is based on two sites and holds books, journals, videos, slides and e-resources, as well as special collections. Liaison with the schools is provided by academic liaison officers, who attend school and programme boards, at which students may comment on library services, and provide advice on a range of matters. Supported by subject enquiry officers, they are also responsible for holding induction and training sessions, advising new researchers, and developing new workbooks and e-guides for their subject area. Together they form the Academic Liaison Team. The SED reported that this integrated approach began in 2002 and a number of areas of future enhancement had been identified, including the development of e-resources, using WebCT in liaison with the RED Centre, and the acquisition of specialist collections. Students who met the audit team were satisfied with the library facilities available to them and made particular mention of the helpfulness of staff.
76 Roehampton reviewed its information technology (IT) support services in 1998 and produced a Communications and Information Technology (C&IT) Strategy the following year. As a result, the C&IT staff was co-located in the Learning Resources Centre, in order to provide 'a better skilled and far more integrated IT team'. Since then the focus for development has been student resources and the teaching and learning infrastructure. The SED reported that Roehampton now had a new network infrastructure operating to sector standards, a clear policy on hardware and software acquisitions and standards, and that ongoing continuing review of the integrated C&IT strategy continued to move the area forward. Roehampton also integrated its Reprographics, Media and Television services in 1999, again with a view to providing support for institutional needs in a resource-efficient manner. The SED stated that the formulation of the learning resources strategy was giving 'additional direction to the activities of these services'.
77 Roehampton is expanding its internet-based learning facilities and recognises the potential contribution that such resources can provide to the learning experience (see above, paragraph 73). It also recognises that studying on-line is an emerging key skill for students and, in recognition, is taking a measured approach to such developments. Through its discussions with staff and students, the audit team was satisfied that Roehampton had established mechanisms for ensuring that its learning resources provision was appropriate, for identifying areas for improvements, and for taking relevant debates and recommendations through the relevant committees.
Academic guidance, support and supervision
78 Roehampton's arrangements for student support and guidance have received maximum grades in all but two of the HEFCE and Agency subject reviews undertaken since the previous audit. The SED reported that, for students following taught programmes, academic guidance, support and supervision were provided at two levels: school and University. This structure is based on the assumption that generally, students will find the help they need at school level but that, should there be a requirement for more specialist support, they can be referred to appropriate staff at institutional level. In particular, Roehampton has a team of four Academic Advisers who are supported by Registry staff and can provide comprehensive advice on matters such as academic regulation, credit accumulation and transfer, and study abroad. English language support for international students is offered by the English for Academic Purposes programme, and support in writing skills is offered for all students by the RED Centre. The support and advice given, at both institutional and school level, is supported by a range of handbooks and internet-based information (see below, paragraphs 81, 102, 114, 115, 127 and 136).
79 The provision of academic tutorial guidance and support is the responsibility of schools within broad 'non-prescriptive Institutional Guidance'. Such advice and support is provided by programme convenors, academic tutors and other academic support staff. In particular, programme teams are responsible for providing feedback to students on their academic performance, including both formative and summative feedback on assessment. Heads of School are responsible for keeping an overview of advisory arrangements, and for ensuring that all members of programme teams are available at published times. Additional support on administrative matters is provided by staff in school offices.
80 Students who met the audit team, at institutional-level meetings and through the DATs, spoke positively about the range and quality of the academic support available to them. On the basis of the evidence available, it appeared to the team that programme teams across Roehampton had developed a range of local but effective practices for communicating with students, and that there had been an increase in internet-based systems, the latter of particular benefit to part-time students. In the view of the team, the accessibility of academic staff to students and the positive way in which they respond to student needs is a feature of good practice (see below, paragraph 123).
81 The SED reported that in 2001 Roehampton had reviewed its arrangements for the support and supervision of research students against the relevant section of the Code of practice and had found them to be consistent. Since then a range of enhancements has been introduced. The current arrangements are set out in the Quality and Standards Handbook for Research Degree Programmes, a comprehensive resource for all those involved with research degrees which includes Roehampton's own 'Code of Good Research Practice'. An Integrated Research Strategy is in place, and a designated Postgraduate Centre is planned to open in 2005. There is already a dedicated computer suite for research students.
82 Project and supervisory arrangements for research students require the approval of Roehampton's Research Degrees Board (RDB). All students belong to a Research Cluster or a Research Centre and have at least four tutorials with their supervisory team, comprising a Director of Studies and at least one Co-supervisor, each year. The SED reported that 'most research students will secure all the academic advice and support they need' from this team, and that a 'Learning Compact for Research Students' 'establishes the rights and responsibilities of students and supervisors within the supervisory relationship'. Should further advice be required, students may seek it from Roehampton's Director of Research and the Research Office. The Office also provides generic research methods courses, which are supported by more specific courses within schools. Annual reports on the progress of each research student are considered at school level, through a process monitored by RDB. The audit team considered that the comprehensive support for, and monitoring of, research degree students was a feature of good practice.
Personal support and guidance
83 Student support services are coordinated by a pro-rector and 'implemented, as appropriate, by the colleges, Roehampton and the SU'. The services provided include accommodation, employment and careers, childcare, welfare, counselling, medical, finance, chaplaincy and support for international students. A Student Services Policy Group, reporting to SPARC, provides 'the forum for information sharing and debate' on welfare issues. The SED also drew attention to the importance of the collegiate structure in the provision of student support. College deputy principals have key responsibility for the delivery of support services, in that they line manage several of the heads of services and are 'the first port of call for many students'. They also work closely with the SU, which has a deputy president assigned to each college. Students who met the audit team were generally complimentary about the full range of personal support services available to them.
84 The Counselling Service, available to all members of Roehampton, has a core staff of full-time counsellors, plus volunteer placement counsellors who deal with less complex cases under weekly supervision. The SED reported that in recent years the Service had expanded to enable it to provide a more holistic experience, through the provision of complementary therapies. It also provides a comprehensive range of training courses for staff and students within Roehampton.
85 In 2001, Roehampton decided to locate Disability Support in the RED Centre to raise the profile of disability matters. New staff were appointed and a team of learning support assistants was created to provide non-medical support to students with disability which provides, for example, signers and note takers. The RED Centre also provides support for students with dyslexia and other forms of learning difficulty. The SED reported that Roehampton had assessed and enhanced its current practices in the light of the relevant section of the Code of practice, and was in the process of working through an action plan to ensure that all practices across Roehampton were compliant with the Special Educational Needs and Disability Act 2001 legislation, and that all staff were aware of their responsibilities in this area. In the view of the audit team, the academic and personal support services provided to students by the RED Centre were a feature of good practice (see below, paragraph 103).
86 The SED reported that adherence to the relevant section of the Code of practice, and 'emulating the good practice outlined in the Harris Review of 2000', had been 'at the top of the agenda' of the Employment and Careers Service since 2001. As a result, student employability and incorporating career management skills into the curriculum had been 'at the forefront of its work'. The audit team noted the range of achievements of the Service since this period, and in particular the development of Careers On-line, and the work towards MATRIX recognition.
87 Roehampton is aware that its planned growth in student numbers and continuing initiatives to widen participation are likely to put increasing pressure on student support services. It recognises the need for strategic planning and coordination, and is keeping its provision in this area under active review.
Collaborative provision
88 Roehampton has a small amount of collaborative provision, delivered mainly through its associated institutions. While some of the institutions deliver programmes validated by Roehampton, in others Roehampton's own staff deliver the programmes. The SED reported that Roehampton had reviewed all of its collaborative partnerships at the time of federation, to ensure that its arrangements harmonised with those of UniS. The review indicated that most of the existing links, 'while academically sound, were too small in terms of student numbers to continue to be viable'. As a result, Roehampton reduced its associations to the Westminster Pastoral Foundation, which now delivers four MA programmes, and a small number of School Centred Initial Teacher Training consortia offering PGCE programmes. Since then, Roehampton has also validated a Foundation degree with a further education college and some further partnerships are planned. In addition, there are plans to increase the number of partner organisations through which Roehampton delivers its programmes off-site.
89 While Senate has delegated to AMC authority in all matters relating to quality and standards, including collaborative provision, its authority in operational terms has been further delegated to CPPG, which reports and makes recommendations to AMC. CPPG was established in September 2002 and a Dean of Collaborative Provision was appointed in the same month. The SED claimed that both of these initiatives had 'significantly strengthened Roehampton's framework for managing this area' of its work and its adherence to the relevant section of the Code of practice.
90 The SED also reported that off-site delivery of Roehampton's programmes had previously been 'a small-scale activity handled largely at school level'. However, Roehampton had recognised potential risks associated with the possibility of a number of ventures being negotiated overseas. As a result, CPPG now takes 'a regulatory interest in such ventures', and the Handbook contains detailed procedures in respect of the establishment, validation and continuing monitoring of all forms of collaborative provision. It was clear to the audit team from the minutes of CPPG that this new Group was playing a strategic role in the management of collaborative provision, in addition to being kept closely informed of new initiatives and conducting detailed examination of matters such as memoranda of agreement.
91 The approval process for collaborative provision, set out in the Handbook, requires the completion of a risk assessment, initial consideration of the proposal by CPPG, a visit to the proposed partner institution, and finally a University validation held at the partner premises. For associated institutions, AMC identifies a University moderator who works with the programme team to develop the documentation. The validation process followed is Method A (see above, paragraph 33), although the QASA is not a member of the UAP panel. The validation report is initially received by CPPG for onward transmission to AMC and the Federal Board for the Associated Institutions, which is responsible for making a recommendation to the Federal Senate. There is a separate procedure for modifications to existing programmes.
92 The audit team noted that, in the event of a decision by AMC that a collaboration should take the form of off-site or distance delivery of an existing programme, either in the UK or overseas, the lighter Method B validation process was used. Within this context, the team noted that the risk assessment tool used by Roehampton did not make specific reference to access to learning resources, nor did the procedures detailed in the Handbook make mention of the processes for the approval of staff who may teach on the programmes delivered off-site. In discussion, the team heard that in fact only on one occasion had an application for off-site delivery been through Method B approval; in other cases, after the risk assessment and visits to the site, the new site had been approved through the modifications procedure. The team saw no evidence that this approach had threatened the quality or standards of Roehampton's programmes. However, in view of its stated aim of expanding off-site delivery, Roehampton is advised to ensure that adequate consideration is given to the scrutiny of collaborative partners and sites where its programmes are to be taught, to include the proposed arrangements for access to learning resources, student support and the approval of teaching staff. In so doing, Roehampton is also advised to ensure that the procedures set out in the Handbook are always observed.
93 Annual monitoring of collaborative provision is undertaken in broadly the same way as that of internal provision. A PAR is produced for each programme, using the standard template, but is supplemented by a report by the appointed University moderator. In the view of the audit team, there was some difference in the quality of the PARs for internal provision and those for collaborative provision. The former were, on the whole, very detailed and showed capacity for self evaluation; the latter were much more variable in detail, as were the accompanying moderators' reports. In addition, while its terms of reference state that CPPG 'receives and considers the external examiners' reports, moderators' reports and annual programme reviews' from associated institutions and where there is off-site delivery, the team could find no evidence from CPPG's minutes that it had, since its inception, received such PARs or external examiners' reports (although it had regularly received moderators' reports). Roehampton is advised to address this matter, to ensure that, in accordance with its terms of reference, CPPG is able to satisfy itself about the maintenance of quality and standards across all of Roehampton's collaborative provision.
Section 3: The audit investigations: discipline audit trail
Discipline audit trail
94 In each of the selected DATs, appropriate members of the audit team met staff and students to discuss the provision, studied a sample of assessed student work, saw examples of learning resource materials, and studied internal review and other documentation The team's findings are as follows.
Business Studies
95 The DAT was based on two programmes within the School of Business and Social Sciences: the BA/BSc Business Studies and the MSc/PgDip in Voluntary Action Management. In October 2003 the BA/BSc Business Studies provision had 423 full-time equivalents (FTEs), almost all full-time students, of which 89 were taking combined honours programmes. The Voluntary Action Management programme had 10 full-time students.
96 The DAT was supported by a DSED prepared for the audit. The DSED included a general overview of provision within the School; a SED covering the BA/BSc Business Studies, produced for the purposes of an internal periodic review in 2003; a SED for Voluntary Action Management produced for the purposes of the DAT; and programme specifications for both programmes. The programme specifications made appropriate reference to the relevant subject benchmark statements and that for Business Studies also made reference to the FHEQ. The accompanying documentation indicated clearly how both programmes were aligned with the FHEQ and the Code of practice.
97 The DSED provided progression and completion data for the programmes, although the statistics were not broken down by category of student. The audit team noted that the PARs for the previous two years made limited use of such data, and that both PARs included a request that the Planning Office made data 'more usable so that time could be spent on commentary'. Within this context, the team noted that the SED for Business Studies identified a high failure rate in year one, and heard from the programme team that the problem was failure at first attempt in examinations, although failure rates after resit opportunities were not a cause for concern. The RED Centre had used focus groups to investigate the matter, in an attempt to identify the categories of students more likely to fail, and a range of measures had been introduced during the current year to help overcome the problem. As work continues in this area, it may be appropriate for Roehampton to give further consideration to how data might be used in more detailed and sophisticated analyses of student achievement (see above, paragraph 61).
98 The audit team had access to recent PARs and noted that they were informed by feedback from students, external examiners and staff. Staff informed the team that, as part of the review process, programme convenors invited staff to comment and to propose matters for inclusion in the PAR, but that there was also a continuing process of monitoring, through regular dialogue between staff and through programme boards. On the basis of the evidence available, the team was satisfied that the monitoring process was working as Roehampton intended.
99 The audit team had access to recent external examiners' reports on the programmes. It noted that the Voluntary Action Management reports were complimentary, with no significant matters raised. The Business Studies reports were also generally complimentary, especially for Economics modules, and the quality of feedback to students received particular praise. One report raised concerns about the quality of some final-year project work, a matter addressed in the subsequent PAR. Another report suggested a need for additional support for students with lower entry qualifications and the team heard evidence that this too had been followed up. The team noted a couple of instances where an external examiner had repeated a request in successive years, suggesting a need for Roehampton to be vigilant in ensuring that all matters raised by external examiners were followed up by programme teams. In general, however, the team was satisfied that appropriate responses were made to external examiners' reports.
100 The programmes are assessed by a combination of coursework and unseen examinations. The SED indicated that some concerns had been expressed, by students and by the relevant UAP, about the extensive use of examinations in the MSc programme. It provided a rationale for the retention of the current diet, drawing attention to the supportive comments of the external examiner. It was clear to the audit team that Roehampton's procedures for assessment were being followed for the programmes. Students who met the team confirmed that they were clear about what was required of them and that they were given helpful and generally timely feedback on their performance. The team noted that visiting lecturers worked with permanent staff in the assessment process and that appropriate support was provided for new visiting lecturers. The School uses a pro forma for the marking of coursework. In the view of the team, there would be merit in reviewing the design of this helpful document, with a view to considering whether the second-marker's report might usefully place a greater emphasis on moderation of the level and consistency of the first marker, rather than on the second-marking of individual scripts.
101 The audit team reviewed a range of assessed student work from all levels of the programmes, including coursework, examination work and projects/dissertations. The team was satisfied that the nature of the assessment and standard of student achievement met the expectations of the programme specifications and the subject benchmark statements, and was appropriate to the relevant awards and their location within the FHEQ.
102 The audit team was provided with copies of relevant student handbooks. The module handbooks gave clear information about the objectives, the assessments and the assessment criteria, although some lacked references to journals or web sites. Students who met the team reported that the handbooks were very helpful. Within this context, however, the team noted what appeared to be an important omission from some of the documentation. The DSED for Business Studies indicated that, as a result of poor quality work in the final-year project 'at the tail end of the distribution', a new condition of entry to the project had been set for 2002-03; students were required to obtain at least 50 per cent in a specified second-year prerequisite module, Business Research. Students who failed to meet this condition were able to take an alternative module, an extended essay, in their final year, and were not prevented from obtaining a single honours Business Studies degree. While this new arrangement was mentioned in the module handbooks for both Business Research and the Business Project, the team could find no reference to it in the Business Studies Student Handbook, which continues to state that the Business Project was compulsory for single honours students. The programme team will no doubt wish to clarify this matter in all of its documentation.
103 Staff informed the audit team that the adequacy of learning resources was monitored through student evaluation forms. The DSED identified a need for additional computer provision, being addressed by the purchase of laptops, although students who met the team reported that access to computers was not a problem. They also praised the provision of e-journals and the training given in the use of e-resources, although undergraduates reported some recent problems when the WebCT system, used for a significant number of modules, had crashed. Staff advised the team that these problems had been overcome. Students also confirmed that the library staff were very helpful and that the designated librarian system worked well. The team heard that there were occasional difficulties in accessing undergraduate textbooks, although staff gave assurances that the provision of books was at the standard level for the number of students on each module. All students spoke highly of the 'excellent' support available from the RED Centre (see above, paragraph 85).
104 In terms of student support, students praised the approachability of staff (although there were occasional problems of availability), and said that most visiting lecturers could be accessed via email. They spoke highly of the various support services available, mentioning in particular Disability Support and the Employment and Careers Service. The students also spoke positively of a strong feeling of community experienced on campus. It was clear to the audit team that Roehampton's student support systems were effective and working as intended.
105 Consistent with Roehampton's expectations, student feedback is obtained through module evaluation forms, programme boards, focus groups, and also a 'programme review of finalists', held when final projects are returned. Module evaluation forms are completed at the end of each module and are used by module convenors to prepare a summary proforma for each module. The audit team was provided with examples of the pro forma for the Business Studies programme and noted that the convenors' summaries of evaluation were accompanied by planned changes for the next delivery of the module. While some of the students who met the team were uncertain about what happened to the feedback they provided, programme board student representatives stated that the results and responses were discussed at programme board meetings. This was confirmed by the programme board minutes available to the team. The team was provided with several good examples of changes that had been made in response to student feedback, both formal and informal. The team was satisfied that students were actively involved in quality management, and that their views and comments received a considered response.
106 The audit team was satisfied that the quality of learning opportunities provided for students was suitable for programmes of study leading to the named awards.
Drama, Theatre and Performance Studies
107 The DAT was based on the BA/BSc Drama, Theatre and Performance Studies within the School of Arts. The programme is the largest in the School, with 183 FTE single honours students and 178.5 FTE combined honours students in October 2003.
108 The DAT was supported by a DSED which comprised a SED and programme specifications, both produced for internal periodic review in 2001-02, and a general update and a programme specification update, both prepared in 2003 for the purposes of the audit. The programme specification made reference to the FHEQ and the relevant subject benchmark statement. Further useful documentation was provided in the form of a specially prepared paper illustrating how the Code of practice, the benchmark statement and the FHEQ had been considered by the programme team with reference to the internal periodic review, and adjustments to provision made as appropriate. It was evident to the audit team that these elements of the academic infrastructure had been subject to appropriate debate and response.
109 The DSED provided comprehensive progression and completion data. Such data were also provided and analysed in the PARs, although in the view of the audit team, the use of statistics in the latter might be enhanced by the inclusion of tables to support the narrative and analysis.
110 The DSED documentation made it clear that the processes of internal monitoring and review were operating in line with Roehampton's expectations. The PAR is prepared by the programme convenor on behalf of the programme board, which is attended by all relevant academic tutors and student representatives. PARs are then presented to QSPG, which considers them, has discussions with programme convenors as required, notes points of good practice and those requiring action, and finally reports to AMC. The evidence available to the audit team demonstrated that the process of internal monitoring and review had involved debate in relation to the programme and, where appropriate, had led to change.
111 The audit team had access to external examiners' reports for the previous three years. A number of reports, while being positive and supportive overall, offered some constructive advice for the enhancement of the programme. The team found evidence to show that such comments had been discussed and evaluated by the programme team, and had been fed into the PAR process as appropriate. One external examiner had not been asked to approve examination papers.
112 A wide range of appropriate assessment strategies is employed within the programme and staff who met the audit team gave examples of the ways in which assessment arrangements followed Roehampton's procedures as set out in the Handbook. However, the team noted one example of a year one module for which the arrangements did not appear to follow fully the institutional procedures. For students, the BA/BSc Programme Handbook contains considerable information regarding assessment, supplemented by further details in the relevant module and programme handbooks. Students who met the team confirmed that the assessment information they received was clear, timely and helpful.
113 The audit team reviewed a range of assessed student work from all levels of the programme. The team was satisfied that the nature of the assessment and standard of student achievement met the expectations of the programme specification and the subject benchmark statement, and was appropriate to the relevant award and its location within the FHEQ. It was also clear that assignments had been returned to students with often comprehensive feedback, although one external examiner and students had commented that the amount of feedback could be more consistent and in some cases easier to read.
114 Programme and module information for students is available in both hard copy and electronic form. The audit team was provided with copies of the relevant programme and module handbooks, which were clear, informative and accurate. Students confirmed that they found such documentation valuable, drawing attention to the way in which it made clear the responsibilities of both students and tutors.
115 Students also spoke highly of the learning resources and support available to them. They outlined the specialist space provision available, and commented positively on library and computing resources, to which they had received a useful and well-organised induction. They were also able to clearly describe the academic and pastoral support provided, and praised the high quality of the support that they received from the programme team and the RED Centre.
116 Consistent with Roehampton's expectations, feedback from students is gathered in a number of ways. Written feedback is gathered and analysed at the end of each module, supported by discussions between staff and students, including an end of year question and answer session with the programme convenor. Students who met the audit team provided examples of modifications to both curriculum and assessment that had been made as a result of student feedback. Student representatives on the programme board felt that they had a real role to play and confirmed that their views were taken seriously. They also confirmed the support now available to them through the SU's Programme Representatives Scheme (see above, paragraph 56).
117 The audit team was satisfied that the quality of learning opportunities provided for students was suitable for the programme of study leading to the named awards.
Education
118 The DAT was based on two programmes within the School of Education Studies: the BA/BSc Early Childhood Studies and the Professional Development Programme. The Professional Development Programme is a part-time programme that embraces three final awards: a Professional Practice Certificate (30 credits at HE1), a Professional Development Certificate (30 credits at HE3), and a Graduate Certificate (60 credits at HE3). The Early Childhood Studies student population is split almost equally between students on single and combined honours programmes (146 and 129 respectively in 2002-03), with smaller numbers studying part-time. The programmes attract a large number of mature students. The Early Childhood Studies provision was included in an Agency subject review of Education undertaken in November 2001.
119 The DAT was supported by a DSED prepared for the audit. The DSED included a general overview of provision within the School, SEDs for the two programmes and a programme specification for the Professional Development Programme. The programme specification had been produced according to Roehampton's requirements and made reference to the FHEQ and the relevant subject benchmark statement. The audit team noted that Early Childhood Studies programme was due for revalidation in 2004 and no programme specification was currently available; however, the DSED included an updated validation document that covered much of the required information, albeit not in the format now required by Roehampton.
120 In discussing matters relating to the design of the programmes with staff, the audit team heard that there had been difficulties in developing a programme specification for the Professional Development Programme, but that the programme had been reviewed in a developmental way, taking account of the views of a wide variety of stakeholders and the available external reference points. From the programme approval documentation and from its discussions, the team was able to confirm that the institutional processes were being applied appropriately and that the programme team was making appropriate use of the subject benchmark statement, the FHEQ and relevant sections of the Code of practice.
121 Progression and completion data for the two programmes is provided by the Planning Office and made available on the internet. The DSED included progression and completion statistics and the audit team noted that such statistics were also included within the PARs. The team heard that training and advice on the use of statistics had been provided for the programme convenor, and staff described how they used this information to review progress and to identify and address potential issues. In some instances, they were able to take action, for example, the assessment process for a particular module had been revised as a result of statistical analysis; in others, such as the relatively low progression rates for combined honours students, they had been unable to identify the causes or any simple remedial action. From its review of documentation, the team was able to confirm that appropriate use was being made of statistics in the monitoring of standards, and that their use was being continuously refined.
122 The audit team had access to the PARs for the last three years. Each contained a detailed analysis of the provision and included action undertaken in response to the previous PAR, a response to external examiners' reports, and student feedback. Staff described to the team how the PAR was constructed, with input from members of the programme team and relevant support areas. The team heard that PARs were considered initially at the Programme Board, then scrutinised by a Quality and Standards Scrutiny Panel established by the School's Learning and Teaching Committee. In the view of the team, the PARs for the programmes, and the manner in which the programme teams had responded to matters raised through the annual monitoring process, were models of good practice.
123 In order to meet the needs of students, the Professional Development Programme is largely taught off-campus and by visiting lecturers. The audit team was provided with a copy of the tutor's handbook, and noted that it provided a range of useful material, including assessment criteria and marking. In discussions with staff, the team was informed that potential new sites were carefully vetted by the programme convenor against a set of criteria that included: resources, accessibility, safety, environment, information and communication technologies access and catering facilities. The convenor reports annually on sites through the PAR. Visiting lecturers are interviewed using standard University procedures; thereafter, they receive induction and additional support is available through the RED Centre. New visiting lecturers are observed teaching and have email support from the programme convenor. Their marking of student work is double-marked initially. The team also heard that increasing use was being made of WebCT to support study at a distance. The students studying by this mode who met the team were very positive about their experience and the support provided. The team considered that the School had appropriate safeguards in place to secure the quality and standards of such provision (see above, paragraph 71).
124 The audit team had access to the external examiners' reports for the programmes for the previous two years. The reports provided positive, supportive and constructive comments about the programmes and there was clear evidence that the programme teams had responded appropriately to the comments made.
125 Assessment arrangements for the programmes include clear procedures for the internal monitoring, marking and moderation of assessed work, which include information on the percentage of work which should be second-marked, which goes beyond the minimum expectations set out by Roehampton. Feedback to students is provided on a standard pro forma for each of the programmes, generally supported by more detailed comments on the work itself. Students who met the audit team confirmed that they were clear about the criteria by which they were assessed and said that they generally received detailed and helpful feedback in a timely manner. The team noted in particular the supportive way in which the programme teams assisted students returning to learning.
126 The audit team reviewed a range of assessed student work from all levels of the programmes. The team was satisfied that the nature of the assessment and standard of student achievement met the expectations of the subject benchmark statement, and was appropriate to the relevant awards and their location within the FHEQ. The Professional Development Programme work also met the expectations of the programme specification.
127 The audit team was provided with copies of the relevant student handbooks. The handbooks provided detailed information on the programmes and were supported by a web site and additional printed information provided both by the School and by Roehampton. Students who met the team spoke favourably of the value of the handbooks provided at both the programme and module level. They also commented that access to the web site was very helpful, given that personal circumstances and studying off-campus or part-time often made it difficult to access traditional forms of support. Many of the lecturing staff provided internet-based material and the ability to link to other resources was particularly valuable. Students also commented that material provided by the RED Centre, especially the study skills guide, had proved useful. It was clear to the team that appropriate and accurate information was made available to students.
128 Learning resources are monitored through feedback from students and through the comments of external examiners. In accordance with Roehampton's requirements, an academic liaison officer sits on the programme board and provides a link between the programme team and students, and the learning resources services (see above, paragraph 75). Students who met the audit team were satisfied with their access to appropriate resources through the learning resources centre, on-line and through a dedicated space within the School. The team heard that issues relating to access to computers, raised within previous PARs, had been resolved.
129 A range of support mechanisms is provided within the School. Each student has a tutor and the School has been piloting the use of an 'Academic Profile', an optional process which involves the student recording and discussing their progress with a tutor after the end of each semester. The audit team heard that about 80 per cent of students were using this process. The School is also piloting the use of an Academic Support Assistant to address the specific needs of the part-time, evening, and mature students on the programmes, who might be uncertain about returning to study and/or require additional support. Students who met the team spoke very positively about the range of support mechanisms available to them. While it was too early to evaluate the effectiveness of the new initiatives, the team considered the range and appropriateness of support mechanisms to be a feature of good practice.
130 Consistent with Roehampton's expectations, formal student feedback is sought at the end of each module through student representation on the programme board and, more informally, through general interaction between staff and students. Students who met with the audit team confirmed that these processes operated effectively and that appropriate action was taken in response to the matters they raised.
131 The audit team was satisfied that the quality of learning opportunities provided for students was suitable for programmes of study leading to the named awards.
Thematic enquiries
132 The audit team did not select an area for thematic enquiry.
Section 4: The audit investigations: published information
The students' experience of published information and other information available to them
133 The SED summarised Roehampton's arrangements for providing reliable and accurate information for students and prospective students. The MRA Department produces prospectuses, leaflets and newsletters, and manages the corporate and prospectus pages of Roehampton's web site. The programme details held by MRA are reviewed annually by programme convenors, admissions tutors, and the Academic Affairs Office, the latter being responsible for ensuring that published details of all modules and programmes are accurately presented. One MRA team member has specific responsibility for storing accurate information about new and closing programmes, and also maintains the web pages. The SED emphasised that the accuracy of information was further assured by the presence of an MRA representative on CDPG, thus ensuring that decisions about new or closing programmes resulted in immediate updating of all relevant marketing literature. Clear lines of responsibilities at institutional and school level are designed to prevent errors on the web site. The audit team noted that Roehampton was planning to enhance its current systems by establishing an electronic procedure for setting up marketing information for new programmes.
134 The SED stated that, once at Roehampton, students were 'entitled to receive accurate information about academic programmes and procedures operating within their schools'. General information, including the Academic Regulations, is provided electronically by the Academic Affairs Office, and as hard copy in schools. There is also a comprehensive, clearly written and informative Student Directory, produced by the Academic Secretary's Office, which covers all major aspects of academic requirements and student support. The Student Directory was the subject of a major review, and student and stakeholder feedback confirmed its value. The currency and accuracy of its information is subject to annual review. School and module handbooks are the responsibility of academic staff, within a framework of centrally defined minimum expectations regarding content.
135 The audit team discussed the provision of information with students at both institutional level and during the DATs. Students were consistently very complimentary about the standard of advance information they had received through prospectuses and other published documentation, and through the web site. They had found the prospectus to be clear and informative, and the more detailed information available on the web site, such as module specifications, to be very useful. They also had unanimous praise for the materials they had received after arriving at Roehampton, commenting that programme and module handbooks were comprehensive, clear and informative, and that they felt fully informed about their programmes and choices, and about the regulations and assessment requirements. While the students were less clear about complaints and appeals procedures, the team was satisfied that they would know where to seek advice in the event that they wished to pursue an appeal or complaint.
136 Two student groups were asked by the audit team if they had been given an opportunity to provide feedback on the information they were provided with, and both groups replied that they had not. This is a matter to which Roehampton may wish to give further consideration, as it continues to enhance its provision. The team was satisfied, however, that Roehampton provided its students and prospective students with comprehensive, relevant and useful information. The sample materials seen by the team were of consistently good quality.
Reliability, accuracy and completeness of published information
137 From 2004, the audit process will include a check on the reliability of the information about academic quality and standards that institutions will be required by HEFCE to publish in a standard format. At the time of the audit, Roehampton was moving in an appropriate and timely manner to fulfil its responsibilities in this matter, and the information that it was publishing about the quality of its programmes and the standards of its awards was reliable.
ISBN 1 84482 048 3
next >> Findings
Top
QAA updates
QAAtweets
QAAtube