1 An institutional audit of Anglia Polytechnic University (the University or APU) was undertaken during the period 24 to 28 May 2004. The purpose of the audit was to provide public information on the quality of the University's programmes of study and on the discharge of its responsibility for its awards.
2 The audit was carried out using a process developed by the Quality Assurance Agency for Higher Education (the Agency) in partnership with the Higher Education Funding Council for England (HEFCE), the Standing Conference of Principals (SCOP) and Universities UK (UUK), and has been endorsed by the Department for Education and Skills. For institutions in England, it replaces the previous processes of continuation audit, undertaken by the Agency at the request of UUK and SCOP, and universal subject review, undertaken by the Agency on behalf of HEFCE, as part of the latter's statutory responsibility for assessing the quality of education that it funds.
3 The audit checked the effectiveness of the University's procedures for establishing and maintaining the standards of its academic awards; for reviewing and enhancing the quality of the programmes of study leading to those awards; and for publishing reliable information. As part of the audit process, according to protocols agreed with HEFCE, SCOP and UUK, the audit included consideration of examples of institutional processes at work at the level of the programme, through discipline audit trails (DATs), together with examples of those processes operating at the level of the institution as a whole. The scope of the audit encompassed all of the University's provision offered within the University. The collaborative arrangements that the University has with a number of partners will be the subject of a separate, future audit.
Section 1: Introduction: Anglia Polytechnic University
The University and its mission
4 The University traces its antecedents to 1858 when a group, which included John Ruskin and Prince Albert, founded the Cambridge School of Art (CSA). In the last decade of the nineteenth century a school for vocational studies was established in Chelmsford. These two institutions were the early forerunners of the present University, which preserves the long tradition of education adapted to the social and economic requirements of the locality.
5 More recently the institution was established as a higher education (HE) corporation under the 1988 Education Act and was named the Anglia Higher Education College. This brought together the former Essex Institute of Higher Education in Chelmsford and the Cambridgeshire College of Arts and Technology in Cambridge. In 1991, the College was granted polytechnic status by the Secretary of State. Under the 1992 Further and Higher Education Act, along with all other polytechnics, Anglia Polytechnic was granted university status. The University holds degree-awarding powers in its own right for undergraduate, taught postgraduate and research postgraduate degrees. The main campuses of the University are in Cambridge and at Chelmsford in Essex.
6 When APU was established in 1989 as an independent HE corporation its student population was about 7,000 full-time equivalent (FTE) students. By 2002-03, there were 18,852 FTE students. In 2002-03, approximately 24,800 students were registered on undergraduate programmes, 3,700 on taught postgraduate programmes and 400 on research degree programmes.
7 The University is organised into eight subject-based schools, the Graduate School, and the Health Business Centre. The subject-based schools are the Ashcroft International Business School (AIBS); Applied Sciences; Arts and Letters; Community Health and Social Studies; Design and Communication Systems; Education; Health Care Practice; and Law, Languages and Social Sciences (established with effect from 2003-04 from two existing schools). Each of the nine schools is led and managed by a dean. Deans are the academic leaders at APU, reporting to the Vice Chancellor on academic planning, quality assurance and curriculum development within their schools and across the University.
8 The University offers a diverse range of subjects and courses covering arts and humanities, business, law, applied sciences, social sciences, engineering, built environment, teacher education, information technology (IT), nursing and midwifery. It has developed a credit-based, modular curriculum primarily delivered on a semester basis and its programmes of study are designed to provide flexible learning opportunities and choice, reflecting APU's distinctive mission and role within the region.
9 A single curriculum framework has been developed (with separate components for undergraduate, postgraduate and post experience delivery) to ensure consistency and comparability in setting and maintaining academic standards and quality in the delivery of programmes leading to APU awards. Curriculum management is based on a set and field structure. A set is a group of related modules within a subject (for example, history) or vocational area (for example, construction management). A module is assigned to one set only. Sets are the responsibility of the academic schools, with a designated set coordinator responsible for the management and delivery of modules within a set.
10 A field is APU's collective term for a group of educationally related pathways whose common element is normally a subject or area of study. Students are registered for a named pathway comprising a range of specified modules, some of which may be classified as compulsory or designated modules. In many cases, given APU's approach to curriculum design and content, a pathway includes modules from more than one set and sometimes from sets managed by different schools. Fields have been established to provide the focal point for the delivery of pathways and the quality assurance and enhancement of the student experience. A field leader is responsible for the management and delivery of pathways assigned to a field. APU's key processes for the assurance of academic standards and quality (annual monitoring and periodic review) are based on the field.
11 At the time of the audit, the University had appointed a new Vice Chancellor, who was shortly to assume the full responsibilities of the role. The audit team met the outgoing Vice Chancellor at the briefing visit and had the opportunity for an informal meeting with the new Vice Chancellor at the beginning of the audit visit itself.
12 APU begins its 10-year Strategic Statement (1998 to 2007) by setting out its mission: 'We aim to be the local University of choice for students and employers for the East of England. We serve Essex, Cambridgeshire, Norfolk, Suffolk and adjacent parts of Bedfordshire and London. We provide higher education, economic and cultural support to communities in this region'.
This statement (and the accompanying strategic objectives for the period 1998 to 2007) was approved by the Board of Governors in December 1997 and provides a context for the University's approach to all its activities.
13 In 2002 the University adopted the following set of 'brand values':
'What APU believes
- Our University should work for you
- People should be given the opportunity to realise their ambitions
- Potential has to be recognised before you can develop it
- Accessibility need not come at the expense of quality
- Education should be relevant to both life and work
- You have to connect with the world of work to understand it
- We should contribute to the future prosperity of the region
The University and You
- Our University should work for you
- We treat people as individuals, not statistics
- We welcome people who want to get on
- We identify individual promise and then cultivate it
- We add context, meaning and direction to talent
- We prepare people for life and work, not just exams
- We genuinely connect with business, the professions and the public sector
- We take a real pride in our work, our people and the communities of which we are a part.'
The statements are encapsulated in the strapline that appears with APU's logo: 'The University that works for you'.
Collaborative provision
14 The University's self-evaluation document (SED) provided for the audit stated that 'In the past 10 years APU has established a network with 24 partners in the East of England - one of the largest and most innovative partnerships in UK higher education'. The partners are mostly further education colleges located across the East of England, involving the delivery of programmes of study leading to an APU award. The partnership is collectively known as the Regional University Partnership (RUP) and is a distinctive feature of APU's approach to HE. This institutional audit focused on the core APU provision at its home campuses. Collaborative arrangements will be the subject of a separate audit by the Agency but the audit team did consider the extent to which recent changes to the quality assurance and enhancement systems, made in the context of the rapidly developing regional partnership, had impacted upon the assurance of quality and standards of APU provision at its home campuses.
Background information
15 The published information available for this audit included:
- information on the APU web site;
- the previous quality audit report for the University, published in May 2000;
- the report of the Agency's overseas partnership audit of the University and the Fachhochschule fÜr Wirtschaft, Berlin (Germany), published in July 2003;
- reports of reviews by the Agency of provision at subject level, published since 1997;
- information on the Higher Education Statistics Agency, UCAS, Higher Education and Research Opportunities in the United Kingdom and HEFCE web sites.
16 In addition, unpublished reports relating to two developmental engagements and one Foundation degree review conducted in 2003 were made available to the audit team.
17 The University initially provided the Agency with:
- an institutional SED;
- discipline self-evaluation documents (DSEDs) for the five DATs.
18 During its visit, the audit team was given access to the University intranet and a range of internal documents.
The audit process
19 Following a preliminary meeting at the University in September 2003, the Agency confirmed that five DATs would be conducted during the audit visit. The audit team's final selection of DATs was: business (postgraduate); graphic arts; history; social work; and surveying. The Agency received the institutional SED and supporting documentation in January 2004 and the DSEDs, accompanied by programme specifications, in April 2004. The SED and the DSED for surveying were written specifically for the audit; the DSED for business was based on a separate document prepared for an internal APU periodic review, and the DSEDs for graphic arts, history and social work included the internal APU periodic review document, accompanied by supplementary information.
20 The audit team visited the University on 20 and 21 April 2004 for the purpose of exploring with student representatives, senior members of staff and the Vice Chancellor, matters relating to the management of quality and standards raised by the SED and other documentation provided for the team. At the end of the briefing visit a programme of meetings was developed by the team and agreed with the University.
21 At the preliminary visit for the audit, students of the University were invited, through the Students' Union (SU), to submit a separate document (the students' written submission (SWS)) expressing views on the student experience at APU, and identifying any matters of concern or commendation with respect to the quality of programmes and the standard of awards. The Agency received the SWS from the students in January 2004. The audit team is grateful to the students for preparing this document, which was not confidential, to support the audit.
22 The audit visit took place from 24 to 28 May 2004 and included further meetings with staff and students of the University, both at central level and in relation to the selected DATs. The audit team comprised Professor M Carswell, Professor A Dean, Dr S Hill, Ms H Johnson, Professor S Sayce, Mr N Watson, auditors, and Mr S Appleton, audit secretary. The audit was coordinated for the Agency by Dr I Ainsworth, Assistant Director, Reviews Group.
Developments since the previous academic quality audit
23 The previous Agency quality audit report (May 2000) had commended three aspects of the University's arrangements, notably: the strong staff commitment to the process of monitoring at school level; the University's approach to, and arrangements for, the quality management of its regional collaborative provision; and the research student training scheme and other initiatives taken by the University in creating an institution-wide research culture. The previous audit also advised the University to give a more precise definition to its strategic approach to quality management; consider whether the University's committee structure and quality assurance processes enabled it to obtain a clear view of the effectiveness of its arrangements for quality and standards; continue to develop its approach to overseeing its collaborative provision; give continuing attention to the provision of data in support of assessment practices; strengthen its approach to identifying, monitoring and responding to the infrastructure needs of postgraduate students; and to secure full implementation of the University's appraisal scheme for academic staff and consider further the link between appraisal and staff development. The SED addressed the University's actions in response to these points and they were followed up in the present audit.
24 The University had been subject to a number of Agency subject reviews since the last institutional audit, and had been involved in the Agency's special review of Foundation degrees and two developmental engagements. Results of subject reviews since 2000 showed a consistently high rating for the University's student support and guidance but less consistently good performance in quality management and enhancement. The SED drew attention to action taken by the University following a negative outcome in the review of a Foundation degree, which led to the Agency's agreement to accelerate its normal follow-up visit and, subsequently, to issue a judgement of 'confidence'. The SED claimed that the action taken and positive outcome achieved within seven weeks of the initial, negative outcome 'clearly demonstrates the seriousness with which APU takes academic standards and quality issues...'. The outcomes of the two developmental engagements in law and chemistry were largely positive while also identifying areas that could be improved. The audit team noted the considerable quantity of external reviews with which the University had recently been engaged and the very active way in which it had responded to the ensuing reports.
Section 2: The audit investigations: institutional processes
The University' s view as expressed in the SED
25 The SED described the University's quality assurance policies and procedures as 'robust and probing' with particular emphasis being placed on the maintenance, monitoring and enhancement of the student experience and the quality of educational support to students. The University has developed in response to internal and external factors over the past 10 years, and views its systems as an 'evolving process'. The SED stated that APU had applied a learning outcomes approach to the setting of academic standards for some years and that the University has developed a set of generic learning outcomes (GLOs), supported by APU level descriptors. These level descriptors were reviewed in 2000 to align with The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ).
26 The SED set out how the University deals with the setting, assessment and maintenance of standards through its teaching teams and the committees of the University associated with quality matters. External reference points, including the subject benchmark statements, the Code of practice for the assurance of academic quality and standards in higher education (Code of practice) published by the Agency, the involvement of professional and statutory bodies (PSBs) and the input of external assessors (external examiners), are prominent. At the time of the audit visit, the University had recently put in place a range of new processes involving a much greater integration between the management of the core APU provision and that of the wider RUP. These new processes are intended to ensure that core APU, international and regional provision are fully integrated within common procedures, reflecting planning, administrative approval, development, programme approval, monitoring and review processes. The extent of the integration of quality processes across the RUP became clear to the audit team in the course of the visit.
The University's framework for managing quality and standards
27 The ultimate responsibility for APU's academic quality and standards rests with Senate. Quality and standards are assured, maintained and enhanced, on behalf of Senate, through the operational oversight of the Academic Standards, Quality and Enhancement Committee (ASQEC). APU's policies and procedures for the setting, maintenance and enhancement of academic quality and standards are set out within a range of University documents. The key documents include the Academic Regulations, Curriculum Regulations, Curriculum Management Handbook, and Senate Codes of Practice and associated procedural documents. Of these documents, the Senate Codes of Practice and the associated procedural documents have recently replaced the four-volume Quality Assurance Handbook. The other documents have existed as separate documents for some time. The audit team heard that the replacement of the Quality Assurance Handbook by the Senate Codes of Practice was the result of the University's view that there was a need to distinguish policy from practice, particularly in light of the complex demands of its regional university mission. However, in discussions with staff, the team found some confusion about aspects of the documentation relating to quality processes.
28 The University's frameworks and processes for the management of academic standards and quality through the Senate committee structure have changed over time to reflect the rapid growth of APU's collaborative regional mission. While the University's collaborative provision was not within the scope of the audit, it was evident to the audit team that the development and operation of the RUP had significantly shaped APU's approval, monitoring and review systems for assuring the quality and standards of core APU provision. In this respect, the role of APU within the context of the RUP and the concomitant impact on core APU provision, as well as on the deliberative and executive structures of the University, provided a context for the work of the team.
29 In 2001 APU undertook a strategic review of the future structure and organisation of the RUP, which focused on the need for improved academic links between the regional partners and APU's schools and for an integrated approach to the quality assurance of learning and teaching within APU schools and partner institutions. The audit visit took place during a period of transition at APU as new structures, policies and systems were being implemented and had yet to be evaluated by the University. The management of the academic quality and standards of APU's educational provision is now operated through a revised Senate committee structure that comprises a range of committees and subcommittees each of which has defined terms of reference and reporting lines. One of the key features of the revised system is the establishment of five regional faculties and associated regional faculty boards (RFBs) with specific strategic planning, curriculum development, and quality assurance responsibilities across APU schools, regional partners and, where applicable, international partners. The RFBs are chaired by an APU dean of school, except for the Health Sciences RFB which is chaired by the Director of the Health Business Centre. The RFBs report on strategic planning and curriculum development issues to a regional academic council, whose membership comprises senior staff from APU schools and partner institutions. RFBs report to ASQEC, the Senate committee responsible for academic standards and quality, for their quality assurance responsibilities. The SED indicated that previous quality assurance policies and procedures have been substantially revised to reflect APU's new internal structures, particularly the establishment of the RFBs, and to take full account of the Academic Infrastructure developed by the Agency.
30 Senate committees transact business in accordance with policies and procedures approved by Senate. Although Senate has delegated certain responsibilities for the assurance of academic standards and quality, it retains oversight of these matters. It receives formal written reports from each committee, in which the committee chair summarises the main issues, outcomes and recommendations to Senate. Senate also requires the submission of a range of annual reports including an annual quality assurance report summarising the outcome of internal approval, annual monitoring and periodic review processes as well as those resulting from visits by external quality assurance agencies and PSBs. Given the scope of such activity within APU, as well as that taking place across the wider RUP, the volume of reporting each year is substantial and, in the view of the audit team, creates significant risk for APU in terms of ensuring that the University receives timely and appropriate information to enable it to be assured that the quality and standards of its awards are, and continue to be, appropriate.
31 Each year an annual quality assurance report is prepared for consideration by Senate. The SED stated that this report 'summarises and analyses the outcomes and emerging themes from a range of internal and external quality assurance activities'. During the audit visit the University confirmed that this document was regarded as a key element of the University's quality assurance and management processes. In the view of the audit team these reports were largely descriptive and lacked the necessary level of detail of critical reflection and analysis that would provide Senate with a secure and rigorous basis for the safeguarding of quality and standards across the University.
32 A single curriculum framework has been developed (with separate components for undergraduate, postgraduate and post experience delivery) to ensure consistency and comparability in setting and maintaining academic standards and quality in the delivery of programmes leading to APU awards. The SED stated that curriculum regulations are the principal means through which the University ensures comparability of academic standards across its awards. Curriculum management is based on a set and field structure that is administered at the point of delivery by set coordinators and field leaders. Directors of studies have a curriculum management responsibility across fields and, in several cases, across schools, for which they report to Senate's Curriculum Committee or ASQEC, depending on the issue under discussion. The Directors of Studies Group, designated as a formal subcommittee under Senate's revised committee structure, has over a number of years developed and approved a number of operational guidelines designed to complement the quality assurance policies and procedures approved by Senate. These operational guidelines are contained in APU's curriculum management handbook which is updated on a regular basis and published throughout the University. Within the SED, the University acknowledged 'that it has a curriculum management structure which is complex and which places heavy demands on staff time'. The audit team would agree with this statement and noted, in particular, the onus that the current systems place upon the directors of studies, identifying this as a key role within the curriculum management structure.
33 The undergraduate and postgraduate student handbooks make explicit APU's marking standards. In addition, the University's approval and periodic review procedures require the link between learning outcomes and assessment methods and strategies to be made explicit in pathway specification forms. The academic standards set for a pathway and its constituent modules are confirmed and assured by the assessment processes. These processes include a two-tiered assessment board structure, namely set assessment panels (SAPs) and awards boards/professional awards boards (ABs/PABs), whose respective responsibilities include the awards of marks for modules within a set and student progression towards a named award. Once approved, module marks are formally submitted to an AB/PAB at which each student's mark profile is considered, normally on a pathway basis, and the final degree classification is determined. An external assessor is appointed to full membership of each AB/PAB. However, many external assessors do not attend PABs and may not have the opportunity to gain an overview of the programme that leads to an award. The University is aware that there remains some diversity of practice at operational level in the area of assessment. Senate is, therefore, planning to strengthen its current approach through the preparation and implementation of a new Senate code of practice on assessment.
34 Annual reporting on the quality and standards of academic delivery leading to APU awards is focused at field level. Each field leader is required to produce an annual monitoring report (AMR) for submission to a designated director of studies, who is responsible for producing a synoptic report, synthesising the significant quality and standards issues from the range of subject-based AMRs drawn together from across the RUP. Synoptic reports are then scrutinised by the relevant RFB. In addition to producing a synoptic report, the SED indicated that directors of studies also provide initial feedback on the AMR to field teams in the form of a 'reader's report', providing a critical commentary. The audit team found evidence within the DATs that the synoptic reporting system was proving problematic for directors of studies in terms of providing a full synthesis of issues drawn from a substantive number of AMRs across some large subject fields, and that the 'reader's reports' were not consistent in highlighting important quality issues.
35 APU has a learning outcomes approach to curriculum design, content and the setting of academic standards that requires explicit statements about student achievement at both module and pathway level. As indicated in paragraph 33 above, the University's approval and periodic review procedures require the link between learning outcomes and assessment methods and strategies to be made explicit in a pathway specification form (which sets out the requirements of meeting a named award) and these statements are also included in module definition forms (MDFs). In the SED, the University acknowledged that relating intended learning outcomes to learning, teaching and assessment strategies had caused particular difficulties for academic staff. Senate has established a set of GLOs used throughout the University in the construction of both module and pathway specification forms. In adopting this approach the University has taken account of its own organisation, internal needs and priorities, particularly in response to identified weaknesses in existing quality assurance arrangements and a range of external requirements, including the Academic Infrastructure.
36 The report of the Task Force on Quality Assurance, 'Empowering Professionals in the Regional University', discussed a number of issues relating to the RUP development and its impact on quality assurance systems and processes. At the time of the audit, the University's revised structures for managing and assuring its quality and standards had not been through a complete annual cycle and had yet to be evaluated. The audit team appreciated the need for the University's framework for managing quality and standards to take account of its distinctive mission and the recognised complexity of institutional structures. However, on the evidence gathered during the audit, the team considered that the current arrangements, which are complex, inconsistently applied and burdensome, create a considerable risk to the security of quality and standards across the University. The team formed the view that the securing of quality and standards would be significantly enhanced through the provision of a single, accessible and definitive reference point that pulls together all the University's codes of practice, policies and procedures for the benefit of staff and students of the University so that the University can be assured that its policies and procedures are known, understood and consistently applied.
37 While the audit team recognised that the new systems and processes are in the early stages of implementation, it was provided with little explicit information in respect of the University's intentions for evaluating the impact of the changes. It was clear to the team that there were already signs of strain in, for example, the feasibility of whole-field periodic review across the RUP, and it would have welcomed more recognition of the problems at institutional level, and some understanding of how the University was developing a strategy to overcome these problems.
The University's intentions for the enhancement of quality and standards
38 The SED did not include a specific section or statement on enhancement but it did contain references to the University's arrangements for the enhancement of quality and standards, which indicated that quality enhancement is 'a primary purpose of the revised annual monitoring process', and enhancement was described as a 'key focal point' for periodic review and linked to the idea of action planning as a way of encouraging course teams to develop a SMART (Specific, Measurable, Agreed, Realistic, Timebound) action cycle. Changes to monitoring and review also include an explicit encouragement of the dissemination of good practice and a summary document, bringing together elements of good practice from the range of monitoring reports for 2002-03, has been widely circulated. Many of the plans for enhancement at institutional level focus upon the student experience. The SED listed a number of strategies to address limitations of the current situation, including a new approach to the provision of academic guidance, new mechanisms for gaining student feedback, and a new Senate code of practice on assessment to improve the quality of feedback to students on assessed work. Structurally, the focus for enhancement in the SED was on the well-established plans for the further improvements at the two APU campuses in terms of accommodation and facilities, and simplifying the curriculum structures by reducing the number of pathways.
39 The audit team noted that a number of the University's committees have aspects of enhancement in their terms of reference and considered that a clearer expression of the purposes/kinds of enhancement each committee was charged with would aid communication and avoid unnecessary duplication of activity. The team also noted that the University's learning and teaching strategy includes an action plan in a number of areas of work related to enhancement. In the course of the audit the team heard that, in some schools, learning and teaching advisers were successfully embedding a number of enhancement opportunities into the operations of the schools. Close working between the University Centre for Learning and Teaching (UCLT) and the advisers was apparent. The team considered, however, that the strategic elements of learning and teaching development, for example, in relation to retention strategies, and the application of technology to learning and teaching, could be strengthened (see paragraph 113, below).
40 The audit team found that processes around SMART action plans had yet to be embedded, in that it saw AMRs without action plans, which served to reduce the effectiveness of an otherwise sound arrangement for gathering ideas for enhancement and ensuring action was taken in respect of them. Consequently, the team considers that it would be advisable for the University to embed fully the SMART action planning procedure, applying it consistently and rigorously, and identifying where the locus of responsibility lies for both action and monitoring.
41 A draft report from the Curriculum Management Structures working group, which was established to review and simplify APU's current curriculum management, makes a number of recommendations for the future, and was made available to the audit team. The team understood that this report had yet to be presented formally to Senate, but would support the University in its intentions to simplify and streamline the curriculum management arrangements. In the light of the possibility of dispensing with the current structure of sets and fields as organisational units, the team considered that directors of studies were pivotal to both the assurance of standards, through their oversight of the annual monitoring process, and in being able to check that all AMRs included action plans. The synoptic report drafted by directors of studies was also vital to the overview of good practice being brought to the attention of the University through the RFBs but, as paragraph 34 above indicates, the synoptic report system has its limitations. The team formed the view that it would be desirable for the University to consider means by which the effective contribution of the directors of studies can be harnessed more strategically, given their key role in quality assurance and enhancement.
42 The audit team considered that there was a risk that the emphasis on enhancement may reduce the quality assurance element of the annual monitoring and periodic review reporting processes, and that this was especially likely while the role of the new RFBs was still being clarified. The team saw evidence of concern, on the part of directors of studies, about the effectiveness of annual monitoring in the new arrangements. The team agreed with these concerns and would encourage the University to continue to debate the place and role of RFBs in the annual monitoring and periodic review processes.
Internal approval, monitoring and review processes
43 Programme approval, monitoring and periodic review processes have been significantly revised since the last quality audit visit, with a new system being introduced over the current academic year. The full cycle for implementing all the changes will not be completed until September 2004 and the visit, therefore, took place at a time of transition when many of the new processes had not been fully embedded. The processes are set out in a Senate code of practice and its accompanying procedural document. The Senate code indicated that the processes are based on the precepts contained in the Code of practice, Section 7: Programme approval, monitoring and review, and states that the locus of responsibility for academic quality and standards lies with Senate but with delegation to the ASQEC. This committee, in turn, has delegated the conduct of certain quality assurance processes to RFBs.
44 The SED indicated that a common planning approval process, overseen by a strategic information and planning unit and subject to consideration by an executive scrutiny group, was to come into effect from 2003-04 with transitional arrangements adopted for the period August 2003 to February 2004 in respect of proposals with a September 2004 start date. Before obtaining formal approval for a new programme, proposing teams must obtain approval in principle through the appropriate administrative planning processes, which include an evaluation of market demand, competition, consistency with University strategy, resource implications and the proposed team's 'track record' in recruiting students to pathways/courses approved in the recent past. Once planning approval has been granted, permission is given to commence marketing and recruitment on the understanding that this is clearly stated to be 'subject to formal approval' and a programme approval board is constituted under the auspices of the relevant RFB. This board contains at least one external panel member (either academic or practitioner) with subject specialist expertise who is not an external assessor (as APU styles its external examiners) and, where appropriate, a member of the relevant PSB. The chair will normally be a member of APU's Senate or ASQEC or a person with extensive panel membership experience.
45 The Senate code of practice and accompanying procedural document set out detailed and largely prescriptive requirements in terms of both timetabling for the approvals process and the production of documentation, which must include pathway specification and MDFs to facilitate discussion and ensure due reference to external reference points. Emphasis is placed on the aims and learning outcomes, learning and teaching strategies and quality assurance and enhancement arrangements.
46 The approval process, as set out in the Senate code and accompanying procedural document, comprises 'one stage' which may include an event or decision by correspondence. It starts with a requirement that proposal teams submit documentation at least 10 weeks before the scheduled meeting, during which time the documents are checked for compliance and key issues for exploration are developed, and culminates in a written report of the outcomes. A standard agenda is followed to ensure consistency between panels. Any conditions stipulated should have specified deadlines, and recommendations may also be made to which proposal teams must respond within set time limits. The Academic Office is responsible for ensuring that conditions and recommendations are addressed. Failure to comply with conditions would be referred to the chair of the relevant RFB who is responsible for deciding on action to be taken, in consultation with the chair of ASQEC, as appropriate. Once a programme is approved, it remains so indefinitely, subject to periodic review. Minor alterations to curriculum structure, content and delivery may be made at any time and approved by the newly constituted Directors of Studies Sub-Committee, acting on behalf of the relevant RFB, in consultation with the student data systems unit. In the case of individually negotiated awards (at both undergraduate and postgraduate levels), proposals from individual students or closed cohorts (such as in-company schemes) are submitted to the Accreditation and Approvals Committee, which reports to ASQEC.
47 The SED indicated that the University is aware that, although it has long taken a learning outcomes approach to standards and new programmes, the old processes were not fully explicit in their relationship with the FHEQ and the new processes are the result of this awareness. The new processes have also been devised to enable teaching teams to ensure that they have fully reflected subject benchmark statements, published by the Agency. The SED acknowledged that the new systems, including the incremental transfer to programme specifications, have been difficult for some staff, as they represent a very different approach to that which was previously in place.
48 The audit team considered that there was appropriate external participation in the approval process and that the process set out in the Senate code was comprehensive and explicit in relation to new awards. The procedural document accompanying the Senate code of practice on the approval, annual monitoring and periodic review of taught programmes of study indicated that new individual modules could be approved in the period between reviews by the Directors of Studies Sub-Committee, on behalf of the relevant RFB. The SED described how a committee decision is required for minor changes which are ratified by RFB. At the audit visit the team saw the procedural document that provides directors of studies with executive responsibility using a 'fast track' pro forma. The team noted that the pro forma provided for such changes to be proposed by a pathway leader, countersigned by the field leader and approved by the director of studies. The minutes of directors of studies meetings confirmed that this process was still in the process of evolution.
49 The audit team considered that incremental changes to modules and to curriculum structure approved in this way could result in incremental 'drift' which could eventually lead to the generic aims and objectives for a pathway being undermined. This was discussed with staff and, while it was clear to the team that there was some good practice taking place locally in some areas to avoid the possibility of drift, it was less clear that there were systems in place on a university-wide basis to address this issue. The team was of the view that the role of directors of studies as focuses for quality assurance in this process was of central importance, but this currently might not be fully recognised as they did not have formal representation on RFBs. The team noted that this was also a matter of concern to the directors of studies themselves. Despite the recent changes, the team considered that it would be difficult to establish and maintain an institutional overview unless the approval and monitoring procedures were implemented in such a way as to ensure a more systematic link between the directors of studies and the RFBs.
Annual monitoring
50 Annual monitoring is conducted on a field basis by field leaders who prepare reports for initial discussion by field committees (that include student representation), which are asked to agree both the content and the proposed SMART action plan. Field leaders use a prescribed template, annually updated by the Academic Office and approved by ASQEC. Evidence in the AMRs is expected to include an analysis of the student assessment profile and student degree classifications within the pathway/course under review; student retention and completion rates on the pathways/courses; external assessor reports; PSB reports (where appropriate); student evaluation/feedback on the modules and pathways/courses; and further information from the separate annual monitoring of modules undertaken by schools on a set basis.
51 Following field committee approval, reports are submitted to the appropriate director of studies (or a designated alternate) who provides initial feedback to field leaders, in the form of a reader's report. This is an important part of the process as it identifies and confirms action required. The implementation of action plans rests with field committees. Subsequently, directors of studies prepare synoptic reports on the annual monitoring process for the fields within their remit. The synoptic report should cover curriculum delivery and assessment; good and innovative practice in learning, teaching and assessment; the student experience and quality of educational support provided to students; the student profile; and action plans, with all field action plans being appended to the main report. It should also highlight any recurring themes or key issues that require a wider audience, for discussion and response by the appropriate RFB annually in January/February. The process also applies to students registered for a negotiated award: an AMR for the Negotiated Field is submitted to the Education, Negotiated and Interdisciplinary Studies RFB.
52 APU recognises that annual monitoring presents opportunities for quality enhancement. The new process of SMART action plans is designed to achieve this at the field delivery level. Matters identified through the synoptic reports are reported to RFBs and thence to ASQEC and Senate. RFBs should identify any generic, university-wide issues for further discussion by the Directors of Studies Sub-Committee and/or the Field Leaders Standing Committee. Good or innovative practice identified through annual monitoring is disseminated by the Academic Office.
53 The annual monitoring process is focused at a local level on modules, with every module subject to student evaluation and potential external assessor comments, which are then addressed through field leaders' AMRs. The AMRs considered in the DATs revealed variability in practice. In one DAT, wide variation of practice was found in the data presented and in their use, with much of the data being over-descriptive and lacking in analysis. In other cases, external assessor comments, while noted, did not lead to any evaluation of appropriate action: discussion of the student profile in one 2002-03 AMR was found to be descriptive and attenuated to the point where it could not helpfully inform judgements about management of the provision. Additionally, the audit team found that some AMRs failed to adopt the SMART action plan approach set out in the University's procedures, provided no comment on the student profile, did not identify strengths and weaknesses or failed to follow though on previous actions.
54 The SED indicated that 'through reflective evaluation' annual monitoring should be a vehicle for enhancement, but during 2003-04 directors of studies expressed concern that the readers' and synoptic reports have not helped to meet this aim. Although the new process is designed to ensure that delays do not occur, the audit team found evidence of AMRs which did not comply with the process laid down by Senate and, as a consequence, the timetable envisaged in the design of the new system had not been fully achieved. The team saw from two sets of RFB minutes that the late submission of annual reports had been identified as an issue and that action had been taken in response.
55 The audit team considered that, where a field included many pathways/courses, the sheer scope of the AMR meant that issues relating to individual pathways might be missed. Furthermore, bringing field reports together into synoptic reports meant that RFBs, which are the identified locus of responsibility for assuring the implementation of quality processes, were at risk of not receiving a full and frank overview of the annual delivery. The inclusion within the synoptic reports of individual action plans should provide a safeguard but it was noted that some reports had either incomplete or missing action plans and no robust mechanism existed to ensure that omissions or errors within individual action plans or reports were systematically identified.
56 The audit team saw evidence that directors of studies had expressed concerns regarding the utility of some key aspects of the revised annual monitoring system and came to the view that the current process of annual monitoring was too complex and potentially not fit for purpose. The current multi-layered approach meant that 'closing the loop' to ensure that actions were identified and taken in a timely way and with robust enforcement measures was not fully assured. This was particularly noticeable where the synoptic report had to address many individual programmes drawn from within the RUP. The complexity of process and of grouping of reports was one that was recognised in part by one RFB which noted that 'the grouping of areas for annual monitoring was illogical...'. The team appreciates that the wider RUP is not included in the scope of this audit; nonetheless, it is concerned that the complexity of RUP has resulted in the design of an annual monitoring system which is reductionist in terms of the core APU provision.
57 Some staff confirmed that action plans were not always prepared and this was further supported by RFB minutes, which also pointed to variability in action plans suggesting inadequacies in the template for action plans that did 'not focus on distinct issues/questions'. While annual monitoring is not currently the mechanism through which conditions and recommendations included in approval and review reports are monitored, at the time of the audit it was not clear how such conditions would be checked. The audit team found evidence, in one of the DATs, that a failure to address a condition of approval and continuing comment from an external assessor had not been picked up by any of the existing processes. While accepting that the procedures have now been revised, the team was unable to find an existing structural link which assured that conditions of approval would be monitored and checked. The University is advised to ensure that programmes approved subject to conditions cannot move into operation without being properly signed-off.
58 The audit team found that the annual monitoring process had not fully bedded down at the time of the audit and formed the view that, while the annual monitoring process is working in some areas as a vehicle for enhancement, it lacks robustness as a means of quality assurance. Discussions with staff indicated that the complexity of the system meant that some staff members were unclear as to the locus of responsibility for action, reporting lines and, in particular, the remit and authority of directors of studies.
Periodic review
59 The procedure for periodic review has been revised significantly with effect from 2003-04 with a new system of subject/field review replacing the previous course review arrangements. The system is set out in the Senate code of practice and its accompanying procedural document. All pathways/courses at all delivery points in the UK (APU schools and regional partners offering franchised delivery) are subject to review as a group (in a field review) every five years within the APU field to which they have been assigned. The SED indicated that a business and management undergraduate review, undertaken in 2002-03, was a prototype for field-based reviews. A forward programme for field reviews for the period 2003 to 2008 has been prepared and will be maintained by the Academic Office.
60 In addition to reviewing pathways/courses within a field, reviews also offer the opportunity to consider further revisions to curriculum structures, content and delivery that may be proposed under the review process. The review process may also include the approval of new awards. The periodic field review is conducted by a review panel, established by the relevant RFB, with principles of membership identical to those adopted for the approval process including external and, where appropriate, PSB members. Alongside the key focal points (academic standards, quality of education, quality management and enhancement) an important aspect of periodic field review is the strategic vision for the future direction and delivery of the constituent pathways/courses, evidence of which is required in the review document. Periodic field review requires a formal meeting between the review panel and review team, preceded by a meeting with students. The SED described how, as with annual monitoring, a new feature is a requirement for review panels to identify examples of good and/or innovative practice in learning, teaching and assessment for dissemination within APU and its partner institutions.
61 A major change in the new approach to periodic review is that the process consists of a single stage. Reviews are designed to be conducted over a 10-week period starting with submission of written documents which are checked for consistency and the identification of issues for debate at the event. A report on the outcome, which should be produced within two weeks of the event, is sent to the relevant RFB. The report may contain conditions and/or recommendations and the Academic Office is responsible for ensuring follow-up action is taken as necessary. The Senate code of practice states that where conditions are not met, the Chair of the RFB determines appropriate action, including the possibility of suspension of approval. One of the stated aims of periodic review is continuous improvement and the Academic Office is responsible for disseminating to the wider APU community any examples of good and innovative practice in learning, teaching and assessment, as approved by the RFBs.
62 The audit team explored the new process of periodic review through meetings, documentation provided and particularly through the DATs in business, graphic arts, social work and history, all of which had been scheduled by the University for periodic review events in the weeks preceding the audit visit, thus providing the team with an appropriate opportunity to view the new processes in action. Staff confirmed that the DSEDs in each case had been modelled on the documentation produced for periodic review. The team noted that the documentation provided in two areas was descriptive rather than evaluative. The SED stated that under APU's new review processes all courses/pathways would be reviewed as a group within the field to which they have been assigned and that the periodic review of business was a prototype for such reviews. The team noted that the review was indeed a prototype in terms of scope (as opposed to process) but was essentially conducted under APU's previous periodic review model which included a two-stage process for approval and review events. The first reviews under the new process had only recently been undertaken and the reports of those reviews had not yet been submitted to the relevant RFBs. Although the report for graphic arts had been published, the history report was in draft form at the time of the audit. In the case of the postgraduate provision in business, the review event, which took place a month before the audit, identified a number of concerns which necessitated a resubmission of documentation and a postponement of the event to a later date following the audit in order to address a range of concerns.
63 The Academic Office assigns a staff member to teams to provide advice and guidance on their preparation for periodic review. Such reviews examine the provider's strategic vision for the future direction and delivery of all pathways/courses within a field, drawing on the experience and lessons learned from the past. Periodic review provides an opportunity to make minor revisions to curriculum structure and content and may also result in major changes to the structure and organisation of pathways within the field. The audit team found an example of this in one area where the proposed review covered 21 existing awards to be rationalised and replaced with 10 masters pathways (incorporating 14 postgraduate certificates and diplomas). School staff acknowledged that the very extensive scope of the proposed review had made it difficult to achieve all that was required within the proposed timeframe. As a result, they had been unable to give sufficient attention to the need to link individual module learning outcomes to pathway learning outcomes and hence to awards. The team considered that grouping together so many courses for this particular review, including provision outside the core of APU, into a day and a half, was too big an undertaking to ensure that appropriate detailed scrutiny could take place at the level of the individual award or module. The team considered that the failure of the process in the case of postgraduate business programmes, despite the training and support provided for the staff involved, was due to the over-ambitious nature of the event. The team would question the validity of a process which attempted to carry out a single stage event on a very wide range and number of programmes, some new and some existing but incorporating changes, and some delivered by UK and overseas partners.
64 A different set of concerns was raised in respect of the review of another area of the University's provision. This event took place two working days before the audit and, as acknowledged by staff involved, was conducted on the basis of documentation that the staff perceived to be problematic. Furthermore, staff indicated to the audit team that the University's procedures had not been followed, to the extent that issues identified by panel members had not been communicated to the subject team formally at the indicated time before the review event. The team noted that, although the field review report was at a draft stage at the time of the audit, in the view of the team the two drafts seen in the course of the audit lacked analysis. The SED stated that 'a new feature is the requirement for the Review Panel to identify any examples of good and/or innovative practice in learning, teaching and assessment', although no examples were given in either of the draft reports. The team concluded that, while some of these matters might be a result of 'bedding in' of the new system, there was sufficient evidence to indicate that the operation of the new procedures should be reviewed by the University.
65 The outcome of a previous review in one subject area also revealed a continuing weakness in the monitoring of compliance with conditions previously set. The audit team found that the need to meet conditions imposed as a result of a previous review of the provision in 2001 had not been fully enforced at that time and had not been identified in any subsequent monitoring processes. The team recognises that the AMR process is not intended to monitor conditions of review but, had this been done, subsequent comments by external assessors would have reinforced the conditions set at the previous periodic review and might have led to a discussion of the difficulties earlier than 2004. The team noted that the 2004 field review had resulted in the imposition of some conditions relating to the same issues as those imposed in 2001 and concluded that, while the standards of awards and the quality of learning in the field were appropriate, the DAT had revealed a weakness in the quality assurance processes at the level of the University.
66 The new programme approval and periodic review processes are acknowledged by the University to place new responsibilities on panel chairs and the audit team noted, and welcomed, the training available for panel chairs and other panel members, although not all staff appeared to be aware of this training. In the case of teams preparing documentation for Foundation degrees, such training is mandatory. However, the team heard from some members of staff that they had not found the training helpful. The University had not undertaken any analysis of the evaluation forms for the training courses, but the team saw forms indicating diverse views on the usefulness of the training provided, particularly in respect of training relating to learning outcomes (see also paragraph 78, below).
67 The audit team was told that, to avoid any possibility of a conflict of interest, an individual responsible for any modules in the provision to be reviewed would not be appointed to chair the review panel in question. Nevertheless, it found that this had happened in at least one case. Given that RFBs are responsible for approving the appointment of panel members, the team concluded that safeguards were needed to ensure impartial panel selection.
68 In the light of the above, the audit team believes that it is essential for the University, as a matter of urgency, to reflect on the effectiveness and utility of its approval, monitoring and review systems with a view to ensuring that these are fit for purpose and enable the University to exercise appropriate institutional oversight over the quality and standards of its programmes and awards.
External participation in internal review processes
69 All programme approval and internal periodic review processes involve external peers, who cannot be current external assessors. The number of external peers may be one or more, depending on the scale of the provision. Where one is used, there is no need for this to be an academic so long as it is someone with appropriate subject specialist expertise. The SED indicated that it is APU's policy to seek professional accreditation for its taught programmes of study where appropriate. The Senate code of practice requires that, where appropriate, a representative member of any relevant professional body should be appointed to validation and review panels. The varying practices of the PSBs mean that the initial stages of applying for (re-) accreditation, including a PSB visit, are undertaken by the school/department with support from the Academic Office. In the light of recent experience, APU is now taking a more active central engagement role in relation to PSBs visits but the new systems adopted have yet to be internally evaluated. The audit team found that the University's approach to external participation in its internal review processes was appropriate.
External examiners and their reports
70 APU uses the term external assessors to describe its external examiners and the roles and responsibilities of external assessors are set out in the relevant Senate code of practice. The associated procedural document sets out procedures for their appointment (which are ratified by ASQEC's External Assessors Subcommittee) and expectations for their reports. APU's procedures are appropriately aligned with the precepts of the Code of practice, Section 4: External examining.
71 External assessors operate at two levels in a two-tier assessment process. They are involved at set level where they operate as subject specialists and at pathway level where they check that correct and fair processes are followed for the classification of individual students. Academic standards for a pathway and its constituent modules are confirmed and assured by the use of external assessors and, in certain cases, an assessor from outside HE (for example, from business, industry or the professions) is used in addition to an external assessor from HE. The audit team considered this to be an appropriate enhancement of the external assessor system in the case of vocational awards. The external assessors perform a valuable role in monitoring standards at module level, and at a pathway level monitor the process. While accepting that similar practice is in place elsewhere for large modular schemes, the team would encourage APU to seek ways to further assure itself of the overall standards of awards in a holistic way.
72 There was evidence to show that, in some areas, external assessors were used proactively as a resource for monitoring and enhancing quality and standards. The SED indicated that APU's formal response to external assessors' reports was firmly embedded in the annual monitoring process and much of the external assessor function in securing standards and enhancing provision is, therefore, dependent on the scrupulous implementation of the annual monitoring process in which external assessors' reports are a major element.
73 In some areas there was evidence that external assessors received a thoughtful reply to their reports and that timely action was taken by course teams. In other areas, however, the audit team saw evidence that comments and recommendations of external assessors were not identified in AMRs or reflected in the synoptic reports and the reader's reports which derived from them. Since these reports are the key element in communicating recommendations and resultant actions to higher committees responsible for quality assurance, the team considered that this compromised the process of reporting external assessors' comment. This potential for selective reporting forward of material in external assessors' reports, which is inherent in the annual monitoring process, has had the effect of allowing slow responses to external assessors' comments to go undetected. In one extreme case, repeated failure to deal with an issue which recurred in the reports of more than one external assessor persisted for some years. The current system for dealing with comments in external assessors' reports appears to have resulted in a situation where such comments can be allowed to lie dormant at field level until the point when they may be identified at periodic review. The University needs to review the implementation of its quality assurance processes in this respect in order to ensure that it has appropriate institutional oversight of responses to external assessors' recommendations and, accordingly, gains full benefit from their efforts to monitor and secure standards.
74 In the sample of material available to the audit team, the external assessors were broadly positive about the provision under their scrutiny, although some concerns were raised about quality and standards. Most external assessors' reports confirmed that programmes conformed to pathway specifications and that assignments appropriately assessed approved learning outcomes. There was evidence that, where external assessors had made negative comments, appropriate and timely corrective action had been taken in most, but not all, cases to secure supported improvement.
75 The audit team considered that the external assessors appointed were well qualified and experienced. It formed the opinion that external assessors were appropriately briefed and prepared for their duties and were generally performing them diligently and effectively. On occasions when reports had not been presented punctually, steps had been taken to remind external assessors, and reports had been forthcoming. APU is aware of the importance of external assessors for enhancing provision and expects them to identify any examples of good practice in teaching, learning and assessment which are worthy of dissemination across the University. Similarly, external assessors' reports for research degrees are analysed to identify recurring themes which can be addressed in training materials for supervisors of research students. The University is currently making further moves to secure the enhancement potential of external assessors by such initiatives as revising the pro forma for their reports in order to evaluate whether students receive sufficient feedback on their assessed work. The pro forma has also recently been revised to ensure that the external assessors' reports for set-level examining will generate appropriate publishable information to meet teaching quality information (TQI) requirements, but the University is currently discussing the outstanding issue of what information can be published from external assessors' reports at pathway level.
76 The University views its external assessors as a key element in confirming and assuring academic standards for pathways and their constituent modules. External assessor comments seen by the audit team generally help to give some degree of confidence in the current standards of achievement of students and the comparability of awards at APU with those of other HE institutions. Nevertheless, the team was concerned that institutional procedures for dealing with external assessors' reports are not sufficiently robust and are not being implemented systematically. This has had the unfortunate consequence that critical comments made by external assessors in their reports have not always been reported to higher committees and have not received the timely and appropriate responses and actions which they required. The team was, therefore, not convinced that the University has sufficient oversight of responses to external assessors' reports at field level and at higher levels in the quality structure to ensure full confidence in its current or future maintenance of academic quality and standards across all disciplines.
External reference points
77 APU claims to make extensive use of external reference points including the Academic Infrastructure and PSB accreditation. It also cites peer participation in quality assurance processes, external assessors and the use of expertise and experience across the RUP, such as discipline networks. It believes that the quality of education at APU has been enhanced by the Agency's subject review processes, which have provided a developmental experience for academic staff to engage in critical self-analysis of established practice in curriculum content, delivery and assessment and articulation of the aims of that curriculum on a subject basis. The University welcomed the principles and purpose of the FHEQ and has embedded the national framework within its quality assurance policies and procedures. Although the FHEQ is consistent with APU's long established learning outcomes approach, formal use of the FHEQ had not been made explicit in the old-style validation and review processes. This acknowledged weakness has been rectified with effect from 2003-04 in the new Senate code of practice and associated procedural document.
78 APU's learning outcomes approach to the setting of academic standards was broadly consistent with the Guidelines on preparing programme specifications (June 2000) (the Guidelines), published by the Agency. There were, however, some significant differences, particularly the Agency's focus on intended learning outcomes and the identification of learning, teaching and assessment methods/strategies to facilitate and demonstrate student achievement of those outcomes. APU's procedures were, therefore, revised to reflect the Guidelines with a pathway specification form (PSF) being introduced progressively throughout the University by requiring, with effect from 2002-03, submission of a PSF for all pathways considered under the old-style validation and review processes. The University recognised that the need to relate intended learning outcomes to learning, teaching and assessment strategies had caused particular difficulties for academic staff in 2002-03 and provided additional staff development. Evidence from recent periodic reviews and from staff indicates this is still problematic, suggesting scope for improvement in the training provided on learning outcomes.
79 The published subject benchmark statements were circulated to appropriate APU schools for information and reference. However, their formal use within the University's quality assurance processes was deferred until March 2002 on publication of the Agency's Operational Description for Institutional Audit when the statements were introduced in the old-style validation and review events. The introduction of APU's new quality assurance processes in 2003-04 provided an opportunity for the benchmark statements to be firmly embedded in the relevant Senate code of practice and procedural document. Explicit references to the benchmark statements have been made in the pathway specification form (and in the APU guidelines on completion) and in the checklist of issues to be discussed at an approval/review event. The audit team found evidence of subject benchmark statements being appropriately used as external reference points in the design, monitoring and review of programmes.
80 APU's institutional approach to the Code of practice is that those APU officers with responsibility for the areas covered by the Code are responsible for ensuring that APU's policies and procedures are consistent with the Code, are disseminated within the University and are implemented by staff. ASQEC oversees this process on behalf of Senate.
81 APU has carried out an audit process as each section of the Code of practice has been published, reflecting critically on current practice and assessing whether academic standards or quality could be enhanced by incorporating the precepts and accompanying guidance more explicitly or formally into existing policies and procedures. This reflective process has been overseen by ASQEC (and its forerunner, the Quality Audit and Standards Committee) and the findings reported to those committees, most recently in December 2003 when an update was received and considered by ASQEC. The thoroughness of this audit process was found to vary between sections of the Code with some of the grids not including the required reference to relevant paragraphs of APU policy documents, some not including the date by which action is required and others with many 'ongoing' actions, some indicated as high priority. In view of the variability identified by the audit team, APU may wish to consider adopting a more consistent approach to this audit process, ensuring that those responsible fully understand the requirements.
82 The audit team considers that, while the University has considered the purpose of the external reference points, has reflected on its own practices in the relevant areas and has taken, or is taking, any necessary steps to ensure that appropriate changes are being introduced, the process could benefit from more robust oversight and greater consistency.
Programme-level review and accreditation by external agencies
83 The SED commented on the reports of the 10 most recent Agency subject reviews since the Agency quality audit that took place in 1999. These reports evidence a reasonably strong grade profile with the most variability evident in quality management and enhancement. These reports were analysed by the Quality Audit and Standards Committee (QASC) and Senate in the wider evaluation of APU's quality assurance outcomes contained in the University's annual quality assurance report as part of its normal quality assurance processes. The University identified a number of common issues to be addressed and these were set out within the SED with the University's responsive actions. Two developmental engagements and an Agency special review of a Foundation degree took place in 2003. The developmental engagement reports highlighted the need for the University to improve the quality of feedback to students on assessed work, develop its student module evaluation at school-level and to review the complex (as perceived by students) curriculum management structure. Initially, the Foundation degree review led to a negative judgement, which was changed to a positive judgment subsequent to a fast track follow-up visit within seven weeks of the initial judgement. The University identifies this outcome as demonstrating the seriousness of its approach to academic standards and quality issues and the rigour of its processes for addressing such issues.
84 The SED acknowledged the recommendations made within the Agency quality audit report published in May 2000 and reported on the subsequent actions taken by the University. APU provided an outline of the response to each of the points for further consideration within the SED. It was clear to the audit team that the University had made significant progress on each of these matters since the last audit visit. Developments in respect of points relating to 'the infrastructure needs of postgraduate students' and 'the link between appraisal and staff development' were positive. However, the team saw evidence that the 'complex and sometimes elongated procedures' by which the University undertook to assure the quality and standards of its provision, although these had been reviewed and substantively reorganised, remained a feature of the current systems. The University had responded positively to the comments made by the previous audit team relating to the 'provision of data in support of assessment practices' and the data available to academic staff were much improved. However, in the team's view, there was scope for further improvement in both the application and analysis of data across the University and it noted that the effective use of data was dependent upon the chance skills of individuals, rather than an overall strategy for ensuring staff training in the use of the systems.
85 APU engages with a number of PSBs and their accreditation processes across a range of subjects. PSB accreditation is acknowledged at subject level with the accreditation arrangements clearly indicated within pathway specification forms. The SED stated that, in view of the varying PSB practices, schools/departments are responsible for the initial stages of applying for (re-)accreditation, including a PSB visit, with support from the Academic Office or other central units, as appropriate. Under the new quality assurance processes introduced by APU, reports of PSBs are reported to RFBs which are responsible for monitoring action required in response. Subsequent to recent PSB visits and a request from Senate (October 2003), the SED stated that protocols have been developed with a view to ensuring 'more effective central engagement in the process' with ASQEC and the Academic Office overseeing the engagements between schools and PSBs. The outcomes of, and any recommendations relating to, PSB accreditation activity are received by the University through its normal quality assurance systems. The audit team found, through the DATs, a markedly more robust approach to the management of quality assurance process at the local level where PSB accreditation was also present and believes that such reports should be considered at a central level to facilitate the sharing of good practice across the University.
Student representation at operational and institutional levels
86 The SED made large claims for the value of student representation in APU's quality systems, noting that the student experience and the quality of educational support to students are key elements of the annual monitoring and periodic review processes, that students are represented at a range of levels on major University committees and that there is a close and productive relationship with the SU, involving regular meetings between SU officers and the pro vice-chancellors and deans of students. Nevertheless, student representation and formal feedback are identified by the University as a limitation and the SWS highlighted student representation systems as an area of particular concern.
87 At institutional level, student members sit on the Board of Governors and its subcommittee, the Joint Welfare and Student Affairs Committee, (a subcommittee of the Board of Governors). Students are also represented on Senate and its main committees (for example, ASQEC, Research Policy and Standards Committee, Curriculum Committee, Learning and Teaching Committee) providing students with direct knowledge of, and the opportunity to influence, the development of policies and procedures relating to learning and teaching. Student representation has been extended to the newly established RFBs established in 2003-04. Representatives of the SU supported the view that these were satisfactory and effective arrangements and also confirmed the value of the regular meetings with senior members of the University, especially the pro vice chancellors and deans of students.
88 At programme level, student representatives have a right of attendance at field committees and are included within the annual monitoring and periodic review processes. Students have drawn attention to the patchy nature of student representation at this level and have identified a degree of dissatisfaction about the representative nature of the system and doubts about its effectiveness. The SWS drew attention to problems with election procedures in some schools, which were repeated to the audit team at some meetings with students, although it was clear that the process was operating appropriately in some areas. The further issue of incomplete communication of election results to the SU has hampered the full realisation of the SU's training schemes for student representatives. The team confirmed that there was variability about the effectiveness of field committees as a means of ensuring staff-student liaison and, while some were clearly functioning successfully and ensuring productive dialogue between staff and students, this was not universal. The team, therefore, endorses the view expressed in the SWS that there is a need for the University to maintain a greater degree of consistency in active support of the development of the student representative system to ensure feedback.
89 The SED acknowledged that APU has experienced difficulty in securing student representation on field committees from all categories of learner, including full-time students, and stated that the University will continue to work in partnership with the SU to ensure that student views and concerns are understood and responded to. The University accordingly lays stress on alternative mechanisms for achieving staff-student liaison including meetings with the SU, SU officer representation on APU committees, student focus groups, informal interactions with staff, and student interaction with the two deans of students and student support services. It was not clear to the audit team how these alternative mechanisms, which would be complementary to broad student representation, would enhance the experience of students at programme level. The team considered that it would be desirable for the University to work more closely with the SU to explore ways of supporting student representation.
Feedback from students, graduates and employers
90 APU has developed a range of mechanisms to communicate with students both formally and informally, and regularly reviews its mechanisms for obtaining student feedback. It does not believe it has fully overcome the difficult issue of obtaining valid and reliable feedback from a representative student sample across a multi-campus University of over 18,000 FTE students. Recognising the difficulty in achieving student representation on committees, APU has developed alternative mechanisms to obtain student views, including an annual student feedback questionnaire and student module evaluations. In addition, as indicated above, both pro vice chancellors hold formal SU liaison meetings every two months.
91 The annual student feedback questionnaire is a general student satisfaction survey that collects views, based on sampling, about the library, IT facilities and student support services. The University believes that the annual student feedback questionnaire has provided useful information about the quality of students' experience but recognises that there have been considerable resource costs in securing a good response rate, and continues to be concerned that it has not effectively achieved a representative sample.
92 A report on the results is produced by the Deputy Dean of Students (Essex) and is then considered by academic guidance panels. The postgraduate results are considered by an executive group of the taught postgraduate panel. The audit team found the report to be descriptive rather than analytical in its approach and had difficulty in identifying where, when and how the findings were debated, and how agreed actions were tracked and communicated to students. Staff appeared to be unclear as to which committees held responsibility for ownership and follow-through of the results. Students whom the team met were unclear about this questionnaire and appeared to know little of its findings or any resultant actions. Also, nearly one third of postgraduate taught students responding to the questionnaire were dissatisfied with the action taken in response to feedback. APU is advised to improve the systems by which feedback from students is analysed, acted upon and communicated, particularly at an institutional level.
93 It is APU policy that the delivery of all modules should be evaluated regularly by module leaders/conveners with student feedback as an important element. In the past, practice has varied, so the University has addressed this acknowledged weakness by introducing, with effect from 2003-04, a standard module evaluation questionnaire to be administered towards the end of a module. Student perception of the value of the process varies, with some believing that it lacks relevance while others find it a positive process. From what it was told, there appeared to the audit team to be some variation in the practice of posting the results on student notice boards. The process used for issuing and collecting the questionnaire also appeared to vary, which could impact on the willingness of students to be open in their feedback. In addition, there was variation in the awareness of students with regard to the opportunity for them to add comments on the reverse of the form although this has now been changed to more clearly invite these additional comments.
94 The results of the questionnaires did not appear to the audit team to be discussed and disseminated in a methodical and consistent fashion across the University. Where the feedback was used effectively, however, there was evidence of positive changes as a result of the feedback. The team considered that the practice, in the AIBS, whereby module leaders were required to provide a report if the deviation from the norm was significant, was worthy of wider adoption. Also with potential for wider use is the practice in surveying whereby interim feedback is collected mid-semester as part of a process where students reflect on their own progress; timely changes can then be made where any concerns are raised. The team concluded that the module feedback system should be made more consistently valuable to both staff and students, and was therefore supportive of APU's stated commitment to evaluate the value and effectiveness of the module feedback process at the end of the academic year 2003-04 and to make amendments where appropriate.
95 The audit team was presented with little information on feedback from graduates. APU has an alumni officer and produces a publication to keep alumni up to date. Within the DATs, the team found significant evidence of links to employers providing a positive contribution to the assurance of quality and standards through means such as an employers' forum, guest speakers, involvement in professional bodies and through APU's own students, many of whom are mature, part-time, and sponsored by employers.
Progression and completion statistics
96 APU recognises that data quality and the presentation and analysis of data have been continuing concerns during previous Agency visits and is committed to giving this high priority. The SED indicated that data quality had improved considerably since the establishment of a student data systems unit in 1999. The audit team found that acceptance by academic staff of the system's ability to calculate degree classifications correctly had contributed significantly to its credibility and had indirectly contributed to enhancing data quality. Since 2002, further work has taken place to ensure that validated modules, pathways, routes and programmes are correctly represented on the IT system. More recently, the University has augmented this work to prepare for the introduction of e-vision - a web interface for use by students and staff to access information on the student database that went 'live' in September 2003. This allows students to view their academic progress including agreed marks and grades.
97 In order to improve the analysis and presentation of data, a statistics quality working group was established in 2000 which agreed a standard template and data extraction process for the production of pathway statistics for annual monitoring purposes. A further statistics working group was established in 2002 to review the format and content of statistical reports in relation to field, pathway and module statistics and to develop, where necessary, specifications to comply with Agency, other external body reporting requirements, and APU, including drill down facilities to individual student level. This working group agreed further revisions to field, field detail, pathway and module statistical reports with the revised reports being used in the 2003-04 annual monitoring process. Direct access to the statistics was facilitated for nominated staff. The ability to interrogate the data is currently limited, although non-standard reports can be requested centrally. In addition to this direct access facility, the set co-ordinators' and field leaders' standing conferences considered APU-wide sets of statistics at their inaugural meetings in October/November 2003 and will continue to do so as a standard agenda item each year.
98 APU has explored ways in which completion rates could be improved. It recognises that student retention presents a particular challenge for institutions with a strong commitment to widening participation, and the diversity of student background and previous educational experience which such a policy actively encourages. The University has successfully targeted an improvement in student retention and HEFCE's most recent performance indicator data for full-time students commencing first degree courses show a significant improvement in APU's performance above its benchmark level. The extent to which centrally generated student data is helping to identify those areas where retention is a concern or informing actions taken to improve retention was not clear to the audit team but it noted that an in-depth study of students in the 'at risk' categories was now underway, building on work elsewhere in the sector.
99 While progression data are now accepted as being accurate and available, staff met by the audit team at discipline level recognised that improvements in the use made of the data were needed. Although AMRs require comment on the data, the team found the quality of this commentary was variable and sometimes poor. The number and percentage of students who have withdrawn from their studies (analysed by school, age, gender, ethnicity and disability, and degree classifications analysed by school) are included in the 'APU Student and Corporate Profile' produced annually and presented to Senate. The report contains no commentary on trends or on differences between schools or categories of student. The team found little evidence of discussion about the report and actions that had been agreed as a result.
100 With effect from 2002-03 APU has also developed a set of performance measures to inform its strategic planning cycle. These measures focus on the University's current strategic objectives and include data presented by school, which include student admissions; student progression, retention and completion; and data on qualifications awarded and student employment destinations. APU is just starting to use this information for targeting areas for improvement, and recognises the need to enhance the sophistication of data analysed and presented by type of student.
101 While APU has clearly demonstrated its commitment to improve the quality and use of data, the audit team met staff who expressed the view that the University was not making as much use of the data as it might. The team concluded that, although data are now more accurate and widely available, their analysis and use in evaluating and enhancing quality and standards would benefit from further improvement. It is therefore recommended that it would be desirable for APU to continue to improve the quality and particularly the analysis and application of data to understand better the performance of different categories of students and to inform the institution's approach to issues such as student retention.
Assurance of quality of teaching staff, appointment, appraisal and reward
102 The University stated that it recognises the importance of the proper selection and the motivation of its staff and links these activities to the continuous improvement of the student learning experience. Responsibility for the oversight and management of these activities lies with a pro vice chancellor who is supported by the Director of Human Resources and the Director of Staff Development. The same pro vice chancellor is currently responsible for the UCLT and chairs the Learning and Teaching Committee. Activity in this area takes place within a policy framework driven by the Human Resources Strategy 2004 to 2007. This document is accessible to all staff through the APU web site, as are its associated policy statements on human resources policy, staff recruitment and selection procedures, induction policy and the staff development and appraisal scheme.
103 The SED explained that the University's selection and appointment practices are explicit and transparent and that this process is linked to the overall University policy on equality of opportunity. There is also a linked scheme for probation, mentoring and induction in place. This is under regular review and has been recently linked to the University's 60-credit Postgraduate Certificate in Learning and Teaching (PGCLT) which is now mandatory for probationers without a formal teaching qualification.
104 The University has a well established appraisal scheme for academic staff which has been running for 10 years and is based on the framework provided in the national collective agreement. The recent achievement of full corporate Investors in People status has led to a debate about the effectiveness of academic appraisal which is currently focused upon staff development. A review with a remit to consider ways in which the scheme can be developed to focus more upon performance management to ensure a balance between individual aspirations and school and strategic plans is currently underway. The University is also considering a revision to workload planning which will further focus the appraisal process. The aim is to overcome inconsistencies in staff workloads and to provide a more transparent and equitable process.
105 Promotion opportunities for staff are also under review. Progression to Principal Lecturer is currently linked to application to posts of additional responsibility (for example, learning and teaching advisers) rather than as reward for individual excellence in teaching or research. The University is hoping that this will be revised with the introduction of the new national pay arrangements. Reward is available for staff through the annual Learning and Teaching Fellowships Scheme. £40,000 is available each year from the University's Teaching Quality Enhancement Fund (TQEF) to support up to 20 fellows in the enhancement of teaching and learning. The University has made funds available for a number of research sabbaticals to encourage greater research activity among the staff.
106 Additionally, the University is undergoing an internal review and consultation process related to the nature of the academic role and its associated activity. The main focus for this activity has been the policy document 'Expectations of Staff at APU'. This was approved by the Corporate Management Group in October 2003 and its aim is to refocus staff activity into the core elements of academic practice and away from administration. The document, which describes ways in which staff are expected to develop in their role as academics at APU, was an important element in decision-making for staff and management during the recent voluntary severance scheme. In meetings with staff, the audit team heard that the staff generally welcomed the document and found it motivating.
107 Overall, the audit team formed the view that the assurance of the quality of teaching staff was effective. There was evidence from the newly appointed staff that the policy was working as stated in the SED and they appreciated the support they were receiving. Additionally, staff from a variety of different grades who met the team confirmed the clarity of the procedures and a number of them expressed particular support for the reorientation of work away from administration and into core academic activity. The team was initially concerned that this might have a detrimental effect upon the management of the student experience if staff were to become focused upon individual research or reach-out activities at the expense of academic related engagement with their students. However, the team was reassured that, in this context, administration did not mean programme or course management, but referred to more basic elements of clerical work. It was also evident that the University senior management was committed to the implementation of the overall human resource developments and was taking a strategic approach in these respects.
Assurance of quality of teaching through staff support and development
108 The SED stated APU's belief that its staff are its most valuable resource and that the University is strongly committed to a staff development programme which ensures that all its staff are given the opportunity to develop their full potential. As indicated above, it has recently achieved full corporate Investors in People status following gradual development through individual schools and services. In a survey conducted in 2002-03, 74 per cent of staff respondents stated that the provision at APU met their development needs. The University claims that staff development provision is 'consistently rated by staff as one of the most positive aspects of working at APU' and all schools and central administrative support units have a specific allocation of budget and resources for staff development activity outside the central University provision.
109 APU has produced a series of papers relating to staff development including its staff development policy, staff development and appraisal scheme and the learning and teaching strategy. The current Human Resources Strategy 2004 to 2007 provides additional targets for specific staff development programmes within its action plan. The University provides information about its range of staff development workshops on the human resources pages of its web site. There are more than 100 courses listed. Some of these courses are mandatory for academic staff including, for example, those relating to effective appraisal. In addition, a printed compendium of all courses is produced and updated annually by the Human Resources Department and distributed to all APU staff. Specific opportunities for staff development in learning and teaching are listed in 'The Purple Book' produced by the UCLT, which organises annual learning and teaching conferences.
110 Newly appointed academic staff are required to undertake the PGCLT and this is linked to probation. While graduate students involved in teaching are offered support in teaching, the audit team learned that not all postgraduate research students who undertake teaching were aware that the PGCLT provision extended to them. The team considered that it would be desirable for the University to ensure that postgraduate research students were made aware of this facility. Staff are supported through the PGCLT and their probationary period by a school-based mentor, who is usually the learning and teaching adviser. In its current learning and teaching strategy action plan the University has set a target of achieving an increase in membership of the Institute for Learning and Teaching in Higher Education (ILTHE) of 20 per cent year on year. TQEF funding has been used, in part, to support the learning and teaching fellowships scheme. Additionally, the University has a scheme of centrally funded sabbaticals, which was reviewed by the Corporate Management Group in 2003 and the decision made to continue with the scheme.
111 Peer observation of teaching has been embedded for some time and the University expects that all academic staff will engage with the peer observation process at least once each year. The audit team found that peer observation was subject to different practices across the University which has recognised that there are difficulties in providing peer observation for hourly paid staff, who are included in some, although not all, school-based systems. The team noted that the University was moving towards a unified approach and that, in order to focus and clarify the purpose of peer observation and its distinction from appraisal and from assessed observation, it had produced a document entitled 'Peer Observation at APU: Rationale, Policy and Practice' in February 2004.
112 Overall, the audit team was able to confirm that the University had presented an accurate representation of its staff development and staff support systems. In meetings with staff there was satisfaction and praise for the systems and levels of provision. Although evidence of attendance records for all courses was provided to the team, it was less clear that this information was used by managers to monitor staff levels of attendance, especially where the courses were mandatory. In one case, attendance at the staff development course for new quality assurance processes did not appear to have helped staff undertake the writing of the new style documentation. In general, though, the team considered that the provision was comprehensive and that it constituted good practice.
Assurance of quality of teaching delivered through distributed and distance methods
113 APU offers a range of distributed and flexible learning opportunities, including distance learning, much of which is delivered through the RUP. The audit team was informed that the philosophy behind this was the alignment of student need for flexible learning patterns with appropriate pedagogies rather than any particular drive to using learning technologies per se. The University has an e-learning strategy group which has collaborated on the development of an e-learning strategy as a strand of the institutional learning and teaching strategy. The Strategy Group, which includes representation from the IT Services and the University Library, is currently discussing mechanisms for supporting e-learning developments in schools in conjunction with UCLT's e-learning unit. The University has invested in WebCT as a virtual learning environment (VLE) and each school is expected to develop an appropriate range of blended and other learning environments to meet their own identified needs. The students met by the team confirmed that there was variable provision of on-line learning environments and some were critical of the pace at which such developments were moving. Learning and teaching fellows have engaged in projects, some of which are related to on-line learning, for example, an in-house project for on-line delivery of some postgraduate modules.
114 The SED did not include specific consideration of on-line and distance learning but highlighted the university unit 'Ultralab', a learning and teaching laboratory which also provides internal technical and logistical support to schools. The University has implemented 'e-vision', an internet-based interface owned and developed by SITS for use by students and staff to access information on the student database. This system went live in September 2003. Some students met by the audit team were aware of the development but it was clear that the system had yet to be fully embedded. The team found that the strategy in relation to on-line and distance learning was led by the schools and, therefore, reflected the variable nature of subject discipline provision in the University. The team would wish to support the harnessing of all available expertise within the University (Ultralab and UCLT's e-learning unit together) in further developing on-line and distance-learning approaches.
Learning support resources
115 The SED included brief descriptive sections about the University's library and IT services. The University library monitors service quality through focus groups, on-line feedback forms and field committees, and both services are included in the annual student feedback questionnaire. Responsiveness to user feedback on library provision has been demonstrated by changes in opening hours. The SWS included data showing some variation in satisfaction with access to library and IT provision between the two main campuses. Students who met the audit team confirmed perceptions that provision at Cambridge was less comprehensive and that difficulties relating to the capital building project at Cambridge were affecting student satisfaction in the short term.
116 Research students met by the audit team were complimentary about the help provided by support staff to enable access to the resources needed, and confirmed that information about access to local and national library collections was also available to them, and that training in the use of the on-line information resources was provided. Students confirmed that disabled students' needs were considered and acted upon. They also confirmed that on-line access to information was robust and reliable, and that the University email system supported them. Some students who met the team were of the view that the University could do more to speed up the progress towards provision of learning materials on the internet.
117 The audit team was satisfied that APU was doing all it could to manage the difficulties of the building programmes, and that the two academic service areas were conscious of the differing student perceptions and addressing these through development programmes. The needs of disabled students are identified early on in the recruitment process and academic services staff are included in devising appropriate support mechanisms. Academic services are involved in University planning processes (school strategic plans are published on the university intranet) and are engaged with developmental initiatives around e-learning through membership of the e-learning strategy group. The team considered that the University was taking appropriate oversight of library and IT provision in relation to meeting the needs of students.
Academic guidance, support and supervision
118 The SED stated that the University has a diverse student population and that recognition of this has led to the belief that the student experience starts with the pre-entry process. The SED further indicated that many applicants to APU are 'first generation' HE students who lack prior personal or familial experience of HE. Consequently, the University has developed a wide range of support for applicants, including fairs, 'freefone' numbers, campus-based learning support teams and a one-stop contact centre.
119 There are two deans of students, one for each of the main campuses in Cambridge and Chelmsford, and the SED stated that they play a major role in the fulfilment of APU's aim to be a student-centred institution. This approach is summarised in an extract from the University's strategic statement which appears in both the undergraduate and postgraduate student handbooks: 'Our priority is to provide for each student a considerate, efficient, effective and enriching experience of education'. The deans of students are responsible for coordinating the University's overall approach to student support and guidance and they report directly to their respective campus managers, both of whom are pro vice chancellors. They convene meetings with directors of studies, field leaders and set coordinators with the aim of ensuring consistency and the sharing of good practice. Additionally, the deans of students work closely with the SU, providing a link between this body and senior management.
120 The audit team explored the support available to postgraduate research students and met a group of research students who conveyed a strong appreciation of the quality of support provided by their supervisors, and by central support services such as the Library and the Graduate School. The team welcomed the improvements made by the University in supporting its research students.
121 The University has decided that, given its diverse student body, no single process of induction is suitable for all students. In general terms this is reflected in the differences between Cambridge, where students are largely full-time, and Chelmsford where more students are part-time. Induction at the former is centrally driven, while at the latter it is more devolved. However, both campuses hold a university day at the end of their induction week in order to give 'focus and a sense of community'. Induction for postgraduate students is delegated to schools since the programme start dates vary. Guidance on minimum standards is available to staff in the form of a checklist. The SED identified a number of local initiatives which have been developed to support and extend the induction process, for example, the Cambridge interactive programme, and indicates the University's intention to evaluate and embed an extended induction process into the student experience, where appropriate.
122 Induction is the point at which students are allocated a personal tutor and receive their copies of the Student Handbook. Information in the Handbook is updated through the web site, but it appeared that students do not receive a new paper copy each year. The Handbook provides essential information for students on all aspects of their experience at APU. In meetings with students the audit team found that there was a high level of awareness of, and satisfaction with, student handbooks.
123 Students are provided with a guide outlining the expectations and the process of relating to their personal tutors. Staff are also provided with a guide: 'The Role of the Personal Tutor - Notes of Guidance for Staff'. All the guides can also be downloaded from the web site and are intended to provide a framework for the relationship as well as providing a source of information such as useful phone numbers and contacts. Students are expected to see their tutor at least twice in the first semester and once in the second. However, the responsibility for arranging these meetings is placed on the student. In meetings with students the audit team heard that this process was not operating satisfactorily and that, in practice, students would approach and ask for advice from staff with whom they most often came into contact. This could be a module or pathway leader who might or might not also be their designated personal tutor. In general, students were very pleased with this kind of support and most of the students whom the team met appreciated the approachability of the teaching staff. However, both the SWS and the SED identified problems with the personal tutoring system. The SED identified local variants on the standard tutoring model which have been introduced to try and improve the effectiveness of the system. There was no indication in the SED that these were being evaluated or that the good practice was being shared.
124 The SED did identify a pilot project which is based in the AIBS and is aimed at providing access to personal tutorial support, initially to undergraduates through central student advisers who are dedicated to this role. They advise on a range of matters including module choice and regulatory issues and are able to refer students to more specialist sources of support if necessary. The SED also suggested that this system should 'release academic staff for their core academic teaching, learning and research'. An interim report on the scheme (May 2004) concluded that there was wide agreement that the pilot had been successful and proposed the extension of the scheme to all schools. However, the audit team heard that there was discussion underway as to whether school-based or one stop campus-based advising services were the best solution and understood that a full evaluation was due for completion by April 2005. While it appeared to the team that this was clearly an improvement on the existing personal tutor system, it was not convinced that student advisers could necessarily overcome the difficulties identified across the University in relation to personal tutoring.
125 In April 2002, the University Centre for Accreditation and Negotiated Awards produced a position statement on work experience and personal development planning activity at APU, providing a summary of the available opportunities for students to reflect upon their learning and to experience the application of their learning within work experience. The University has now developed an APU progress file implementation strategy which defines two models and outlines the process for the implementation. Initially, in the pilot phase, schools will choose either an embedded or a stand-alone model. The latter is the University default position and will be incorporated into standard review and approval processes. The audit team formed the view from discussions with staff, that if monitored and evaluated, progress files would provide an additional opportunity to improve upon personal tutor support, as well as encouraging student reflection upon their learning experience.
126 Overall, the audit team was satisfied that the University had taken appropriate account of the relevant codes of practice and the FHEQ in its provision for student support and guidance. The team also considered that the SED provided an accurate picture of the support mechanisms at APU.
Personal support and guidance
127 The University views student support services as an integral part of its student-centred approach to the student experience. Student support services are campus-based and are the responsibility of the deans of students for Cambridge and Essex respectively. The range of services includes careers, counselling, financial aid and advice, and learning support. In addition, the SED lists mentoring activities, active community fund activity and an access centre for the assessment of students with disabilities. These services are complemented by services provided by the SU; Chaplaincy; Employment Bureau, International Office, Sport and Health; and the Nursery. The University consistently received high scores for student support and guidance in Agency subject reviews.
128 The International Office is responsible for ensuring that the specific needs of international students are met. A range of services and events is arranged, including a specific student guide and advice on visas and immigration issues. Monitoring is through an international panel, which recommends actions across the University, for example, in the provision of English language support. Students with disabilities are supported from the application stage if they declare a disability. They are then directed towards learning support teams which arrange suitable support, disseminate information to schools and attempt to ensure equity of treatment. The University hosts the National Disability Team and the East Regional Access Centre which provide an additional focus for the needs of students with disabilities. The Graduate School is responsible for postgraduate research students. Their progress and support needs are monitored annually and they have access to all of the support systems available to other students.
129 Through meetings with staff and students the audit team was satisfied that the pastoral support systems were broadly as described in the SED and the student handbooks. Overall, it appears that effective efforts have been made to recognise and meet the needs of a diverse student body. The team was also satisfied that the University had taken account of appropriate codes of practice and that an effective overview is maintained through the offices of the deans of students. However, the first point of contact for referral to support services may be the personal tutoring system. The University recognises that there have been problems with this system and the students specifically identified personal tutoring as an area of concern in the SWS. The University has introduced a student adviser scheme to address this problem and the pilot was evaluated in May 2004. The report on the pilot scheme recommends that the scheme should be widened and that further advisers should be appointed, either on a school or on a campus basis.
Section 3: The audit investigations: discipline audit trails
Discipline audit trials
130 In each of the selected DATs, appropriate members of the audit team met staff and students to discuss the programmes, studied a sample of assessed student work, saw examples of learning resource materials, and studied annual module and programme reports and periodic school reviews relating to the programmes. Their findings in respect of the academic standards of awards are as follows.
Business (postgraduate)
131 The DAT covered all full and part-time non-research degree postgraduate awards in business and management offered by the AIBS at Cambridge and Chelmsford, and comprised the following: MBA; MA International Business; MA International Business Economics; MA Business and Management; MA Arts Management; MSc Management; MSc Health and Social Care Management; MSc Human Resource Management; MSc Strategic Marketing Management; MSc Call Centre Management; MSc Education Management; MA Corporate Administration; MA Business and Marketing; MA Business and Human Resource Management (HRM); and MA Accounting and Financial Analysis. These awards are currently in the process of being brought together from previously separate postgraduate and continuing professional development portfolios.
132 In response to the Agency subject review of business and management, in March 2001, AIBS decided to develop an integrated undergraduate programme and in November 2002 agreed to adopt an integrated approach for its postgraduate and continuing professional development provision. The DSED therefore contained an evaluation of existing provision and details of the proposed new integrated postgraduate scheme. This area of provision was subject to periodic review in April 2004 but it was decided, at that event, that a resubmission of documentation was required to address a range of concerns and that a reconvened event would be held in June 2004. The review sought to cover 21 existing awards to be rationalised and replaced with 10 masters pathways (incorporating 14 postgraduate certificates and diplomas) in addition to four initial franchises and six franchises converting to the new awards. AIBS found that it was difficult to achieve all that was required within the time available and recognised that insufficient attention was therefore paid to the need to link clearly individual module learning outcomes with the pathway outcomes and hence the awards.
133 The DSED was separate from the documentation prepared for this periodic review and contained the pathway specifications for the current, rather than the new, scheme. These were dated May 2000 and, therefore, did not take account of the FHEQ, the subject benchmark statements or the Code of practice. The pathway specifications and MDFs for the proposed scheme were prepared using APU's revised templates, which reflect these requirements; it was evident that the proposals had been checked against the relevant subject benchmark statements.
134 The audit team recognised that the quality and availability of progression and completion data had improved and that APU was starting to be more responsive to the School's needs. However, AIBS staff also accepted that there was further scope for improvement in data and in its use. This was reinforced through the AMRs where there was wide variation of practice in the data presented and in their use, with much being over-descriptive and lacking in analysis. The team considers that AIBS should revise its approach to progression and achievement data and provide the necessary support and guidance to ensure staff are able to adopt a consistent and meaningful approach. The lack of a consistent approach to the analysis and use of progression and achievement data reflects a wider question about the use of data for performance analysis within the University.
135 The inconsistencies referred to above in the AMRs extended beyond the use of data. As reported in the 2002-03 synoptic report for business studies and business, law and forensic sciences,'...very few appear to follow the guidelines exactly...what is not clear is why this is so. It might be because the AMR guidelines are not suitable in all cases'. This variation in practice meant that some reports had not adopted the SMART action plan approach set out in the APU procedures, had provided no comment on the student profile, did not identify strengths and weaknesses and failed to follow through on previous actions. This inconsistency is recognised by senior staff in the School and is considered, in part, to be a consequence of bringing staff together from different cultures and practices. The audit team was told that this was being addressed through an implementation plan soon to be considered by the School's senior management group, but the team believes that the degree of acceptance of variation in practice for the 2002-03 reporting cycle was unhelpful to the School during its crucial period of preparation for periodic review. Despite this matter being raised in the reader's and synoptic reports, AIBS failed to address the identified need for improvement to the AMRs and the team came to the view that the School should ensure a more consistent and evaluative approach to its annual reporting process in line with APU guidelines. The team considered that the apparent failure of APU to require comment and/or further action by AIBS also reflected on the effectiveness of the University's procedures.
136 The 2002-03 external assessor reports seen by the audit team included recommendations to which the School had provided written responses, although some assessors commented that written feedback had not been received for the previous year. Examples of assessed work seen by the team appeared to match the module and pathway specifications and no significant concerns had been raised in relation to quality and standards by the external assessors. The team formed the view that the standards of student achievement were appropriate to the titles of the awards and their location within the FHEQ.
137 Most examples of assessment contained assessment criteria and clear guidance to students, who confirmed that they were generally aware of the assessment criteria but did cite an example of one programme where guidance for two modules was unclear and clarification from staff had been received very late. Feedback to students on assessment was generally clear and linked back to the assessment brief. Assessments did not always cover all the learning outcomes, which were often extensive in relation to the credit value of the module, and it was not always clear how the assessment linked directly to these learning outcomes. For the benefit of both students and staff, the audit team considered that AIBS should ensure that learning outcomes more clearly reflect the credit value of the module and provide a more explicit link between learning outcomes and assessment. Some of the examples seen used a module report form to summarise the marks, comment on performance and propose changes for the future. This provided very helpful evidence of ongoing evaluation and the team would encourage AIBS to ensure this is standard practice for all modules.
138 Students considered the induction process, including the APU Student Handbook, to be good and confirmed that it provided the information they required. The sample of module guides seen by the audit team varied in terms of coverage and accuracy. Given this variability, the team came to the view that AIBS should develop and adopt a minimum requirement for all module guides.
139 The audit team found that learning resources and support were appropriate for the programmes and, although students did comment that resources at Cambridge were less extensive, there was no evidence of any significant concerns in this area. Students confirmed that staff were, in the main, approachable and available.
140 Students now complete the university-wide module evaluation questionnaire for each module and the analysis of these is fed back to module leaders who, in AIBS, are required to provide a report if the score is more than 10 per cent below the postgraduate mean. A report on any aspects of good practice is required if the score is more than 5 per cent above the mean. The first use of this questionnaire was in December 2003 although AIBS had used its own methods of generating student feedback previous to this. However, there was no evidence of where the recent feedback was then discussed and disseminated in a methodical and consistent fashion. Students seemed unaware that the form invited them to add any comments on the reverse and consequently thought that the module evaluation was too limited. In the light of experience the University has revised the questionnaire to invite more clearly additional comments. The audit team would encourage AIBS to continue to seek ways of making the module feedback system more valuable to both staff and students.
141 Student representatives attend field/pathway committees but the School's style of minute writing makes it difficult to identify actions and track them through. Students on one programme considered that serious concerns they had raised had not been responded to adequately but staff appeared unaware of the students' strength of feeling. Consequently, the audit team formed the view that AIBS should review the way in which actions taken are communicated to students and ensure that, in its minutes of meetings, actions are clearly identified, owned and followed through.
142 The audit team accepted that AIBS was going through a transition phase as it attempted to bring together staff and programmes reflecting different cultures and that the AIBS management team had recognised the need to achieve greater consistency within its programmes and its procedures. In general, the team was assured that the quality of learning opportunities was suitable for programmes of study leading to the named awards.
Graphic arts
143 The DAT was based on APU's academic subject field of graphic arts within the CSA that forms a department within the School of Arts and Letters. At undergraduate level the scope of the DAT included single honours degree pathways in graphic design, illustration, fine art printmaking and photographic and digital media. The graphic design pathway is delivered at both APU CSA and Cambridge Regional College (CRC). With the exception of illustration, the field also offered each of these subjects as combined honours pathways. At postgraduate level, the DAT included the MA in Children's Book Illustration.
144 The DSED provided for the DAT was based substantively upon the document prepared for the internal periodic review process in May 2004. A draft report on the outcome of the periodic review was made available to the audit team in the course of the audit. The DSED incorporated the pathway (programme) specifications for each of the pathways and these demonstrated an overall adherence to the FHEQ and were clearly informed by relevant subject benchmark statements. While the team appreciated that the DSED had been adapted from one written for the purpose of internal review, the document provided by the subject was generally bland and lacked a sufficient level of critical reflection.
145 The University provides a standard data set, including module assessment data, to the field which is included as part of the AMR in support of the annual monitoring process. There was evidence within the DSED of some limited reflection on statistical data on degree classifications and module assessment statistics, although the latter data set was unhelpful in including data relating to modules at levels 1 and 2 delivered across the RUP. In general, though, student statistical data are regularly monitored and analysed through the annual monitoring process in accordance with the Senate code of practice. The staff team confirmed that the quality and reliability of the University's statistical data had improved considerably over recent years but recognised that they needed to use the data more strategically and systematically than was currently the case.
146 The annual quality assurance process is focused at set and field levels. As a cognate grouping of modules, the set is the focus of the annual external assessor (examiner) reports. These reports, together with staff responses, are included in an AMR that provides a quantitative and qualitative evaluation of the activities of the field over the previous year. The AMR includes a SMART action plan identifying proposed actions and reporting on progress against the actions confirmed within the AMR of the previous year. The key aspects of the AMR are included within the synoptic report, formulated at school level by a director of studies, that is received and scrutinised by the relevant RFB.
147 External assessor reports seen by the audit team were broadly positive about the provision. The reports contained several references to generous marking and the team saw some evidence of this within the sample of written work viewed during the audit, although this had been identified within the synoptic report for the School and actions were being taken to address this concern. However, strong concerns expressed within the AMR regarding the quality of the student experience for those studying on the graphic design pathway at CRC were not picked up in either the synoptic report or the reader's report that was returned to the field. In the audit team's view, this exemplifies an underlying weakness in the University's quality assurance processes, notably an over-reliance on the synoptic and reader-reporting systems to safeguard adequately the quality of the student learning experience across large and complex subject divisions.
148 Students met by the audit team raised a particular concern about the accessibility of some module options as they found that these were not always available to them owing to restrictions of the timetable. Staff were very sympathetic to this issue and were able to describe to the team the various measures taken to address this concern, while recognising that there were always likely to be modules that would not be available to every student. They described their practice of providing multiple delivery of popular modules or, as when a specific module was requested by a small number of students, making modules available through tutorial delivery. Staff were keen to point out that students also had the option of independent study modules that would enable them to cover specific areas outside their core studies. In addition, students could broaden the scope of their studies by taking one level 2 module at level 3.
149 The audit team saw a range of students' written work during the visit and evidence of written assessment feedback from practical work. Where examples of assessed student work were seen, these were appropriately ranked and conformed to the FHEQ level descriptors and pathway specification forms. Written feedback to students was relatively brief but was generally appropriate to the grade awarded. Students spoke enthusiastically about the value that they placed upon the oral feedback from assessment, particularly that presented in the context of studio critiques. However, some students reported that they had on occasion been surprised by differences between oral feedback and the grade awarded. The team considered that the staff's practice of re-issuing module briefs towards the end of each practical module to help keep the students on track constituted good practice. On the basis of the evidence of the student work seen during the visit and the external assessors' comments, the team can confirm that the standard of student achievement is appropriate to the titles of the awards and their location within the FHEQ.
150 Student information and handbooks are provided with a field handbook, supplemented by module information guides for each module. The field handbook provides some basic information about the Department of Art and Design as well as outline information on modules, pathways, learning outcomes and grade descriptors. Module information guides and handbooks are available to support student learning. The documentation provided for students was of variable quality and did not always make an explicit reference to the relevant assessment criteria. However, the students were very appreciative of the 'How to Write an Essay Guide' produced by the staff as an additional support to student study. The audit team would encourage the subject staff to consider undertaking a thorough review of the documentation provided to students to ensure a greater level of consistency and a full description of assessment schemes.
151 The DSED makes a number of references to learning resources and the improvement, maintenance and development of material resources is clearly a continuous concern to the field, as is evidenced through the minutes of staff and field committee minutes. The students showed a realistic set of expectations in relation to physical resources and enjoyed a good dialogue with staff about such matters. They were also very appreciative of the quality of the staff resource. The students were unable to cite any examples of the impact of e-learning on their learning experience apart from their use of the internet to access research resources.
152 Formal staff and student liaison takes place at field committee meetings which have a high level of student participation and minutes of these meetings demonstrate a productive staff-student dialogue. Students were able to provide ready examples of improvements to their learning experience that had been made as a direct result of their feedback. Module questionnaires were another source of student feedback and the summaries of module feedback viewed during the audit visit were broadly positive about the provision; students welcomed the opportunity to comment on their learning experience in this way. The audit team was impressed by the student-managed module evaluation process, with student representatives both distributing and collecting the questionnaires and working directly with the Field Office to process the forms. The students clearly liked the informality of staff-student relations and, although aware of the formal personal tutoring systems, they indicated to the team that they rarely needed to rely on these.
153 The audit team was satisfied that the quality of student learning opportunities provided for students was generally suitable for the pathways of study leading to the named awards. However, the team would wish to encourage the staff team to review student module guides, project briefs and module handbooks and ensure that these, and the associated assessment documentation, contain a full description of the assessment criteria.
History
154 The DAT covered the following pathways which are delivered on the Cambridge Campus: BA History (single honours); BA History (combined honours); BA 20th Century European History and politics (single honours); BA 20th Century European History and politics (combined honours); and MA Victorian Studies. These pathways are managed by the history field which is part of the recently formed School of Law, Languages and Social Sciences.
155 The DAT was supported by a DSED prepared for the internal periodic review conducted the week before the audit. There were some difficulties inherent in this approach and staff acknowledged that using their SED for both internal and external purposes had been unhelpful.
156 The pathway specification forms attached to the DSED were those prepared for the internal validation process, and an updated set was provided for the audit team at the beginning of the audit visit. The team found that the final versions of specifications were, in most cases, clear and useful. It was evident that in writing them the subject team had given careful thought to articulation with APU's institutional level descriptors which are themselves explicitly aligned with the FHEQ. This articulation was further evidenced from consideration of module specifications, demonstrating clearly to the team how subject content and related skills were identified in a progressive manner in clear alignment with the level descriptors in the FHEQ, enabling well-articulated and coherent programmes of study. The team considered that all the curricula under review provided for very clear coverage of the areas and skills identified in the Subject benchmark statement for history with appropriate attention being paid to time depth and geographical spread and to the development of subject-specific and generic skills. Although the staffing base for history is relatively small (calculated at 5.25 FTE), nine members of staff contribute to the programme, thereby allowing for a range of modules to be offered covering British, European and American topics, both modern and early modern. The team noted that collaboration with other units (particularly philosophy and politics) has been exploited to extend the time depth to include material relating to the cultural concepts originating in the Classical world.
157 The audit team noted that the history pathways had been designed in such a way as mainly to meet the precepts in relevant sections of the Code of practice. Useful documents, mapping achievement of outcomes by module against pathway specifications and defining overall assessment methods, had been prepared for the recent periodic review. These documents demonstrated that an appropriate variety of assessment methods was used in the history pathways.
158 While the current curricula in history appropriately relate to the Subject benchmark statement, students met by the audit team expressed concerns that the range of choice in modules has been significantly reduced as a result of the voluntary severance scheme leading to a significant number of redundancies in history. External assessors and subject staff also voiced this concern. Students were particularly critical of the way in which decisions about staffing, which were bound to have substantial impact on their learning experience, had been communicated to them.
159 The audit team noted a recurrent curriculum matter, identified by students, external assessors and periodic review teams, relating to some modules which could be studied for the award of either 10 or 20 credits. Doubts had been expressed concerning the appropriateness of content in terms of academic level and about assignment loads for the differently weighted versions of these modules. The periodic review report of 2001 stipulated, as a condition of approval, the need to revise module learning outcomes to achieve clarification and differentiation. The problem had nevertheless been allowed to persist and the conditions had not subsequently been met and, while they appeared to be signed off by the panel chair shortly after the event, the University had drawn the Department's attention to the need to meet the conditions in 2002. Later external assessors' reports had continued to raise the matter and the need for action on this issue was identified in the 2002-03 AMR action plan. Although learning outcomes for some relevant modules were eventually adjusted after November 2003, in time for the periodic review of 2004, it was apparent that the issue was still not completely resolved at the time of the audit. Consequently, the 2004 periodic review report included a condition for future approval which repeated the substance of the 2001 condition. While it is to be hoped that the matter will now finally be resolved, the team was concerned that appropriate action on the question, which continued to be signalled by external assessors, had not been taken in a timely manner.
160 Although statistics relating to progression and achievement were presented and analysed purposefully in the 2000-01 and 2001-02 AMRs, discussion of the Student Profile in the 2002-03 AMR was, by contrast, descriptive and attenuated to the point where it could not helpfully inform judgements about management of the history provision. The subject was treated equally minimally in the SED prepared for the 2004 periodic review and the DAT. Subject staff expressed doubts about the usefulness and accessibility of data provided centrally, expressing the view that they did not allow easy comparison of input and output in terms of students' entry qualifications and final achievement.
161 The 2004 periodic review was conducted on the basis of documentation which the subject team considered to be problematic. Furthermore, the procedures laid down in the Senate code of practice do not appear to have been followed, the audit team hearing that matters identified by panel members in the light of their consideration of the DSED had not been formally communicated to the subject team before the review event. The team was provided with two draft versions of the periodic review report and, while their draft status was recognised, both were found to be lacking in analysis and did not identify examples of good practice in learning, teaching and assessment although this is one of the stated purposes of periodic review.
162 AMRs seen by the audit team relating to the History pathways were of variable quality, especially in respect of trend analysis based on quantitative data and the development and achievement of SMART action plans.
163 External assessor reports confirmed that students were achieving learning outcomes appropriately. The audit team saw evidence demonstrating that external assessors' comments were appropriately addressed. The team was able to confirm, from its discussions with staff and students and consideration of students' work, that explicit concentration on supporting students' development of academic skills had been particularly effective in securing a great improvement, to the considerable benefit of the students as learners, in terms both of generic and subject-specific skills. Examples of assessed work amply demonstrated achievement of the outcomes set out in pathway specifications. Students' work was fairly assessed and feedback provided was formative and helpful. The team met students who confirmed their appreciation of the careful marking and informative feedback provided on their work.
164 Students indicated that they found module guides helpful and the examples provided to the audit team were generally full and informative, although one would have benefited from minor updating from year to year. The team considered that the overall History Student Handbook was a less full document than those provided for modules and heard that not all students were aware of it as a resource.
165 Students were content with the library and information resources, and the examples of students' work demonstrated that they had had access to an appropriate range of materials. They were particularly appreciative of the support provided by library staff. Although the students valued highly the friendly atmosphere within history, they were strongly critical of the physical environment in which they, and staff, had to work. In part, this was a result of current building work at the Cambridge Campus and some students acknowledged that there might be improvement in this respect in the future.
166 The University processes for module evaluation are fully implemented in the history pathways and some tutors conduct deeper evaluation than is required. The results of analysis of the evaluation returns are posted on notice boards and students confirmed that reports on evaluation exercises appropriately reflected their views. There is a growing culture of enhancement within the field, with minutes of staff meetings reflecting issues arising from student feedback, full implementation of developmental peer review informed by thoughtful guidelines prepared by the School's teaching and learning adviser and a series of subject-based staff development events.
167 Students are represented on field committees, although it was accepted by staff both that it was not always possible to secure secret elections and that there were problems with communication of election results to the SU. Students confirmed that staff reacted in a timely way to their proposals and cited, as an example, the case of a recent module offered through a different field for which the assessed assignments were not entirely suitable for history students but where history staff had secured the addition of extra topics for history students.
168 The audit team considers that the standard of student achievement in the history pathways is appropriate to the titles of the awards and their location within the FHEQ, and that the quality of the learning opportunities available to students is suitable for programmes of study leading to the named awards.
Social work
169 The DAT covered the following pathways: Diploma in Social Work; BSc (Hons) Social Work (top up); BA (Hons) Social Work; BSc (Hons) Professional Social Work Practice; Practice Teaching Award; and the Child Care Award. The pathways are delivered at a number of sites in addition to Cambridge and Chelmsford and are available in part-time, full-time and supported open learning (SOL) modes. The pathways are managed by the social work field which is part of the School of Community Health and Social Studies.
170 The DSED provided in support of the DAT was a modified version of materials prepared for periodic review and for accreditation by the General Social Care Council (GSCC) (the latter leading to approval and commendation). The DSED incorporated the pathway (programme) specifications for each of the pathways and these were in accordance with the FHEQ and were informed by the relevant subject benchmark statement. It was evident that the subject staff had engaged with the process of preparing the documentation for the audit and they reported they had received assistance, when necessary, from the Academic Office. The audit team found the supplementary information section of the DSED helpful in relating the periodic review documentation to the requirements of the audit.
171 The documentation made very little use of data but the staff indicated that data were used as a part of the annual monitoring review process and the available annual monitoring review reports confirmed this. The staff stated that the data now available to them had improved although they felt that there was little central guidance on how and where data should be used and reported. They did confirm, however, that they had received good support from a member of staff in producing their data, which included individual tracking of students and comparison of cohorts, gender or ethnic groups, if required. The audit team formed the view that the data system was adequate but heavily reliant upon particular individuals with developed skills.
172 The audit team found that AMRs in respect of the field conformed to University requirements. The field had introduced the practice of checking on the progress of the action plan during the academic year, which staff had found helpful. Similarly, the staff had made efforts to ensure that there was feedback from the RFB following the presentation of the synoptic report. The team considered that these were examples of good practice and improvements upon the system. External assessor reports seen by the team were positive about the provision and the team noted that any actions suggested had been followed through and reported upon to the satisfaction of external assessors.
173 The audit team saw a range of students' assessed work and these also conformed to the FHEQ level descriptors and were appropriately ranked. The subject team is innovative in its assessments and has introduced a range of methods, specific to the subject, including patchwork text and the recent introduction of inquiry-based learning (IBL). These are clearly imaginative and provide students with opportunities to reflect upon their learning in the subject and in professional practice in creative ways. Since the student experience of IBL was varied, the subject team is encouraged to ensure that staff support for this new approach to learning is closely monitored by course managers. Students also stated that the BSc Social Work (top up) was a small pathway and that some of the teaching, undertaken in conjunction with students on health programmes, was not relevant or satisfactory to the social work students.
174 The audit team heard that some students had been disappointed with their first year placement because they had not been placed with the organisation they had requested. Other students said that they had gone ahead and organised their own placements to be sure of being able to get the specific experience they wanted. The staff recognised the difficulties and told the team that, while there were always constraints in giving all students their exact first choices, they were working to provide as close a match as possible. The team would encourage the subject team in its endeavours.
175 In its discussions with students, the audit team was told of a concern that students following some study modes might be at a disadvantage compared with others. It was claimed that some students had been given an opportunity to submit assessed work for formative feedback prior to formal submission and had been given an indicative grade. This was considered more difficult for some students to do than others, giving rise to questions of fair practice and equity. The team noted that there was a clear University policy that such feedback should only apply to a specified amount of the work and that no indication should be given on likely gradings. The staff who met the team stated that their practice conformed with the University's policy and were unaware of this as a matter of concern. The team considered that the full-time staff were fair, committed and supportive of students and this was reinforced by the students. However, the team also formed the view that there was a question to be asked about the management and dissemination of information to part-time staff and to those staff who contributed to the programme from other fields. The subject team indicated that it would give further explicit guidance to such staff.
176 Overall, the audit team was satisfied that the social work provision offered an excellent example of the university processes in practice and, in some cases, that the subject team had exceeded these requirements in order to enhance the student learning experience. There was also clear evidence of good practice in relation to employers, including a forum which brought together partner agencies engaged in teaching on a regular basis. However, the team also came to the view that more needed to be done to ensure a consistency of experience for students across the various modes of delivery and especially where this involved part-time staff or staff from other fields.
177 The audit team found that the quality of learning opportunities was suitable for the programmes of study leading to the awards named above and that these programmes were appropriately located within the FHEQ.
Surveying
178 The DAT covered surveying, located within the Department of the Built Environment and the School of Design and Communication Systems. The provision included: BSc (Hons) Building Surveying; BSc (Hons) Building Surveying Engineering, BSc (Hons) Quantity Surveying; BSc (Hons) Construction Economics and Management; BSc (Hons) Real Estate Management; BSc (Hons) Planning and Development Surveying; BSc (Hons)/BSc/HND/HNC Property and Surveying; BSc (Hons) Surveying; MSc Conservation of Historic Buildings; Diploma of Credit: Building Surveying; Diploma of Credit: Quantity Surveying; and the Diploma of Credit: Real Estate Management. The provision is based at the Chelmsford Campus and offered in part-time, sandwich and full-time modes. Approximately 65 per cent of students study part-time. The field is due for review in 2005 and has not, therefore, been subject to the University's new periodic review process. Similarly, there have been no validations using the new process.
179 The DSED, which incorporated programme specifications for all the pathways included in the DAT, was written specifically for the purposes of the audit. The audit team found the DSED to be well written and helpful. Although the programme specifications mainly predate formal publications of the Subject benchmark statement for building and surveying in 2002, the stated aims and learning outcomes of the programmes demonstrated an overall adherence to the FHEQ and are compatible with the Subject benchmark statement, insofar as these apply. The pathway forms are not in the current APU format and make reference to the old systems of levels of learning outcomes. The fact that they are not dated renders them confusing to the reader. However, they do contain useful information and there is a clear articulation of the overall aims and learning outcomes of the programmes. Additionally, staff have devised a detailed grid indicating where and how each of the skills identified in the specifications are dealt with within modules.
180 The main programmes in surveying were last reviewed and revalidated in 2000-01 to meet the revised requirements of the Royal Institution of Chartered Surveyors (RICS) with new courses coming on line more recently to provide pathways for students not meeting the professional body's entry profile requirements. A significant number of students studying within the field are on sandwich programmes and staff indicated, in particular, their recent involvement with the Code of practice on placement learning in which an audit of their processes mapped against the Code had indicated adherence. Staff indicated that they were currently undertaking a further mapping exercise in respect of the Code, Section 3: Students with disabilities.
181 Student progression and completion are monitored using centrally provided data. This informs the student review process, carried out each semester, to identify students with problems in order that remedial action can be initiated to try and prevent academic failure before it occurs. Analysis of progression, awards, gender and entry qualification statistics is undertaken as part of the AMR. The interpretation is descriptive but shows some reflection on the figures produced. The commentary in the AMR points to some trend analysis, but this is limited and relates primarily to entry profile and progression.
182 Staff met by the audit team considered that there was an effective system in place to identify modules needing attention due to high failure rates. Internal monitoring follows the University model and focuses on module and field levels. The surveying field draws primarily on modules from within two sets: Surveying and Construction Management. The DSED states that this means that the field AMR is 'one stage removed for the delivery of the pathway'. This is acknowledged to be unsatisfactory and efforts are being made to address this matter. At module level, staff indicated that module leaders were a key component in the quality chain. All modules are reviewed briefly every year in the light of external assessor reports, student evaluation and the statistical analysis. If the module is deemed to need some minor modification this takes place through the fast-track procedure, by referring the request for changes to the relevant director of studies. The Set Co-ordinator is also consulted to ensure that proposed changes are mapped against the skills and outcomes grids to ensure that drift has not taken place. Staff spoke enthusiastically of the technical assistance provided by student data services in relation to module review. Staff told the team that it was customary, within the field, to review the modules 'more radically' every three years. A further check on adherence to learning outcomes takes place at the module level as coursework assignment briefs are circulated internally among staff.
183 The DSED claimed that the AMR was reflective and analysed strengths and weaknesses of the provision and the audit team found this to be so. However, although the AMR was found clearly to follow up issues raised in the field committee, it did not fully evaluate external assessors' comments. The previous year's action plan is reported and progress against targets is detailed. Similarly, an action plan with responsibilities for action and monitoring, together with clear time scales, is included in the AMR. The AMR is subject to a reader's report and the approved report feeds into a synoptic report that is submitted to the Business and Technology Regional Faculty Board. Papers supplied with the DAT indicated that this process was regarded as unhelpful in achieving enhancement at the field level. Staff met by the team said that enhancement was now much more a feature of the process than previously and that learning and teaching conferences provided the mechanism for dissemination of good practice. The DAT provided clear evidence that at a local (field/pathway) level annual monitoring is undertaken in a timely and efficient manner with some evaluation of evidence. However, and as identified elsewhere in the report, the team identified weaknesses in the institutional oversight of the University's quality assurance processes.
184 External assessor reports are generally supportive and indicate that standards are comparable to those achieved at other institutions and appropriate to the requirements of the programme and the professional body, the RICS. External assessors are attached to sets, rather than pathways. Accordingly, they only review the work of modules within the set with which they are aligned. Even in a comparatively small pathway such as surveying, students will normally take some modules from outside their 'home' set. The audit team noted that one assessor had identified a wish to be able to review the work of students for modules belonging to the Construction Management set. The team learned that external assessors are sent copies of all assignment briefs so that they can select a work sample to inspect and considered this to constitute good practice. The team found that all external assessors receive a full reply to their reports from the set coordinator and most assessors were able to report that action had been taken by subject staff by the time of their next visit. Where assessors do not submit their reports by the required date, evidence was provided that follow-up took place. In general, there was clear evidence that external assessors contribute positively to the assurance and enhancement of quality and standards at module and set levels.
185 Assessment strategies are provided in MDFs and these are then either repeated or expanded in module guides. The DSED indicated that staff had made 'considerable efforts' to ensure that assessments were aligned with intended learning outcomes. The audit team found that students appreciated this and the clear information on assessment normally provided. The team saw a range of student written work which demonstrated that the work set was appropriate to the pathway specifications. Assessment criteria are issued with assignment briefs to ensure that students are aware of the expectations in terms of coverage and standard. Some of the project work set was imaginative and, generally, work was of a standard appropriate to the titles of the awards and their location within the FHEQ. External assessors had confirmed that marking was fair and the team noted appropriate and supportive feedback in many cases. Students confirmed that feedback was generally timely, constructive and related to criteria, although the depth of feedback was variable. A process of internal moderation in which a random sample of work is check marked by another member of staff is routinely used to ensure consistency across modules and assignments. All dissertations are blind double-marked.
186 The audit team saw a number of module guides together with a school document providing guidance for writing study guides. Almost without exception these guides were examples of good practice and students confirmed that they were useful and informative. They are particularly appreciated by part-time students, who make up the majority of the student body in the pathway.
187 The DSED states that the Head of Department is responsible for learning resources and that resource allocation is driven by curriculum need. The adequacy of resources is checked at the point of approval and monitored by module leaders and the annual monitoring process. The academic liaison librarian monitors the library resource and inducts students in the use of the library. The staff resource is monitored through the appraisal scheme and the resource has enabled several members of staff to be granted sabbaticals in recent years. Additionally, several staff have achieved ILTHE membership. The use of peer observation has been a special feature within the pathway provision with the DSED stating that all staff are involved in the process. The scheme adopted, which is now to be replaced by a university-wide scheme, was innovative in that it was based on reflection by the observer on his or her own practice instead of concentrating on that of the observed.
188 The DSED indicated that the physical location of the pathway accommodation is currently 'blighted' due to the continuing building works, and occasional shortages of space have had to be overcome with 'ingenuity'. The view expressed in the DSED is that both human and physical resources are adequate and suitable and the audit team was able to confirm this self-evaluation. Learning resources and support were found to be both fit for their purpose and sufficient. Students spoke of their satisfaction in particular with library provision and the growing use of e-learning possibilities, with learning materials for some modules now supported by materials offered through WebCT, the e-learning platform adopted by the University. Students spoke highly of the practice orientation, supportive attitudes and professional standards of the academic staff, several of whom are ILTHE members. The Department also makes appropriate use of part-time staff to ensure the relevance to practice, and this is appreciated by external assessors who view them as a way of ensuring currency of the curriculum.
189 Students complete standard module evaluation forms at the end of each module and some of these were made available to the audit team. In general, students were very supportive of the teaching they had received. The evaluations are included within the annual monitoring process which gives several examples, module by module, of issues raised and action taken. These range from concerns relating to over-assessment to the quality of overhead projectors. Students confirmed that the issues they raised both in student evaluation forms and in field committee meetings were taken seriously by staff and that action followed.
190 The audit team observed a commendable element of practice in relation to feedback. All students are supplied with a mid-semester module evaluation form that is used within the Built Environment Division. This form asks students to evaluate the module delivery and to consider their own learning in respect of it. It calls on them to review whether they need to make adjustments to ensure their success as well as inviting them to pass comments back to staff for changes to delivery etc. within the module. Given the concerns about attrition rates, this is viewed as commendable practice worthy of wider dissemination.
191 Although the SED indicated that it is difficult to gain active participation in field committees, especially among the part-time student body, this is not a concern within surveying. Students confirmed this evaluation and commented that field meetings were very positive and well attended. They found that staff were responsive to their needs and they were able to cite specific examples of action following their comments. They also commented that inter-field committee meetings were not well attended by students.
192 The audit team considers that the standard of student achievement in surveying is appropriate to the titles of the awards named above and their location within the FHEQ and that the quality of the learning opportunities available to students is suitable for programmes of study leading to the named awards.
Section 4: The audit investigations: published information
The students' experience of published information and other information available to them
193 The audit team was able to consult the University's undergraduate and postgraduate prospectuses, the institutional undergraduate and postgraduate student handbooks, the Research Student Handbook, a range of targeted publications for regional and international students, a range of subject-based course and module handbooks, and the University's web site, on which many of the aforementioned publications are available on-line. The quality and accuracy of these publications was discussed with representatives of the SU and with students in subject areas where DATs were conducted. The team was also able to take account of the evaluation of information presented in the SWS.
194 Student feedback on publications which informed their learning was generally positive, with the SWS confirming high levels of satisfaction with institutional handbooks and general, but not universal, approval of module handbooks. Institutional student handbooks are updated annually by the deans of students and are given to all new students at registration. The audit team concluded that institutional student handbooks were comprehensive and useful, with the Research Student Handbook being particularly helpful. The team saw examples of module handbooks which were found to be variable in extent, currency and quality, a view confirmed by students who indicated that module handbooks were of differing usefulness, ranging from those that were useful and informative to those that contained old or misleading information. There were also examples of discrepancies between material presented in handbooks and material appearing on the web site. In most cases, students and external assessors confirmed that handbooks contained accurate information about learning outcomes and assessment, although there were examples of discrepancy between module handbooks and approved pathway specifications.
195 In its SED, the University acknowledged a limitation in respect of communication with students in relation to non-standard items (for example, unforeseen timetable changes). The SWS and students met by the audit team confirmed that this was an issue, drawing attention to low levels of satisfaction with the University's general communication with students, particularly about changes. The University is aware of the problem, and various mechanisms are in place and being developed in order to improve communication with students. The University routinely uses articles in the student newspaper, Apex, to convey information about such issues as plagiarism and academic practice. Information visual display units, controlled through field offices, have also been installed in all major entrances to University buildings and gathering points, providing daily information to students on both academic and non-academic matters. Since September 2003, all students have been allocated email accounts and the University expects to see increasing use of electronic modes of communication with students. A recent innovation has been the introduction, in September 2003, of e-vision, a facility giving all APU students direct access to aspects of their personal and academic record, as part of a general move towards the use of internet-based communication systems. The system had been accessed 22,000 times by students by December 2003.
196 While acknowledging that the University was aware of some limitations in respect of communication with students, and was taking steps to address them, the audit team concluded that, in the case of published materials intended to support student learning, it would be desirable for the University to achieve consistency of practice by establishing minimum standards for a range of documentation, including internet-based materials. This would secure equity of support for students and would enable the identified good practice in some areas to be replicated in others.
Reliability, accuracy and completeness of published information
197 The SED was mainly written prior to the availability of HEFCE's document, Information on quality and standards in higher education: Final guidance (HEFCE 03/51), and before it was possible for the University to have considered its implications fully. The SED made clear that HEFCE 03/51 is under current consideration and that progress was being made using HEFCE's document, Information on quality and standards in higher education (HEFCE 02/51) as a reference point. The audit team established the University's current position in terms of the latest guidance from HEFCE and was able to confirm that considerable progress had been made in respect of information collection and management. The information sets recommended in HEFCE 02/15 were currently available in paper form, with some being available electronically. The annually prepared student and corporate profile provides an overview of student, staff and financial information and is available on the University intranet. Its potential as a management tool is being realised progressively, and the statistical data available are being used to plot trends against performance measures introduced to inform strategic planning at institutional level from 2002-03. The datasets have been used to inform decisions on staffing levels across the University.
198 After receipt of HEFCE 03/51, the University established a working party on TQI, which will oversee final developments to ensure availability of the complete dataset by December 2004. Appropriate additions have recently been made to the pro forma for external assessors at set level to enable collection of distinctive or innovative features in relation to assessment processes. Other aspects of HEFCE 03/51 are still under active consideration but it is clear that the University is aware of the requirements and is working towards achieving them.
199 The audit team was able to review a wide range of University, school and field publications as well as the institution's web site. The University has established protocols for checking the accuracy of its publications and information sets. Overall, the information provided to students appeared to be appropriate and accurate and, in spite of some evidence of variability in its comprehensiveness, there was wide support from students who met the team, and in the SWS, for the quality of institutional and subject handbooks. Programme specifications were generally full and accurate, although there were occasional minor discrepancies between their content and other materials circulated to students, including material on the web site. The University is aware of this and has plans in hand (including greater control of web content through content management software) to ensure greater general consistency.
200 The audit team considered that the University was alert to the requirements set out in HEFCE 02/15 and 03/51 and was taking steps to fulfil its responsibilities in respect of the requirements. It also considered that the information published by the University about the quality of its programmes and the standards of its awards was reliable.
ISBN 1 84482 201 X
next >> Findings
Top