177 An institutional audit of Goldsmiths College (the College) was undertaken during the week 14 to 18 March 2005. The purpose of the audit was to provide public information on the quality of the College's programmes of study and on the discharge of its responsibility for conferring degrees of the University. As part of the audit process, according to protocols agreed with the Higher Education Funding Council for England (HEFCE), the Standing Conference of Principals and Universities UK, four discipline audit trails (DATs) were selected for scrutiny. This section of the report of the audit summarises the findings of the audit. It concludes by identifying features of good practice that emerged from the audit, and recommendations to the College for enhancing current practice.
The effectiveness of the institution's procedures for assuring the quality of programmes
178 Proposals for new programmes are considered for approval against academic and resource criteria. The procedures are set out in the Quality Handbook and elsewhere. Initial scrutiny by senior College managers of the resource implications of a proposal may be followed by its further development for academic consideration by the Programme Approval and Review Sub-Committee (PARSC). If PARSC recommends approval, the proposal is passed to the Academic Committee, and thence to the Academic Board for final approval. The self-evaluation document (SED) stated that the programme specification is now established as the key document in the approval process, and that the College associates this step with an improvement in the quality of submissions. External readers' reports are said by the SED to be a key reference point for PARSC in providing specialist academic input on new proposals. The audit team was of the view that the value of this input would be enhanced if external peers were present at the relevant meetings of the sub-committee and enabled to participate more directly in their outcomes.
179 The College has introduced this year a process of (APR). The process draws together the various programme monitoring inputs, already in place in College departments, in a cohesive annual review of the programme's operation. This is assembled into a report by the programme leader which is circulated within departmental committees and provided also for the Quality Affairs Office (QAO). According to the Quality Handbook, summaries of these programme reports are sent up into the central committee system and feedback returns to the level of the Programme Monitoring Committee (PMC), a staff/student liaison committee into which students make inputs on their experience of the programme. The audit team found a low level of familiarity among staff with the procedural aspects of this very new annual review mechanism. Indeed, the SED itself stated that the dissemination routes through committees and to external examiners and students remain to be determined. The College is therefore advised of the need to ensure both that the process is reviewed early, as intended, and that the process going forward is effectively embedded across the institution.
180 Periodic programme/departmental review is also a recently introduced process and borrows in its aims and methods from external practices in quality assurance. The College intends that cumulative APRs will inform the periodic review of programmes. Reviews are based on an SED and supporting evidence, and are conducted by a panel which includes external subject reviewers. The review leads to a report and a consequent departmental action plan, the process being centrally overseen by the Academic Committee. The SED stated that departmental feedback on periodic review had been generally very positive. Evidence seen by the audit team indicated that the process is robust and effective and a valuable addition to the College's quality assurance arrangements.
181 The College considers that student feedback is an integral input to the quality assurance process for programmes and this is collected via several routes: programme monitoring committee meetings with student representation; course evaluation forms and by student representation on departmental and College committees. The QAO works closely with the Students' Union in ensuring feedback mechanisms are working effectively. This initiative should enhance the effectiveness of the College's procedures for assuring the quality of programmes. A range of programmes is offered, many of which have a professional or vocational focus, in a significant number of areas the College enjoys strong ties with employers working in the relevant occupational fields. Links with employers are also cultivated through placement schemes offered in a number of College departments. While departments maintain a range of contacts with graduates, there is no formal College policy or provision for collecting feedback from them.
182 The College's involvement in distance-learning methods is presently confined to its work as the lead college for English and Computing in the External Programme of the University of London. While academic standards are, through the assessment process, under University control, quality assurance of the student's learning experience is shared between the University and the College under a new framework established by the external programme. This incorporates joint annual and periodic programme review processes.
183 The College's collaborative provision is presently limited to the validation of programmes in Art Psychotherapy leading to the award of a College Diploma. These programmes, which the SED stated are essentially the same as the internal College programme, are delivered at two partner institutions, in Germany and Switzerland. The validated programmes are subject to the College's internal quality assurance procedural framework including programme monitoring, course evaluation, external examiner reports, and annual and periodic programme review. The SED notes recent training and guidance provided for partner institutions on the quality assurance procedures and the Academic Infrastructure, with particular attention to the new APR process and the development of programme specifications. The audit team saw evidence of these initiatives, which it considered represent good practice. The College may wish to ensure that this good practice is consolidated through the embedding of processes assuring quality and standards; their manifest integration with College processes; and the continuing development of central oversight of collaborative quality assurance by the Academic Committee.
184 The SED acknowledged the unresolved issue of how comparability of standards across the Diploma programmes in Germany and Switzerland and at the College is to be measured and demonstrated. This was a particular concern of the QAA Report of December 2002 on the collaboration in Switzerland, which stated that 'the College's stewardship of the standards of its validated award [is] weakened by the limited information it receives relating to the comparison of the achievements of students [at the partner institution] with those completing a similar programme in the UK'. The College is seeking to establish a process of retrospective cross marking to scrutinise relative student performance and marking standards across the programmes. Consideration is also being given to identifying and appointing a single external examiner for the award. The audit team's view is that while the cross-marking initiative may provide some useful clarification in respect of standards of achievement and marking, it falls short nonetheless of moderation by College staff of work being assessed. Further, the validated programmes continue to lack an external examiner with the linguistic scope and experience of UK higher education that would seem to be required for full and effective oversight of the assessment process and standards of achievement. The College is advised of the continuing need to resolve the issues relating to the assurance of the standards of awards in its collaborative provision. For the Lausanne Diploma programme offered under direction of the College's partner institution in Bern, staffing and communication difficulties have led to a cessation of recruitment. The College is advised of the need to give close monitoring attention to the quality and standard of the learning experience provided for continuing students.
185 The College considered in its SED the stage it had reached in the development of its quality management arrangements for the approval of new programmes; monitoring and reviewing existing programmes; gathering the views of students; employers and other external stakeholders; and its external and collaborative provision. The audit team noted the emphasis placed on the development and consolidation of the links between the centre and the departments, in particular through the roles of the the Academic Committee and the QAO, and the recent initiatives that have been taken by the College to ensure that all its academic provision, including its external and collaborative provision, receives equal critical oversight from the centre. These initiatives will be reinforced by the continuing development of processes for monitoring, reviewing and embedding institutional quality assurance systems which the College has begun to undertake.
The effectiveness of the institution's procedures for securing the standards of awards
186 Although, the College's committee system as a whole has been under review since 2003-04 and the precise relationships between the components of that system may still be evolving, the audit team noted the formal involvement of the Academic Board, the Academic Committee, the College Board of Examiners, departmental boards and the oversight of the Warden and the Pro-Warden (Academic) in the maintenance of academic standards. At the same time the regulatory framework for the College's programmes of study at institutional and programme levels has been the subject of extensive review following recommendations made in the QAA audit report of 2002.
187 It is intended that this revised framework will address more fully the needs of the Academic Infrastructure and be implemented from 2005-06. The revised framework will provide a potentially more robust basis upon which the sub-boards and the College Board of Examiners particularly can achieve a view on the comparability of awards and their standards across the College's programmes, the current claim for which the audit team found insufficient evidence in support. At departmental level the team found appropriate evidence in support of programme and assessment standards being consonant with the intentions of the Academic Infrastructure and that these standards were clearly signalled to students and endorsed by external examiners. At departmental level it was also clear that comparability of standards was based on the views of external examiners and external intra-subject comparisons rather than any significant internal comparison across disciplines.
188 Statistics on recruitment, progression, withdrawals, and degree classification are collected by the College and in some instances have been analysed by gender, age, home region, ethnic origin, and disability. Retention is identified as an important part of the College's learning and teaching strategy and this has led to close monitoring of retention rates at departmental and programme level. The College has created a post of Data Analyst specifically charged with improving the quantity and quality of data concerning recruitment, admissions and widening participation. It also recognises the need to undertake a more systematic analysis of student withdrawals, and acknowledges that there are limitations in the way that the current student records system is constructed. It is acknowledged that there is still room for improvement in the level and scope of analytical data used to underpin recruitment and admission processes. A working group has been established to consider, progression, recruitment and retention and there are plans to provide departments with comparative data so as to enable them to benchmark their performance in terms of completion, progress and retention.
189 The College invests 'the key role in the assurance of standards of awards' with its external examiners who act by 'providing an external peer view of the achievements of students'. The College has procedures in place for the nomination and approval of external examiners which ensure appropriate peers are engaged, and such individuals receive supportive documentation which equips them adequately for their role. The reports produced by external examiners, together with the input of staff external to the College into programme approvals, provide a concerted support to the assurance of the standards demanded by its programmes and those achieved by its students. External examiners attend assessment sub-boards in departments and therefore help to assure standards at programme level. However, although they are invited to attend the College Board of Examiners they rarely do, again making the claim that the College Board of Examiners ensures comparability of standard difficult to sustain fully.
The effectiveness of the institution's procedures for supporting learning
190 The College oversees its provision to support learning through a number of committees and service departments. The Finance and Resources Committee (FRC) undertakes long-term learning resource support planning and reports direct to Council. The Communications and Information Policy Committee, which is a sub-committee of FRC, advises on policy and produces an overall strategy to promote the use of resources that includes the library, IT, human resources and teaching accommodation. Information Services manages a wide range of resource provision, including the library and Computer Services. Academic and pastoral support for undergraduates is overseen by the Student Services Committee, which reports to the Academic Board. Student Services manages a comprehensive range of support offices, as well as working with academic departments in order to enhance the student learning experience. Individual academic support is supplied via a personal tutor system, which is the responsibility of heads of department. Postgraduate support is managed by the Postgraduate Research Committee (PGRC) that reports to the Academic Committee and ultimately to the Academic Board. Human resources are managed by the Human Resources Strategy Committee, which reports to FRC. The constituent service departments have developed detailed plans and self-evaluation procedures.
191 Information Services is housed in the Rutherford Information Services Building (RISB) that offers library and IT support to students. The College has invested in book stock and has a pro-active strategy to develop learning technologies, in particular, the VLE 'learn.gold'. Responding to student concerns, the College has undertaken a pilot to extend the RISB's opening hours. Induction and information provided by Information Services is detailed and user friendly. Teaching accommodation has been increased, particularly with the development of the Ben Pimlott Building. One of the DATs, media and communications, highlighted serious concerns about the quality of teaching accommodation, which are recognised by College, and plans are being draw up to remedy the situation.
192 All undergraduate students are allocated a personal tutor, although the system is not employed uniformly across departments and, consequently, the College has set up a working party to review the process. This working party has now reported with recommendations that are to be implemented from 2005. Individual academic support from staff is uniformly good, although feedback on assessed work varies across departments. The PGRC has been proactive in the development of provision for postgraduate students, initiating learning support and successfully monitoring its effectiveness. Handbooks providing academic guidance are generally comprehensive and useful.
193 Through Student Services, the College offers a wide range of personal support, particularly at admissions where there is special provision for non-traditional entry and disabled students. At department level, personal support is primarily provided by the senior tutor, who liaises with Student Services as necessary. Additional support is offered via the departmental administrators. Pre-entry and induction information is thorough.
194 In matters of the quality of teaching and staff development the audit team noted that the staff Training and Development Unit (TDU) has a College-wide focus. The unit designs and delivers customized development activities, provides general advice to individuals on continuing professional development and provides guidance to departments on the design and delivery of local staff developments. Programmes are offered with a planned and bespoke approach to help staff develop within the College. The audit team also noted the good practice of Centre for Excellence in Learning Teaching Technology (CELT) which was established to support College staff in their use of using all aspects of technology in their learning and teaching activities and in particular the availability of CELT Fellowships to departments.
195 The individual service departments and committees generally offer students good learning support, in terms of resources and in academic and pastoral terms. In particular, the College has been proactive in assuring and enhancing the quality of IT and postgraduate provision, as well as in staff development. However, the responsibility for learning support is situated in a number of committees, with the FRC reporting to Council and the Academic Committee and Student Services Committee reporting to the Academic Board, and thereby to Council. While the QAO is undertaking to provide feedback and liaison in some areas, notably teaching accommodation, currently there is no clear and effective decision-making process at senior committee level. As a result the effectiveness of learning support provision remains variable.
Outcomes of discipline audit trails
Design
196 The DAT covered the following programmes offered by the Department of Design: the BA (Hons) in Design; BA (Hons) in Eco Design; BA (Hons) in Computing with Design for the World Wide Web; MA in Design Futures; MA in Design - Critical Theory and Practice; and MRes in Design. The DSED was written specifically for the audit and this exemplified the Department's approach by focusing on two programmes. The audit team examined a range of material including assessed student work, comments of external examiners and met with students and staff.
197 The audit team formed the view that the standard of student achievement in the programmes was appropriate to the title of the awards and their location within the FHEQ. This view was reaffirmed by external examiner reports. Programme specifications were detailed and clear and followed the Subject benchmark statement for art and design, the relevant codes of practice, the FHEQ and College guidelines. Programme and module aims, assessment criteria and intended learning outcomes, were clearly described in student handbooks and effectively communicated to students. There is effective monitoring of student progression and retention data and module evaluation. Student feedback is effectively obtained although the auditors noted that some issues were not always addressed in a timely fashion. Provision is enhanced through a range of mechanisms, including staff appraisal, staff development and external links with the profession. Published information provided to students is comprehensive and accurate. Teaching accommodation is acknowledged to be not fully adequate, however, measures are in place to address this. Personal and academic support for students is good. The team concluded that the quality of learning opportunities available to students was suitable for programmes of study leading to these awards.
History
198 The DAT covered the degree programmes for BA (Hons) in History, BA (Hons) for English and History and the MA in Cultural History. From its study of students' assessed work, and from discussions with students and staff, the audit team formed the view that the standard of student achievement in the degree programmes was appropriate to the titles of the awards and their location within the FHEQ. The programme specifications set out appropriate educational aims and differentiated learning outcomes for levels of study, and that these are linked to the teaching and learning styles, support and assessment that students could expect to receive. The programme specifications are referenced to the Subject benchmark statement for history.
199 Student evaluation of the programmes is favourable, noting in particular the academic support provided by the staff of the Department both formally and informally. The audit team concluded that the quality of learning opportunities provided for students, including postgraduate research students was suitable for the degree programmes offered.
Media and communications
200 The DAT covered the BA honours degree programmes in Anthropology and Communications; Media and Communications; Communications and Sociology; International Media; Media and Modern Literature; and the MA programmes in Digital Media; Media and Communications; Transnational Communications and the Global Media; Image and Communications; Feature Film; Journalism; Screen Documentary; Television Journalism; Radio; and Script Writing.
201 From its study of students' assessed work, and from discussions with students and staff, the audit team was able to confirm that the standard of student achievement is appropriate to the titles of the above awards and their location within the FHEQ. Programme specifications were made available for each of the awards used to 'exemplify the Department's policies and processes' and these had been informed by the appropriate subject benchmark statements. They set out the appropriate educational aims and learning outcomes and gave details of learning and teaching methods, and details of assessment. In general, the examples of assessment sampled by the team were satisfactory, and work is being undertaken to decrease the variability between tutors.
202 The Department makes a substantial effort to widen participation and has a commitment to offering places to mature candidates, and those from the working class and ethnic minorities. Many students come from the College's PACE programmes and are recruited from the local area. Overall, student evaluation of the programmes was positive. The audit team concluded that the quality of learning opportunities is suitable for programmes of study leading to the named awards.
Psychology
203 The DAT covered the following programmes: the BSc (Hons) in Psychology; MSc in Occupational Psychology; and MRes in Research Methods in Psychology.
204 The definition of the relevant programmes was clearly stated for students and was fully referenced to the Academic Infrastructure and the relevant professionally defined standards. Assessment was clearly defined and demonstrated appropriate congruence with institutional expectations, and the methods used were fairly applied. Annual monitoring adhered to the newly introduced procedural requirements of the College and illustrated beneficial outcomes for the Department. Handbooks and other programme support materials formed a worthwhile contribution to the academic support of students, who endorsed their learning environment and the support to learning they received from staff. The audit team formed the view that the student achievement was appropriate to the titles of the awards and their location within the FHEQ, and the quality of the learning opportunities is suitable for programmes of study leading to the named awards.
The use made by the institution of the Academic Infrastructure
205 Since 2001 consideration of the implications of the Academic Infrastructure for the College and its programmes has passed through a number of stages. These have included a Quality and Standards Working Party which through its successor body (Programme Specification Working Party) promoted the development of programme specifications which took account of the FHEQ level descriptors and subject benchmark statements. Substantially this objective has been achieved by the College. Subsequently the management of the College's ongoing engagement with the Academic Infrastructure has been through the QAO, which has produced guidance on the preparation of programme specifications in relation to the Academic Infrastructure and has more recently undertaken a further mapping of the College's position with respect to the QAA Code of practice. The QAO has also reviewed this position and advised the Academic Committee on the policy committees and offices with primary responsibility for specified areas of the Code. While recognising this engagement with the Academic Infrastructure at institutional level is fully complemented by appreciation of the guidance it offers to departmental staff in their use of the Code, most of the key quality assurance and standards related processes do not make explicit their linkage with the Academic Infrastructure in their descriptions. Notwithstanding the relative lack of explicit reference to the Academic Infrastructure in College documentation the audit team found ample evidence at programme level of its use and the impact it was having on the maintenance of quality and standards.
The utility of the SED as an illustration of the institution's capacity to reflect upon its own strengths and limitations, and to act on these to enhance quality and standards
206 Overall, the SED provided a clear and fair account of the College's state of development in the assurance of quality and standards. While it provided only relatively limited evidence of the engagement by the College and its departments with the Academic Infrastructure, there was evidence that there had been a systematic approach to the consideration of the elements of the Academic Infrastructure. The SED acknowledged the recency and indeed newness of a number of its initiatives and was frank about instances where arrangements had not yet been finally determined. The SED recognised the need, which the College is addressing, for strengthening of central oversight of quality assurance across the academic departments. This means, as the SED said, combining an ethos of monitoring with one of facilitation and support.
Commentary on the institution's intentions for the enhancement of quality and standards
207 In its SED the College indicated its belief that learning and teaching development and enhancement lies at the heart of its approach to the assurance of quality and standards. The College was also able to demonstrate that at both institutional, service and departmental levels initiatives in support of the continuous improvement of learning and teaching were occurring. The institutional component of this was based upon the emerging impact of the work of the LTO and the clarification of its distinctive role, relative to the QAO.
208 The College also looks upon its evolving learning and teaching strategy and the responses it is designed to evoke at departmental level as a future major source of enhancement opportunities and activity. However, in general terms both the Academic Strategy and the institutional and departmental learning and teaching strategies are of relatively recent origin and as yet cannot be seen to be the demonstrable origin of concerted enhancement across the institution.
209 In terms of its quality and standards related processes, their recent re-appraisal, augmentation and presentation indicate an intention by the College to continue to improve such processes although it was more difficult to ascertain how the changes would enhance the capture and dissemination of the good practice that might be identified from their use. Indeed, although enhancement initiatives were a feature of the College's activities there was little evidence of a systematic or strategic approach being taken to the identification of good practice and the assurance of continued institution wide enhancement.
Reliability of information
210 Under the College's Publications and Publicity Policy, there are clear lines of responsibility for ensuring that materials are accurate. Heads of department are responsible for ensuring that the content of their printed and web based recruitment materials is factually accurate. The prospectuses, handbooks and other promotional literature scrutinised by the audit team appeared to it to be accurate, and the processes for ensuring continuing accuracy to be generally reliable.
211 The QAO is responsible for coordinating the College's Teaching Quality Information (TQI) for the HERO website. The audit team can confirm that the College has already provided material to meet its responsibilities for the provision of TQI. The team was informed that the College has a project team working on the data and that they are rewriting and revising programme specifications in order to meet the needs of the target audience.
212 On the basis of the information available to it at the time of the audit the team concluded that the College was alert to the requirements set out in document HEFCE 03/51, the final guidance on Information on quality and standards in higher education, and was moving in an appropriate manner to fulfil its responsibilities in this respect.
Features of good practice
213 The following features of good practice were noted:
i the initiatives of the QAO in developing the College's quality agenda, including staff development initiatives in collaboration with partner institutions (paragraphs 32 and 118)
ii the commitment and proactive contribution of members of institutional and departmental support staff to the enhancement of the student experience (paragraph 33)
iii the sharing of good practice via regular meetings of the departmental administrators with representatives of the QAO and the Registry (paragraph 33)
iv the support being given to learning and teaching through initiatives being undertaken by the Learning and Teaching Office and through the availability to departments of CELT fellowships (paragraph 101).
Recommendations for action
214 Recommendations for action that is advisable:
i to monitor and evaluate its proposed arrangements, when and where implemented, for Academic Board's exercise of its responsibilities for quality and standards, with particular reference to the role of Academic Committee and other key committees (paragraph 28)
ii to assure itself that it has in place comprehensive means of monitoring, evaluating, developing and improving the effectiveness of its quality assurance systems, and their impact on enhancement (paragraph 30)
iii to provide on a continuing basis such support for the QAO as will enable it to ensure that the College's quality assurance processes become fully and thoroughly embedded within departments (paragraphs 32 and 51)
iv to consider further the efficacy of its current systems for ensuring the comparability of its awards across departments at both undergraduate and postgraduate levels (paragraph 90)
v to develop a clearer and more effective decision-making process for ensuring that the learning infrastructure meets student needs (paragraph 107)
vi to resolve outstanding issues relating to the assurance of the standard of awards in collaborative provision (paragraphs 20, 48 and 119), including the close monitoring of the ITP programme delivered in Lausanne in order ensure a comparable level of learning experience (paragraph 120).
215 Recommendations for action that is desirable:
i to provide a clearer and more strategic formulation of its intentions for enhancement (paragraphs 43 to 46)
ii to consider enhancing the involvement of external peers in the final approval of new academic programmes (paragraphs 48 and 56)
iii to encourage a more systematic approach to preparation for and response to professional bodies in order to enhance College learning through the sharing of experience and good practice (paragraph 149).
Appendix
Goldsmiths College's response to the audit report
The College is very pleased with the positive and supportive nature of the audit team's report and QAA's broad confidence in the management of the quality and standards of our provision, both at institutional level and in all four Discipline Audit Trails. We welcome the commendations of good practice, particularly in relation to new developments and initiatives in learning and teaching and the College's quality agenda, as well as the recognition of the important contribution of the support staff to the student experience.
We are pleased that the audit team was able to concur with the College's own view as expressed in the self-evaluation document that there had been significant and effective work on issues identified in the previous audit of 2001. The College was also gratified to note that the areas the audit team advised for further attention were those on which the College was already working, such as the committee review and aspects of collaborative provision.
The College is also pleased that the audit team found that its self-evaluation document provided a clear and fair account of the College's state of development in the assurance of quality and standards and that it had addressed appropriately the Academic Infrastructure and the requirements of Teaching Quality Information.
The College is confident that the quality assurance policies and processes commended by the audit team will prove effective in supporting a strategic approach to the further enhancement of the quality of learning and teaching and the student experience.
A revised action plan taking into account the audit team's valuable and constructive feedback will be considered by the Academic Board at its meeting in October 2005 and will provide an important input into the next round of the strategic planning process and the continuous evaluation and review of the College's quality assurance framework.
The College would like to express its thanks to the audit team and QAA for its professional and constructive approach and for the helpful and thorough commentary on its work provided by the report, which will inform us in the further development and improvement of the management of quality and standards in the College.
ISBN 1 84482 309 1
Top
QAA updates
QAAtweets
QAAtube