Higher Education Quality Council

Trinity College, Carmarthen
Quality Audit Report
February 1997

ISBN 1 85824 304 1

---

CONTENTS

• FOREWORD
• THE REMIT
• THE COLLEGE CONTEXT
• THE AUDIT PROCESS
• SYSTEMS AND ARRANGEMENTS FOR QUALITY ASSURANCE
• DESIGN, APPROVAL AND REVIEW OF PROGRAMMES OF STUDY
• TEACHING, LEARNING AND MONITORING THE STUDENT EXPERIENCE
• STUDENT ASSESSMENT AND THE CLASSIFICATION OF AWARDS
• FEEDBACK AND ENHANCEMENT
• STAFFING, STAFF DEVELOPMENT, PROMOTION AND REWARD
• CONTENT OF PROMOTIONAL MATERIAL RELATING TO ACADEMIC PROVISION
• VALIDATION, FRANCHISING AND OTHER FORMS OF COLLABORATIVE PROVISION
• THE CHARTER FOR HIGHER EDUCATION
• CONCLUSIONS AND POINTS FOR FURTHER CONSIDERATION

FOREWORD

1 The Higher Education Quality Council (HEQC) was invited by Trinity College, Carmarthen to undertake a quality audit of the College. The Council is grateful to the College for the willing co-operation provided to the members of the audit team.

THE REMIT

2 The Quality Assurance Group of HEQC undertakes quality audits according to the following terms of reference:

(i) to consider and review the mechanisms and structures used by those institutions in membership of the owner bodies of the Higher Education Quality Council to monitor, assure, promote and enhance their academic quality and standards, in the light of their stated aims and objectives; and to undertake a similar consideration and review in respect of other institutions of higher education, at their request;

(ii) to comment on the extent to which such procedures in place in individual institutions reflect appropriate good practice in maintaining and enhancing quality, and are applied effectively;

(iii) to prepare and publish a report on each audit undertaken;

(iv) to prepare and submit an annual report to the Board of Directors of the Higher Education Quality Council;

(v) to liaise with the other groups of the Higher Education Quality Council, drawing their attention to such matters and findings which may be of interest to the higher education system and which may merit further research and development; likewise receiving benefit from the work of the other groups.

Terminology and abbreviations

3 Throughout this report, the following abbreviations have been used:

ACR Annual Course Review;
AP(AA) Assistant Principal (Academic Affairs);
AP(QA) Assistant Principal (Quality Assurance);
AP(SS) Assistant Principal (Student Services);
ADWG Academic Development Working Group;
BA (Hons) Dyniaethau BA (Hons) Humanities Welsh medium
HEFCW Higher Education Funding Council for Wales;
ITT Initial Teacher Training;
KPMG Klynveld Peat Marwick Goerdeler;
Lampeter University of Wales, Lampeter;
OHMCI Office of Her Majesty's Chief Inspector for Education in Wales;
PGCE Postgraduate Certificate in Education;
QAU Quality Assurance Unit;
QSC Quality and Standards Committee;
USP University (of Wales) Subject Panel.

THE COLLEGE CONTEXT

Origins and development

4 Trinity College ('the College')was founded in 1848 by the National Society for the Promotion of Religious Education and is the only denominational voluntary college of higher education in Wales. Before 1993 it was directly funded by the Welsh Office. In that year, however, the Higher Education Funding Council for Wales (HEFCW) assumed responsibility for its funding.

Location

5 The College is located on a single campus on the outskirts of the market town of Carmarthen, overlooking the Towy estuary. The College regards its location and its single-site operation as attractive to potential students, a significant proportion of whom are drawn from surrounding counties. The College currently provides 466 residential places in single rooms and 182 in shared accommodation.

Academic portfolio

6 Until 1982, the focus of the College was the initial and in-service training of teachers. In 1982, it began to extend and diversify its academic provision. Initially this took the form of ordinary and general combined studies BA awards, subsequently extended to BA and BSc single and joint honours awards and, more recently, studies leading to taught postgraduate awards and to MPhil and PH D awards in certain subjects. In addition, the College offers two Humanities programmes of study: one through the medium of Welsh (BA (Hons) Dyniaethau) and one through the medium of English (BA (Hons) Humanities). The College's undergraduate and postgraduate programmes of study lead to awards of the federal University of Wales ('the University').

Student characteristics

7 In 1994-95 the College enrolled 1,601 undergraduate and PGCE students, of whom 831 were studying for BEd and PGCE awards and 699 for BA and BSc awards. In 1995-96, as a consequence of the Government's policy of consolidating the numbers of those following higher education programmes of study, the College's enrolment was reduced to 1,530. The College is keen to develop postgraduate studies and to enhance its research profile. At the end of the 1994-95 academic session there were 39 students following programmes of study leading to taught postgraduate and research awards. Of these 13 were full-time students (12 MA and one MPhil) and 26 were part-time (16 MA and 10 MPhil). There have been significant changes in the number and characteristics of academic staff since 1982. These changes are considered elsewhere in this report (see below, paragraph 93).

Mission Statement and Strategic Plan

8 The College's Mission Statement gives prominence to its religious foundation; its commitment to bilingual teaching; its continuing role as a major provider of teacher education and training and the development of a diversified programme of undergraduate and taught postgraduate awards, together with research initiatives and research degrees in selected areas. In its Strategic Plan for the period 1995-96 to 1998-99, the College declares its aim 'to be a significant provider of higher education within the total all-Wales System by providing a distinctive College style, campus-based education offering:

'(i) a high standard of teaching based on scholarship and research and delivered within a strong framework of student support services;

'(ii) a bilingual, stimulating environment providing access to national and international culture;

'(iii) a Christian environment tolerant of the faith and beliefs of others and encouraging intellectual, emotional and spiritual growth demonstrated in academic achievements and in a cohesive social community.'

9 The College is active in the wider Christian community and assumed a leading role in establishing the Church Education Foundation, an organisation involving the Church of England and the Church in Wales. The College considers that the availability of its Halliwell Centre, a conference and meeting venue, has assisted in building links with industry, commerce and the public sector in the Dyfed area, and more widely throughout South Wales. The College has made the expertise of its staff in Welsh language teaching and environmental protection available to commercial and public sector organisations.

Governance and management

10 The Board of Governors is responsible for determining the educational character of the College. It delegates responsibility for the conduct of the College's affairs to the Principal. The College is a registered charity and the majority of the 33 members of its Board of Governors are 'Foundation Governors', appointed to represent the Anglican and trustee interests of the charity. At the time of the audit, the College was preparing to change its status to that of a company limited by guarantee, whilst retaining its status as a registered charity. A shell company had been established and a 'Memorandum and Articles of Association' had been prepared, which provide for a reduction in the membership of the new Board from 33 to 18, but with Foundation members remaining in the majority. A statement of the College's constitutional and management arrangements, prepared by the College, is appended to this report.

Senior Management Team

11 The Principal is supported by a Senior Management Team (SMT) consisting of the Deputy Principal, three Assistant Principals and the Director of Finance and Support Services. The constitution of this team was settled in September 1995, following a review of the structure and remuneration of the College's senior management by the consultants KPMG. As part of a redistribution and rationalisation of responsibilities, the Deputy Principal took over responsibility for the College's relations with funding and other external agencies and relinquished his role as Superintendent of Examinations. At the same time, the Assistant Principal (Student Services) (AP(SS)) assumed responsibility for student support services, student registration and examinations, and academic responsibilities formerly shared by a number of senior postholders were consolidated within the brief of the Assistant Principal (Academic Affairs) (AP(AA)).

Assistant Principal (Quality Assurance)

12 In addition to the changes described above, the Board of Governors designated a new post of Assistant Principal (Quality Assurance) (AP(QA)) intended to emphasise the importance attached by the College to academic quality and the structures and processes intended to assure and enhance it. The responsibilities of the AP(QA) include quality matters throughout the College and the audit team noted that she had been given primary responsibility for developing the College's systems of academic quality assurance over a three year period and, in addition, was responsible for oversight of the College's personnel function (see below, paragraph 94).

Heads of department and course directors

13 The College is organised into 14 departments, each organised on a subject or discipline basis with its own head of department. Responsibility for managing programmes of study, including the College's four major undergraduate and taught postgraduate programmes, rests with course directors. The heads of department and the course directors together with the SMT constitute what might be described as the 'extended executive' of the College, which takes a prominent part in the work of the Academic Board and its committees.

Relations with the University of Wales

14 The College prepares students for the awards of the University of Wales and members of its teaching staff participate in the work of the University's academic structures. The College is represented on the University's Committee on Academic Quality and Standards. The part played by the University and its Subject Panels (USPs) in the quality assurance of the College's academic provision is discussed further in paragraph 48, below.

THE AUDIT PROCESS

15 The College provided a wide range of clearly presented briefing materials, including an Overview document expressly written to support the audit, examples of documents describing and illustrating the work of its principal quality assurance processes, and its promotional materials. At a briefing meeting held to discuss these materials the audit team proposed a programme of meetings for the visit and requested further information to assist it to understand the working of the College's quality assurance arrangements in their proper context.

16 The audit visit took place from 30 April to 2 May 1996 and involved meetings with some 120 staff and student members of the College, including:

• the Principal;
• the Deputy Principal, the Assistant Principals, and the Director of Finance and Support Services;
• heads of department and course directors;
• members of the Academic Board; the Quality and Standards Committee; the Research Committee; academic development working groups and course, co-ordinating and department committees;
• members of the teaching staff, and the learning support staff, including members of the Library, resources centres and the Information Technology Centre;
• members of the student support staff including staff engaged in counselling, welfare, accommodation, chaplaincy and careers advisory work;
• officers of the Students' Union and undergraduate and postgraduate full- and part-time students.

17 The College normally conducts its written and oral business in both Welsh and English. The audit team wishes to express its gratitude to all those with whom it discussed the College's quality assurance arrangements for their readiness to conduct the affairs of the audit, where appropriate, in English.

18 The audit team comprised Dr J Cowan, Dr D H Furneaux and Dr G H Williams, auditors, and Mr R A Platt, audit secretary. The audit was co-ordinated for HEQC by Dr D W Cairns, Assistant Director, Quality Assurance Group.

SYSTEMS AND ARRANGEMENTS FOR QUALITY ASSURANCE

Policy for the quality assurance of academic provision

19 The Mission Statement of the College commits it to attain 'a high standard of teaching based on scholarship and research and delivered within a strong framework of student support services' (see above, paragraph 8). It has adopted a range of policies to assist it to achieve this aim including:

'(i) Promoting Quality by the operation of a Quality Assurance System based on sound procedures and commanding full staff commitment.

(ii) Encouraging the professional and academic development of teaching staff and the skills training and general development of all staff.'

Responsibility for quality

20 The College Academic Board is responsible 'for ensuring that [its] awards are consistent and comparable in standard with similar awards in other Colleges' and for 'defining the aims, mission and standards of the College's academic programme.' The Board's formal terms of reference appeared to the audit team to be broadly specified and few in number and, as set out in the College's briefing materials, did not include the responsibilities described above, taken from the 'Overview' document prepared by the College. Members of the Board appeared to the team to be unsure of the Board's specific role in relation to academic quality and standards, but also spoke of the openness of its discussions and the ease with which items concerning such matters could be placed on its agenda.

Quality and Standards Committee

21 The Academic Board has delegated the operational performance of its quality assurance work to its committees. The Board's Quality and Standards Committee (QSC) is responsible 'for the oversight of all matters relating to the standards and quality of taught courses and their academic environment', including course and programme validation and monitoring; the consideration of external examiners' reports; and the organisation of quinquennial reviews. It is chaired by the Deputy Principal and performs an intermediary role between the Academic Board and the course committees (see below, paragraph 24). Its membership consists of the three Assistant Principals, the course directors of the College's four major undergraduate and postgraduate programmes of study, the Head of Welsh, the Librarian, and two members nominated by the Academic Board. A place has recently been provided for a student representative on this Committee, the membership of which is dominated by senior members of the College. Elsewhere in the College's briefing materials the QSC is referred to as 'the Assistant Principal's (Quality Assurance) vehicle for co-ordinating, implementing and developing approved policies and procedures.'

Academic Development Working Groups

22 Following the reorganisation of senior management responsibilities described above in paragraph 11, the College established six academic development working groups (ADWGs), for Information Services; BA (Hons) (English Medium); BA (Hons) (Welsh Medium); BSc (Hons); Initial Teacher Training (ITT) and Higher Degrees. Each is chaired by the Assistant Principal (Academic Affairs) (AP (AA)), and its membership includes appropriate heads of department, course directors and lecturers with particular responsibilities. The remit of each Group is set by the AP (AA) and includes providing a forum for the discussion of curriculum matters and academic development, functioning as a 'think tank' for innovation, offering advice to other College committees, and liaising with them. Reports of meetings of ADWGs are presented to the Academic Board by the AP(AA) and are widely circulated throughout the College.

23 Members of ADWGs who discussed their work with the audit team considered that they provided valuable forums for discussion. The College intends that ADWGs will make a contribution to its processes for the design, development and validation of new programmes of study, (see below, paragraph 25), and to its work on monitoring and developing academic standards (see below, paragraph 80). At the tine of the audit the ADWGs had only recently begun to meet, and matters such as their relations with other College committees and reporting arrangements had yet to be considered (see also below, paragraph 31).

Course committees

24 Each course committee is normally convened by the AP(QA) and chaired by the AP(AA). Course committees bring together representatives of each discipline or subject area which contributes to a particular programme of study, together with elected student representatives from the programme, and are responsible for monitoring teaching and learning and the student experience. They receive annual reports from the relevant course director and feedback from student questionnaires. Course committees normally meet twice or three times a year with the most important being the first meeting at which the annual course review (ACR), prepared by the course director, and supplemented by subject (sometimes referred to as 'strand') reports, is considered (see below, paragraph 52). Course committees report to the College Academic Board through the QSC.

25 The audit team noted that one of the intentions underlying the College's designation of ADWGs in September 1995 had been that they should take on a key role in the initiation, planning and development of new programmes of study (see above, paragraph 23). At the time of the audit it was too early to assess their likely impact on this aspect of the College's quality assurance arrangements.

Co-ordinating committees

26 Heads of department and course directors form the membership of the College's co-ordinating committees, which are intermediaries between the course committees and departmental committees. Departmental committees provide information to the course committees but appeared to serve more as working groups than committees with a formal status within the quality assurance structure.

Quality Assurance Handbook

27 Many of the College's regulations, for example, those relating to assessment and examinations, derive from those of the University of Wales and have been consolidated in the College's Quality Assurance Handbook ('the Handbook') which was first issued in 1993. It has since been revised annually, most recently in 1996, and is a substantial document consisting of eight main sections, a bibliography, and an annex. The Handbook brings together information from the Strategic Plan on the history and mission of the College, developments in teaching and research, academic support services, staffing and administration strategies, and estates and financial strategies.

28 Sections in the Handbook describe the role and remit of key academic support units and committees, and the College's processes for internal validation and monitoring of programmes of study and courses. Other sections deal with external examiners and their duties, examinations, and the operation of examination boards. The Handbook also contains information on staff appraisal and staff development. The annex consists of a set of standard forms, guidelines and checklists used as instruments in the operation of the quality assurance system.

Quality Assurance Unit

29 The Quality Assurance Unit (QAU) provides administrative support for the quality assurance and staff development work of the QSC including servicing course, programme and curriculum development and validation; the administration of the annual monitoring process, including checking on the implementation of action plans; the management of the College staff development programme; the development of a course database and liaison with bodies such as HEFCW and HEQC on quality assessment and audit. The QAU is also responsible for reviewing and updating the Handbook and issuing it to staff, and for ensuring the College complies with the terms of its associated college status with the University of Wales.

30 The QAU also produces an annual report for the College's Academic Board, the contents of which include an account of its own work; the main issues identified by course committees in their reports; matters considered and actions taken by the QSC; the outcomes of the validation programme; the outcomes of quality assessment visits and an analysis of feedback from student questionnaires. These substantial responsibilities are carried out by the Deputy Principal and AP(QA), supported by an administrative assistant and a secretarial assistant.

31 The audit team met members of the Academic Board, the QSC, ADWGs and others of the College's committees with quality assurance responsibilities and explored their understanding of the terms of reference of each committee as set out in the Handbook and how they were interpreted. Many of those with whom the team discussed these matters appeared unsure of the roles of College committees, and the relationships and linkages between them. The College may wish to consider the advisability of satisfying itself that committee terms of reference continue to be appropriate, clarifying them where necessary, and disseminating them to members of staff.

Content and presentation of reports

32 The College's present quality assurance arrangements are dependent on the accuracy and reliability of the written reports and minutes which its various committees consider (often in turn), and which are finally received by the Academic Board. The audit team found wide variations in the form and content of the reports and minutes it read, notwithstanding the advice offered by the College in the Handbook, for example. Many reports and minutes the team read were descriptive, and appeared to offer limited analysis and evaluation, despite the College's apparent intention that such reports and minutes should also identify action plans. More particularly, it appeared to the team that communications upwards to the Academic Board from its committees and sub-committees tended to be insufficiently clear or detailed to allow the Board to discharge its substantial responsibilities effectively. The team also noted a tendency for weaknesses identified by its own or external quality assurance processes to be played down (see also below, paragraph 90). The College may wish to consider the necessity of reviewing and clarifying the guidance it offers its staff on its requirements for the production of reports and minutes. In the course of such a review, it may also wish to consider how best it might encourage staff to give greater prominence in their reports to evaluating and analysing the data they have collected, to offering recommendations, and to identifying issues of College-wide significance for discussion.

Communications

33 The College has an intricate internal committee structure operating at five levels: departmental, course, and co-ordinating committees; with above them the QSC, which in turn reports to the Academic Board. It was not clear to the audit team that the College was formally able to assure itself that the information needed to monitor the operation of its quality control arrangements (and, if necessary, to intervene in a timely fashion) was passed upwards through this deliberative chain. Nor was it clear how responses to items of information, decisions and recommendations, passed up through the same chain, were formally passed down again, closing the 'feedback loop', other than through the informal channels of the College's 'extended executive'.

Role of the University of Wales in quality assurance

34 Almost all courses and programmes of study offered by the College lead to awards of the University of Wales and are subject to its quality assurance arrangements, in addition to those of the College. As an associated institution, the College enjoys a large measure of delegated authority for the validation of new programmes and courses up to and including the final validation event. The College has the University's approval to prepare students for research degrees in Theology and Bilingual Studies.

35 Reports of College validation events are sent to the University Registry, which refers them for scrutiny to two assessors identified by the chair of the relevant University Subject Panel (USP). Where issues raised by assessors cannot be satisfactorily resolved, their recommendation against final approval by the University may lead to the rejection of a validated scheme by its Academic Board, as had happened shortly before the audit to a College proposal for an undergraduate honours degree programme in Welsh. At the time of the audit, the audit team was unable to clarify the grounds for this rejection, and consequently was unable to pursue the matter further in order to establish what light, if any, it might shed on the College's quality assurance arrangements.

Quality assurance arrangements: some general observations

36 At the time of the audit the College was translating largely informal processes for internal communication, and assuring the quality of its academic provision, into more formal arrangements. Reviewing these newer arrangements, it appeared to the audit team that, commendably, the College had sought to sustain and enhance informal processes which it had previously found effective, rather than simply replace them. Indeed, having established which of its informal arrangements are currently working well, the College might well consider itself justified in retaining and developing those which it found demonstrably effective. At the same time, however, the continuing dependence of the College's quality assurance arrangements on the energy and knowledge of a small number of key individuals did not appear to the team to be conducive to the development of robust formal or informal quality assurance, or to wider staff participation in their operation.

37 The College told the audit team that it planned to review its quality assurance arrangements in the near future; the team would encourage such a development, the benefit of which to the College would be enhanced by the inclusion amongst the reviewers of several external peers. In the course of such a review, the College might wish to examine its committee structure, its reporting conventions, and how and to whom reports and minutes are distributed, and the advantages of carrying out further such reviews periodically. The College might wish to begin such a review by seeking and analysing the views of students and members of its staff at all levels on its present arrangements.

DESIGN, APPROVAL AND REVIEW OF PROGRAMMES OF STUDY

Academic planning

38 It is the formal responsibility of the College's Academic Board to ensure that its academic provision 'reflects [the College's] sense of place and identity, and its strengths and distinctiveness in addition to its responsiveness to national and local needs'. Academic development within the College has primarily been focused upon full-time programmes of study, including the conversion of a combined studies undergraduate programme into an extended range of single and joint honours awards; the translation of a four year BEd into a three year BA (with the encouragement of HEFCW); and the development of programmes of study leading to taught postgraduate awards.

39 The planning framework within which these developments take place appeared to the audit team to be essentially reactive: proposals for new developments, endorsed by the Academic Board, are incorporated within the College's Strategic Plan. The team could find no evidence in the papers it scrutinised that proposals had been sought by the SMT from members of the College in the light of the former's perceptions of the College's strategic needs. It appeared to the team that one consequence of the College's planning arrangements was that those responsible for managing the College's programme of validations appeared at times to have experienced difficulty in keeping pace with the volume of potential submissions. The team noted that the greater part of the 'bottom-up' developmental activity, the supporting papers for which it read, had been directed towards full-time programmes of study, and that the Principal had called on members of staff to devote more attention to the development of part-time provision.

Unitisation, modularisation and academic credit transfer

40 At the time of the audit the College had carried out the unitisation of a majority of its academic provision, but opinion was divided amongst those who met the audit team on the extent to which programmes should be modularised. The College has participated in the work of the Higher Education Credit Initiative Wales leading to the Welsh Higher Education Credit Framework, and in November 1995 the Academic Board had established a working group to assess its importance for the College.

The development, approval and validation of new programmes of study

41 The initial proposal for a new programme of study may originate with any member of the teaching staff, or more usually with a group of staff within one or several departments. Subsequently, an outline proposal will be considered by the SMT, which examines the likely resource requirements of the proposed programme, and its relationship to existing provision, the College's mission, and the expectations set out in the Strategic Plan. If the SMT approves the outline proposal for further development, the proposers may seek approval from the Academic Board to allow it to proceed to validation.

42 Members of the SMT told the audit team that they drew up estimates of the resources required for new developments at this preliminary stage, and that the Librarian was drawn into the consideration of programme developments and changes through attendance at course committees. A number of the team's enquiries revolved around recent validations of taught postgraduate programmes of study. Members of the SMT told the team that the development of such programmes had been judged by the College to be a strategic necessity, and that their approval had been confirmed on the basis of marginal rather than full-cost estimates. Reviewing the records of these validations, and subsequent course monitoring reports for the approved programmes, it appeared to the team that the SMT's arrangements for matching resources to developments might not be completely effective, as several had not been supported with the required library provision and accommodation (see below, paragraph 70). The team noted that the College intended to reassess the viability of such provision in the light of its current financial circumstances, with the assistance of the annual statistical analyses undertaken by the Deputy Principal for the SMT, a move which the team would encourage the College to take forward.

Development and preliminary validation

43 In the first stage of the College's two stage validation process an outline proposal is developed into a submission document for consideration by a preliminary validation panel. The proposing group developing the submission is expected to follow the detailed guidance in the Handbook, and it is normal practice for advice on curriculum and subject matters to be sought from personal and professional contacts, and networks outside the College. Submissions under development will also be discussed by the relevant heads of departments and existing course directors, and be examined at meetings of course and other committees, where the comments of students and external representatives may be obtained.

44 The membership of preliminary validation panels is determined by the QSC and the preliminary validation event is organised and supported administratively by the QAU. Members of the College who had participated in such events as panel members told the audit team that the focus at this stage of the validation process was on testing the adequacy of the curriculum and supporting statements put forward in the submission document. Panel members may suggest substantial modifications to the form and content of submissions, and require that they be carried out before agreeing to allow the proposal to proceed to final validation.

45 Members of preliminary validation panels are drawn from departments other than those making the submission, and involvement in the validation process as a panel member is considered to provide a valuable staff development opportunity. Members of the teaching and research staff at all levels told the audit team, however, that they were not aware of the criteria used by the QSC to identify individuals to act as members of validation panels. Teachers and researchers added that such opportunities when they arose were generally welcomed. The College may consider it helpful to set out the criteria used to identify potential members for validation panels in its Handbook.

46 As with preliminary validations, membership of the panel for a final College validation is identified by the QSC but in this case it will include three or four internal members, and three or four members from outside the College. The AP(QA) is usually a member of final validation panels together with a further member of the College's SMT, who will act as the chair of the panel.

47 The conclusions recorded in a number of reports read by the audit team from meetings of final validation panels took the form of undertakings given to the panel, rather than the panel's judgements and recommendations. Reporting the outcome of a validation panel's meetings in this way would not, it appeared to the team, permit those who subsequently read the report to understand fully the panel's judgements, and its reasons for reaching them, thereby denying members of staff and other panels the opportunity to use such reports to develop their own judgemental abilities. The College may wish to frame clear guidance on how to record and set Out the deliberations, judgements, and conclusions of preliminary and final validation panels for inclusion in its Handbook. It is the responsibility of the QSC and QAU to ensure that undertakings given at final validation events are fulfilled.

Approval by the University of Wales

48 Following formal approval by the College's Academic Board, each College-validated programme of studies or course is subject to further scrutiny by the appropriate University of Wales' Subject Panel (USP) (see above, paragraph 14). Two members of an USP, identified by its chair, act as assessors and report to the University of Wales' Registry.

Minor and major modifications to programmes of study

49 In keeping with its status as an associated college of the University, the College operates two procedures for approving modifications to programmes of study. For modifications involving less than 25 per cent of the content of a programme of study, reference to the University of Wales is not required. Under this procedure, relatively minor amendments to the syllabus or recommended reading of a course or courses are dealt with by the relevant course committee.

50 Major changes to programmes of study may require reference to the University of Wales. A major change might involve alterations to more than 25 per cent of a programme; the addition of new courses or units of study; the withdrawal of courses or units; the change of a named award; the substantial changes to content; alterations to the assessment regime, or structural amendments. Such changes are considered by the QSC, and in some cases the audit team noted that the QSC might require the proposed changes to be supported by detailed documentation, and might establish an ad hoc sub-committee to consider the proposal on its behalf. The level of scrutiny afforded to such proposals varied within the sample seen by the team, which was not able to establish how the QSC determined the procedure it would follow in particular cases. The College may wish to set out in its Handbook the criteria it uses to determine how a proposal to amend a programme of study or course will be handled.

Programme monitoring and review

51 All programmes of study are subject to an annual monitoring process, the main feature of which is an annual report from the course committee to the QSC (the annual course review, or ACR) and thence to the Academic Board. In addition to annual monitoring reports from course committees, the QSC receives a separate annual report from the QAU identifying matters raised in annual monitoring reports with College-wide significance (see above, paragraph 29). The QSC reports to the Academic Board and is responsible for monitoring the implementation of action plans agreed by course committees, a duty which it discharges through the QAU.

52 The information brought together by the course director in the ACR includes reports submitted to the course committee from each subject, (or strand), or discipline area contributing to the programme, compiled by the relevant head of department; external examiners' reports; responses from heads of departments to matters relating to the programme identified in student questionnaires, and the course director's report. The format adopted in the College Handbook for the headings and prompts provided to assist heads of department to complete their sections of the ACR, and course directors to finalise the ACR, have been developed by the College from those used by the Quality Assessment Division of HEFCW. The College expects ACRs to lead to an action plan for implementation in the course of the session (see above, paragraph 32). The completed ACR is presented by the course director to the relevant course committee for discussion and endorsement, sent to the QAU for analysis, and submitted to the QSC for its scrutiny.

53 The College expects an ACR to provide an opportunity to monitor and evaluate the relevance of the chosen assessment strategy of the programme of study against the learning objectives, content, teaching methods and student profile, in the light of external examiners' comment, feedback from students in questionnaires and course committees, and statistical information about student intake, cohort progression, and examination results. There was a wide variation in the format of the Reports seen by the audit team and in their apparent thoroughness. On the basis of this sample, it did not appear to the team that the College's expectations for the completion of ACRs were being consistently met.

54 The audit team was told that in finalising ACRs, staff were mindful of the wide circulation they received, and were accordingly careful in their comments. This suggested to the team that observations made by those teaching courses and subjects (or strands), which might be considered 'primary' evidence, could be 'filtered', as contributions were edited for inclusion in the overall report. The wide variation in the form and content of the sample of ACRS seen by the team is the subject of comment elsewhere (see above, paragraph 32). Members of the course committees and QSC who met the team were satisfied that the annual monitoring process was conducted with rigour at all levels, a view which the team found it was unable to share in view of the variations in the quality of the Reports which it read, and the possibilities for 'filtering' primary evidence, noted above. In the light of these comments and those in earlier paragraphs, the College may consider it advisable to assure itself that its annual course monitoring arrangements are as effective as it would expect, and that ACRs are consistently and thoroughly completed (see also below, paragraph 75).

QAU annual reports on programmes

55 The QAU provides statistical information for course committees on the development of the programme throughout the previous session. Statistical returns cover matters such as student intake, cohort progression and student attainment, and include an analysis of completed standard College questionnaires. Members of staff and students told the audit team that the collection and analysis of feedback information, particularly through questionnaires, had given rise to considerable comment and expressions of concern throughout the College, a matter which is noted elsewhere in this report (see below, paragraph 85).

TEACHING, LEARNING AND MONITORING THE STUDENT EXPERIENCE

Selection, admission and induction of students

56 The selection and admission of students to undergraduate programmes of study is administered by the Registry, and the College offers interviews to a high proportion of applicants. Students who discussed their experiences of interviews and admission to the College with the audit team had found the processes commendably friendly, informative and well-organised.

57 The College offers all undergraduate students a formal induction programme. Students considered that it provided a comprehensive and friendly introduction to the College and their intended programme of study, and an early opportunity to meet with the staff who would be teaching them. Students from the EU and beyond greatly appreciated the arrangements the College had made to collect them from an assortment of arrival points in the UK and the opportunities that they had been given to get to know the country in which they would be studying.

Tutorial support

58 The College assigns each undergraduate and taught postgraduate student to one of a number of members of the teaching staff, who is expected to provide tutorial support through meeting regularly with her or his tutee, and to monitor their academic progress informally. Staff who undertake this role spoke warmly of the training they had received from an external consultant in counselling skills. Students stated that they had ready access to advice and support from the teaching staff, many of whom provided out-of-hours contact arrangements. The audit team commends the College for the responsiveness of its teaching and support staff to the needs of its students and the caring attitude they evinced (see also below, paragraph 66). The team was, however, unable to discover any monitoring arrangements whereby, should this cease to be the case, the College could assure itself that tutorial support was being provided in accordance with its expectations. The College may consider it desirable to establish such monitoring arrangements.

Teaching Practice Advisory Committee

59 To support the development of its substantial academic provision in initial teacher training (ITT) the College has established a Teaching Practice Advisory Committee, membership of which includes representatives of head teachers and advisers from local education authorities. The College has also developed links with employers in several fields to support students on other placements The audit team discussed the management of these arrangements with members of staff and students and concluded that they were operating successfully.

Course handbooks

60 On joining the College each student is provided with a course handbook. The College expects course handbooks to meet a set of requirements which are themselves derived from the HEQC Guidelines on Quality Assurance. The audit team was provided with a number of such handbooks in the College's briefing materials and noted significant variations in interpreting the College's requirements. The College may consider it desirable to survey the form and content of current course handbooks in order to identify good practice in their content and disseminate such good practice throughout the College.

Taught postgraduate and research studies

61 The Research Committee is formally responsible for overseeing postgraduate research studies and more recently the College has established an Academic Development Working Group for Higher Degrees. At the time of the audit the relations between these two bodies had yet to be determined.

62 Students following taught postgraduate programmes of study confirmed that they received timely feedback on their assignments, and that there were opportunities for collaboration with their tutors which could lead to joint publications.

Postgraduate supervision arrangements

63 Staff who are required to supervise research students have been given the opportunity by the College to attend workshops, run by an external consultant, to help them to understand the demands of the role and to develop the skills necessary to perform it. Postgraduate students who met the audit team spoke warmly of the provision made for them in the departments and research centres to which they were attached, and with which they identify strongly. Postgraduate students spoke positively of the responsiveness of tutors and supervisors to their needs.

64 The audit team was interested to hear that the College had made arrangements to collect fees from postgraduates in a manner which spread the financial burden throughout their studies. The College recognises that further development will depend in large measure on the outcome of the Research Assessment Exercise, the submissions for which had just been completed at the time of the visit. The team commends the College for the level of support it provides for its postgraduate students.

Student Support Services

65 The AP(SS) is responsible for the management of the College's counselling, chaplaincy, welfare, careers, accommodation and medical services. Until recently, the College shared the cost of operating a careers service with University of Wales, Lampeter, but since April 1995 it has employed its own full time Careers Adviser.

66 The College's Welfare Officer is on call to students throughout the session, providing an advisory service which is freely used and highly valued. Students considered the various services effective and helpful. The audit team commends the College for the high reputation its support services have established amongst its students, and for the work of all its teaching and support staff in sustaining the caring culture throughout the College to which its students testified. Notwithstanding the effectiveness of the College's student support services, however, it appeared to the team that the individual services were operating to some extent independently of one another and that their activities might benefit from even closer co­ordination.

Support for teaching and learning: the Library and the Information Technology Centre

67 The College's Library, two Research Centres, and an Information Technology Centre and two Teaching Resources Centres constitute its Information and Resources Service (IRS). The College publishes a booklet on the IRS for users, which describes the range of services and its availability; the management and co-ordination of the IRS is the responsibility of the AP(AA).

Information Resources Service Committee

68 The operation of the IRS is overseen by the Information Resources Service Committee (IRSC) which reports to the Academic Board and is chaired by the AP(AA). The remit of the IRSC charges it to monitor and co-ordinate the various academic support services and their strategic development; to ensure their effective operation; and to deal with budget and human resource allocations. Whilst many meetings of the IRSC appeared to the audit team to focus on operational details, the draft 'Library Services Strategy 1995-1997', which had been prepared by the Librarian for submission to the Academic Board for discussion and approval, appeared to address an appropriately wide range of concerns. The 'Strategy' follows the recommendations of the Follett Report and gives prominence to user evaluation, performance indicators and the introduction of service level agreements specifying performance targets. This latter point is considered necessary for the proper implementation of the College's Students' Charter. The College may wish to encourage all academic and student support services to follow the lead of the Library by developing, adopting, and publishing standards for the delivery of their services, and may subsequently wish to monitor the performance of the services against such standards.

The Library

69 At the time of the audit, reading space within the Library was reported to be under pressure pending the completion of the second phase of a library development plan. The audit team noted in the College's briefing materials students' comments on the shortage of reading space in the Library. It noted with interest, therefore, the steps taken by the College to meet this need, including extending the opening time of the Library to midnight during week days. The College is to be commended on the service provided by its Library under severe pressure and of the practice employed of monitoring user perceptions of the service. Notwithstanding these efforts, however, and students' ready acknowledgement of the speed with which the Librarian moved to address matters capable of resolution, the team was told that reading and study space throughout the College was difficult to find.

70 The distribution of learning resources and particularly textual materials between the Library, the Teaching Resources Centre and the two Research Centres has given rise to critical comment from staff and students. The audit team noted that gaps in the library provision of texts, journals and study spaces had been the subject of comment in a number of the annual reviews that it saw.

Information technology (IT) provision

71 The contribution of information technology to the College's teaching and learning arrangements appeared to the audit team to have received sustained attention in recent years with the opening of a new Information Technology Centre and the installation of a local area network throughout the campus, which is linked to the UK-wide Joint Academic Network (JANET). The team noted that following this rapid expansion of services the provision of user support and training for staff and students had clearly placed great strains upon the staff supporting the College's IT provision for teaching, learning and administration. To meet a high level of demand for access to IT facilities the College has made the IT Centre available around the clock, other than when required for teaching. The team commends the College for the priority given to enhancing its information technology provision, and for the range of training and support services for users provided by the Centre (see also below, paragraph 92).

STUDENT ASSESSMENT AND THE CLASSIFICATION OF AWARDS

Regulatory frameworks

72 For the awards of the University of Wales the College adheres to the University's regulations. These apply to examinations and assessments in the later stages of a programme and also cover academic appeals and unfair practice. For awards involving initial teacher training, the College adheres to regulations set out in Welsh Office circulars. There are no approved College-wide regulations to provide an institutional assessment framework for courses or programmes of study not covered by University of Wales regulations (see below, paragraph 76).

Assessment of students

73 The main sources of information for students on the assessment strategies, processes and criteria for their programmes of study are to be found in the course handbooks, which each programme provides. The course handbooks seen by the audit team generally offered clear guidance on assessment practices and regulations. Staff and students who met the team were aware of the main sources of information on assessment and were familiar, for the most part, with the arrangements that applied to the programmes of study they were associated with. Where changes were made to assessment criteria mid-course, however, the team found that difficulties in formalising and communicating such changes could arise.

74 Students who discussed the submission and return of their assignments with the audit team reported some inappropriate variation in the ways departments treated work submitted after a published deadline, and expressed a wish to see greater consistency applied across all departments, in accordance with an Academic Board resolution of 1990. In some cases course-work assignments had been returned in good time accompanied by helpful feedback presented in a standard form. Other students, however, spoke of receiving inadequate and sometimes contradictory briefing on their assessments, and had experienced erratic turnaround times for marked assignments. As it reviews its assessment arrangements the College may wish to consider the advisability of clarifying where responsibility lies within its structures for monitoring and reporting on assessment practice.

Amendments to assessment procedures

75 The College's validation processes allow for variations in assessment practice between programmes and from time to time changes are made to assessment regimes in the light of experience. These may be introduced on the initiative of a course director on behalf of a course team, or by a course committee and for a variety of reasons, including, the audit team noted, changing the assessment strategy in response to poor examination results. As part of the annual review process, programme teams are expected to comment on the compatibility of the programme's aims, teaching and learning methods and learning objectives, and its assessment strategy (see above, paragraphs 53 and 73).

76 Changes to the substance of an assessment scheme, such as those described above, require the approval of the Academic Board and may generate extended debate. The practice at Academic Board appeared to the audit team, however, to focus on the particular instance rather than to relate that instance to the College's provision as a whole. No doubt the College will wish to continue to allow departments and programme teams some variation in their assessment arrangements to suit particular circumstances. The College will wish to consider the necessity of bringing together and codifying all its assessment regulations in a single document for consideration and approval by the Academic Board, and consider the necessity of identifying those principles and practices which it expects all its programme teams to apply to assessments. Having codified its present assessment regulations and arrangements the College may consider it desirable to identify instances of good practice and disseminate information about these to its staff.

Appointment of external examiners

77 The selection, appointment, and briefing of external examiners is carried out in accordance with University of Wales' regulations; guidance is provided in the Handbook. Since 1993, the powers delegated by the University of Wales to the College for the nomination of external examiners have been exercised by the QSC on behalf of the Academic Board. The nomination of a new or replacement external examiner is sent by the appropriate head of department or course director to the Superintendent of Examinations, the AP(SS), using a standard form, together with the curriculum vitae of the nominee. Following scrutiny by the AP(SS), the nomination is placed before the QSC. After further close scrutiny, suitable nominations are sent to the University of Wales for formal approval and appointment. Although in many respects the College's arrangements for the selection and appointment of external examiners appeared to the audit team to be satisfactory, and in accordance with University of Wales' regulations, the team noted recent expressions of concern over 'ambiguity and lack of co-ordination' in the appointment of external examiners, suggesting that arrangements do not always operate as intended, a matter which doubtless the College already has under review.

Briefing and induction of external examiners

78 The audit team was unable to find formal guidance on the briefing of external examiners in the University of Wales or College documents it saw. Newly-appointed external examiners are normally invited to visit the College to meet appropriate staff and receive an informal briefing on the College and the programme of studies with which they are associated. The University of Wales' standard external examiners' report form suggests that newly-appointed external examiners will be provided with copies of the retiring examiners' reports. On the evidence of the external examiners' reports provided for the team, this appeared not to be happening. Discussions with staff revealed uncertainty over the locus of responsibility for ensuring that new examiners automatically received this information, although it was assumed that the onus rested with the University Registry rather than the College. The College might consider it advisable to consider how this gap in the briefing of new external examiners might be closed, and whether a general briefing document might be devised for them.

Submission of external examiners' reports

79 External examiners submit their reports to the Senior Vice-Chancellor of the University of Wales in a prescribed standard form, which may be accompanied by a supplementary text. This is sent by the University Registry to the Principal of the College, who arranges for its internal distribution and consideration. The terms of reference of the QSC and co-ordinating and departmental committees indicate that each is to receive, consider, and respond to external examiners' reports. As noted elsewhere, course committees consider external examiners' reports as part of the review of annual reports (see above, paragraph 53). In the absence of explicit guidance, however, practice varies in the level of consideration and significance attached to external examiners' reports, and it appeared to the audit team that opportunities were frequently missed to address generic issues arising from individual reports. The College may wish to consider the advisability of reviewing how the QSC discharges its responsibilities for the receipt and evaluation of external examiners' reports, including the monitoring of their consideration by other committees.

Academic Standards

80 The University of Wales is responsible for the academic standards of the awards to which most of the College's programmes of study lead and the College has yet to formulate a policy on academic standards. The College acknowledges the need to develop such a policy and is being kept informed of UK-wide developments through the regular attendance of senior members of staff at seminars and conferences. Information from these and published sources has been disseminated within the College through staff development workshops. It is hoped within the College that the newly formed ADWGs will give prominence to academic standards in their deliberations (see above, paragraph 23).

81 At the time of the audit, the College considered that external examiners constituted the primary means whereby it monitored and assured the standards of its academic provision, through ensuring 'that students are assessed equitably, that the standard of the award is maintained, and is comparable to that of similar awards throughout the HE sector'. Their role in fulfilling these responsibilities is set out in the Handbook, where prominence is given to their role in supporting academic standards. Further steps taken by the College to assure and enhance academic standards include the introduction of double-marking into all departments and courses for use as 'a moderation and standardising mechanism', and the annual production by the Deputy Principal of a comparative institutional analysis of student attainment in final awards. The College may wish to consider the advisability of assigning a more prominent role in the development of an institutional policy for academic standards to its Academic Board, building on the steps the College has already taken to raise awareness amongst staff of the matters that remain to be addressed.

FEEDBACK AND ENHANCEMENT

The collection and evaluation of student feedback information on teaching and learning

82 The College seeks feedback from its students in two principal ways: through their membership on committees at all levels, and through questionnaires, circulated by course committees and departments to gather information about their courses and programmes of study; and centrally, usually through the QAU, to gather information on matters of College-wide relevance. For the collection of non-specific feedback, the College has formerly largely relied on informal methods, although the President of the Students' Union is an ex-officio member of the QSC, the Academic Board and the Governing Body, and the attendance of student representatives at course committees is well-established.

83 The College recognises a need to assist student representatives to prepare for this role and has produced briefing materials and organised workshops for them. A booklet produced by the QAU and the Students' Union, Students and Quality, emphasises the importance of the College's quality assurance systems to students, and describes the opportunities provided for their involvement in quality matters, including representation and the use of questionnaires. The booklet concludes with useful check lists on the work of the College's various committees, and the best way for student representatives to prepare for meetings.

84 A training workshop for student representatives had taken place in January 1996. It had consisted of an hour's briefing session with presentations made by the Librarian and the AP(QA) but attendance had apparently been disappointing, with only 12 out of a possible 33 student representatives present. Even so, the audit team was told that students have ready access to the AP(QA) for advice on the discharge of their representational role. The College is to be commended on the increased attention being given to student representation but may consider it desirable to give further attention to improving its current arrangements for training and supporting student representatives.

Compilation, completion and analysis of questionnaires

85 The audit team found expressions of staff and student concerns relating to the design and use of questionnaires, and low rates of student responses to questionnaires at several points in the College's briefing materials (see above, paragraph 55). Having reviewed the design and content of a number of questionnaires provided in the College's briefing materials, the team shared these concerns. In a number of instances the purposes to be served by questionnaires for both College-wide and departmental circulation, appeared unclear, and the design, content, and analysis of questionnaires did not appear to meet generally acknowledged technical standards, a matter which the College may wish to address. The team also noted that the usefulness of summaries and analyses of responses had been reduced in some cases by the unhelpful way in which information was presented.

Responses to student feedback

86 Heads of department are expected to respond to student feedback and to use standard forms of response. This appeared to the audit team to be likely to prompt them to offer defensive and adversarial responses to specific matters, rather than to encourage the development of a useful partnership between departments and students. As most responses from heads of departments the team saw were hand-written and photocopied, the team could understand the difficulty students appeared to have experienced in establishing their content. The College may wish to review the standard form it provides for heads of department to respond to student feedback, and ensure that the formal responses of heads of department to student feedback information are more clearly presented in future.

87 In questionnaire returns, and in course committee reports, students remarked on the College's slowness in responding to their comments. In one particular instance, relating to catering arrangements on the campus (a subject which is itself outwith the remit of the audit team), the College had employed a firm of consultants to report on the matter, but had not kept students informed of the steps it had taken. In view of the importance attached by the College in its briefing materials to 'customer care' it might wish to consider how it might ensure that it informs students that it has received their comments and what it has done to respond to them.

External involvement in quality assurance processes

88 The concern of the College to secure external perspectives on its work was evident to the audit team in the care taken to invite the participation of external peers in course and programme development and validation, and in monitoring and evaluation. Employers, practitioners and other external peers are represented on course committees and one such external member who met the team testified from his experience to the thoroughness with which business at the annual review meeting was conducted. The College's response to the comments of external examiners is considered elsewhere in this report (see above, paragraph 79).

89 The College encourages its staff to draw on their external contacts and networks to learn of developments in their field or discipline. Members of the teaching staff who had served as external examiners and teaching quality assessors told the audit team they had valued the experience, and were able to provide examples of good practice which they had observed, brought back to the College, and implemented. In the main, however, such activities appeared to take place at the departmental level, and opportunities to exploit these additions to the collective experience at corporate level appeared not to have been taken up. The College might wish to consider the desirability of collecting details of the experience gained by its staff in their various external contacts and how such information might be disseminated and put to use.

Response to teaching quality assessments

90 Quality assessment reports from the Higher Education Funding Council for Wales (HEFCW) and on ITT provision from the Office of Her Majesty's Chief Inspector of Education for Wales (OHMCI) represent important sources of external commentary on the College's academic provision. The College told the audit team that its consideration of the content of these important documents had been hampered by delays of up to 18 months in receiving them; nonetheless, in examining the College's responses to the reports from external agencies that it had received, the team noted some tendency to miss the significance of points requiring remedial action. It also appeared to the team that such reports had not been used to identify instances of good practice for internal encouragement and dissemination, to identify generic issues of importance to the effective operation of quality assurance and enhancement processes, or to improve the College's ability to produce accurate and helpful self-assessments. The College may consider it advisable to review its arrangements for receiving, analysing, and acting upon the reports it receives from external quality assurance agencies.

Quality and enhancement in teaching and learning

91 The College intends that its quality assurance processes should generate sufficient information from a variety of sources to assist it to assure the quality of teaching and learning, and achieve the effective evaluation of the learning opportunities provided for students. It appeared to the audit team, however, that opportunities to disseminate good practice, identified in external examiners' reports and ACRs at departmental level, and to encourage innovation in teaching and learning, had not been seized.

92 The audit team found little evidence in ACRs or in discussions with staff that new adult learning strategies were being tried and formally evaluated. For example, notwithstanding some examples in ACRs of the application of information technology to teaching and learning, the team could find no evidence of an active College-wide policy for its exploitation (see also above, paragraph 71). Where new procedures had been introduced, such as the widespread adoption of double-marking, arrangements to evaluate the outcome of such a development did not appear to have been made. The College may wish to consider giving greater prominence in its academic policies to innovation in teaching and learning and to using its quality assurance processes more actively to promote quality enhancement.

STAFFING, STAFF DEVELOPMENT, PROMOTION AND REWARD

93 There have been significant changes in the number and characteristics of the College's teaching staff since 1982.

	Age % under 40	Gender % Women	Origins % born outside Wales	Total staff
1981-82	24.5	13.2	5.6	53
1995-96	62.8	36.2	28.5	105

The change in the gender balance set out above has begun to bring staffing into closer alignment with the gender balance of students, where the proportion of women to men is three to two. In 1982, members of the College's teaching staff had been principally recruited on the basis of the contribution which they were expected to make to teacher training and qualified teacher status had then been more important than a track record in research. In recent years priorities have changed and a number of staff have been appointed specifically for their success in research and publication. The College takes the view that in future a greater proportion of its academic work will be focused on non-teacher-training courses. At the time of the audit, this change in approach had not found universal support among staff.

Personnel policies and management

94 The College's Personnel Officer reports to the AP(QA), and the first postholder was appointed in 1995. Since her appointment, initial steps have been taken to consolidate and codify College policies and procedures and a general statement was issued in January 1996 setting out the basic principles which should inform a College Personnel Strategy and a Personnel Strategy Framework. These are currently documented in an Academic Staff Handbook, which covers the professional contract and related matters, and in publications containing statements of policies, standard operating procedures, and other documentation relating to the appointment, probation, and appraisal of members of the teaching staff, and equal opportunities.

Staffing Committee

95 The Staffing Committee reports to the Academic Board. Its membership includes the three Assistant Principals, the Director of Finance and Support Services, the Director of IT, the Personnel Officer and four elected members of academic staff; it is chaired by the Deputy Principal. The remit of the Committee has recently been extended from consideration of staff workload and deployment, staffing levels, new appointments, replacements and promotions to principal lecturer grade, to include staff development.

Appointment, induction, probation and confirmation

96 In general, full-time and part-time teaching posts are advertised externally in English and Welsh. The administration of all applications is dealt with by the Personnel Officer, and short-listing and the convening of appointment panels is undertaken in accordance with published standard procedures. For academic posts, it is usual to select up to seven candidates for interview.

97 The Principal normally chairs appointments panels and determines their membership, which will usually include the Deputy Principal; the AP(AA); the AP(QA); the appropriate head of the area which is recruiting; and the Personnel Officer. Candidates are normally offered an opportunity to visit the relevant department and meet staff informally. The chief recruitment instrument is the selection interview although the College is considering requiring candidates invited to interviews to make presentations to staff and students, among other pre-interview selection methods. Newly-appointed members of staff who met the audit team considered the briefing material they had been sent as applicants had been adequate, and the selection process fair and thorough.

Appointment and induction

98 Staff who take up their appointments at the beginning of the academic session may visit the College during the summer vacation for a briefing on their duties and responsibilities. Such informal induction arrangements are supplemented by the attendance of newly-appointed members of the teaching staff at the staff development events organised for all teaching staff before students arrive in September, details of which are set out in the College's Staff Development Programme. Responsibility for the induction of new staff is thereafter delegated to departments, some of which currently provide mentoring support, an arrangement that the College has decided should be adopted by all departments in future. Consideration is currently being given to the adoption of the scheme developed by the Staff and Educational Development Association, of which the College is a member, for the development of new staff who lack prior teaching experience.

Probation

99 Once newly-appointed members of the teaching staff have satisfactorily completed a 12 month probation period the AP(QA) confirms the appointment by letter on behalf of the College. The audit team found the documentation on probation it read to be unclear: for example, although probationers are formally notified in a letter of the four general criteria which they must satisfy, and on which their heads of department are required to report, no indication is given as to how these are to be converted into measurable performance criteria. Members of staff who were undergoing, or had recently completed their probationary period, were uncertain of the arrangements for confirming their appointments and unsure how to distinguish such arrangements from those for appraisal. The team was told that the College was aware of the deficiencies in existing arrangements for the confirmation of probationary staff, and of the intention of the Personnel Officer to improve practice, a development which the team would encourage.

Staff development

100 In the view of the College, staff development is 'central to the enhancement of teaching and learning'. Staff development within the College is viewed as encompassing attendance at internal and external workshops, courses and conferences, involvement in management and administration, membership of external and internal committees and the undertaking of consultancies.

101 A variety of workshops has been organised for staff by the Quality Assurance Unit including a series of three-day workshops, held each September since 1993 on quality issues, and workshops on the use of the computer network installed in the College. Opportunities have also been provided for tutors to develop academic counselling skills and for developing the potential uses of information technology for teaching and learning. Members of staff with whom the audit team discussed the application of information technology to the learning environment appeared to lack an appreciation of the opportunities for their development presented by the College's new Information Technology Centre.

102 The audit team was told of instances of annual reports highlighting needs for staff development to which the College, through the QAU, had responded. The team formed the view that the College's provision of staff development was relevant and responsive to individual and Collegiate needs. The team noted the College's responsiveness to appraisal and annual monitoring and that it was generally supportive to staff development. The College is to be commended for the prominence and attention it has given to staff development, and the variety of needs-related development opportunities it has provided for members of the teaching staff.

103 At the time of the audit the Staff Development Budget was administered centrally. Almost half this Budget in 1995-96 was divided amongst departments on a per-capita basis, with the remainder being shared between support staff, research centres, and College-wide activities such as those of the QAU. Members of the teaching staff who discussed current staff development arrangements with the audit team were generally satisfied that they were flexible and met the needs of individuals.

104 In 1996-97, the College intends to delegate staff development budgets to departments, a move which it sees as demonstrating its appreciation of the continuing need for staff development. The Staffing Committee will monitor these new arrangements, but members of staff who discussed this development with the audit team were not sure that it would be an improvement on current arrangements. The team noted that the Academic Board intended to review the effectiveness of staff development activities and funding at the end of the session, drawing chiefly on the formal evaluations to be provided by staff of the events which they had attended. The College might wish to consider utilising additional sources of evaluation data, such as follow-up surveys, designed to measure the impact of staff development on practice and performance.

Appraisal

105 The College's appraisal scheme for members of the teaching staff has been in operation for three years and aims to 'increase the involvement, commitment and responsibility of staff for their own development and to raise the quality of performance'. An annual appraisal interview is held each year with the appropriate line manager, at which performance is jointly reviewed, future targets are established, and training and career development needs are assessed. Details of the scheme and associated documentation are provided in the Handbook. Guidance notes are provided and standard forms are completed, including a summary statement of the main areas of agreement and action points arising from the appraisal interview, which must be signed by the appraiser and appraisee. The content of the various summaries and reports remains confidential to the parties involved. Any further access is limited to the relevant Assistant Principal who is required to maintain an overview of the process for his or her area of responsibility, and the Deputy Principal, who is required to monitor the operation of the scheme to ensure that 'appraisals are being carried out, staff development issues addressed and equal opportunities monitored'. For dissatisfied staff there is access first to a 'grandparent' (usually the appraiser's appraiser), and then to the Deputy Principal, before they may choose to invoke the formal grievance procedure.

106 The appraisal scheme provides for routine monitoring by those participating and, with the recent appointment of the Personnel Officer, the College has decided to review its operation with a view to increasing its effectiveness and extending its scope to administrative and other support staff. The College considers that the scheme has successfully met its objectives. The audit team was able to confirm from discussions with staff their positive attitude to the scheme and its strong developmental orientation. The College is to be commended on the successful introduction of an appraisal scheme which has won the support of academic staff and for the opportunity being taken to review, upgrade and extend the scheme.

Promotion

107 Opportunities to apply for promotion are treated by the College in the same ways as other appointments and follow the same procedures of external advertisement, short-listing and selection.

Equal opportunities

108 The College's equal opportunities policy is 'to ensure that all actual or potential employees and students are not unfairly discriminated against on the basis of gender, colour, ethnic origin, age, marital status, lesbian and gay sexuality, disability, social background, religion, trade union activity or employee category'. It is set out in a comprehensive document which has been widely circulated throughout the College, but which itself contains occasional examples of the sexist use of language. The audit team was told by the College that the implementation of its equal opportunities policy was monitored co-operatively by the College, staff trades unions, and students in an Equal Opportunities Forum. The College intends to review the operation of the policy regularly and the team would encourage it to find ways of formalising the review of its equal opportunities policy.

CONTENT OF PROMOTIONAL MATERIAL RELATING TO ACADEMIC PROVISION

109 The College produces a wide variety of bilingual promotional materials including a Prospectus, course leaflets, an Annual Report and advertising copy for newspapers, the media, Prestel, and the World Wide Web. The External Relations Officer is responsible for co-ordinating promotional activity and for ensuring the suitability of promotional materials and the accuracy of their contents. The Prospectus is compiled and edited by him; he checks the accuracy of its contents with heads of department and may seek the guidance of the Principal on matters of policy. The audit team noted that the Prospectus and other materials that it saw did not refer to the College's policies on equal opportunities and special needs. Students who discussed the College's promotional materials with the team considered that they were helpful. Students who met the team were not aware of any arrangements to consult them on the accuracy or otherwise of the content of promotional materials, but welcomed recent improvements in the presentation of the College's promotional materials, and considered that whilst there were occasional inaccuracies, such materials gave a fair impression of the College and its context.

VALIDATION, FRANCHISING AND OTHER FORMS OF COLLABORATIVE PROVISION

College policy

110 The College values its collaborative links with educational institutions at home and abroad and with its wider community, but has chosen not to give this area of activity strategic prominence in its Mission. It does not franchise courses but is involved in a number of joint ventures and collaborative agreements. These are described in the Strategic Plan and the Handbook.

Collaboration with institutions within the UK

111 The College has links with four constituent Colleges of the University of Wales, the closest being with Lampeter in the areas of bilingual studies and theology. Lampeter and Trinity College share an appointment to a Chair of Pastoral Theology, which is currently held by the Mansel Jones Research Fellow at Trinity College. Staff from Lampeter are involved in jointly teaching a postgraduate programme of studies leading to the University of Wales 'MA Bilingual Studies: A European Dimension' which offers students opportunities to undertake part of their studies in one of a number of centres of study in specified higher education institutions throughout Europe. In view of the links developed between Lampeter and Trinity over the years, a 'Joint Statement on Collaboration' was agreed and issued by the two institutions in 1994, which confirmed existing forms of collaboration, and looked forward to future collaboration across a number of areas including teaching, research and library provision. The College also values a recent collaboration with De Montfort University in the delivery of the University's two year Diploma in Environmental Protection by distance learning and a residential period, an area of specialist study which the College has also established in its own undergraduate programme. These linkages are seen by the College to make an important contribution to extending its external contacts.

112 The College has also established links with Pembrokeshire College of Further Education and Carmarthenshire College of Technology and Art to provide access courses. It recruits a total of between 20-30 students each year from the two centres, mainly to humanities courses and programmes of study.

International collaboration

113 The College operates a student exchange programme for undergraduates with the Central University of Iowa in the USA and for postgraduate students following the MA Bilingual Studies: European Dimension. Other collaborations include the short observation visits of bilingual practice made by undergraduate students to a number of institutions in Scotland, Ireland and Europe, and the BEd degree in Pre-Primary Education, taught in conjunction with the Frederick Institute of Technology, Nicosia, Cyprus and the Athenian Centre of Pedagogical Studies and Research in Athens. Students from these two latter centres follow a two-year University of Wales' diploma in their own country, which is concluded by a three year programme of study for an honours degree at the College. To assist its own staff and members of staff in its partner institutions, the College has drawn up a 'Code of Practice on Collaboration' which provides advice 'on what is generally regarded as good practice by the awarding institution.'

114 The audit team met undergraduate students from Trinity and Iowa who were enthusiastic about the exchange arrangements between the two institutions and the valuable experiences it provided for them. Notwithstanding the close working relations between Trinity College and Iowa, including periods in residence at Trinity of a professor from Iowa, reports from College committees suggested to the team that the choices of pathways and modules open to Trinity College students studying in Iowa had given rise to concerns, which also extended to assessment practice. The College has encountered similar difficulties in the achievement of comparability and consistency of assessment between different units and different international centres associated with the 'MA in Bilingual Studies: European Dimension'. In this programme, an annual report from the College's Course Director had highlighted problems experienced by a student at a European centre which had arisen from differences in the teaching and learning practice of the College and the centre, notably the degree of learner autonomy, and the level of tutorial support in each. These difficulties had arisen notwithstanding the guidance offered by the College's 'Code of Practice on Collaboration'.

115 More recently, the College has established links with colleges in southern Africa. One link with Botswana is supported by the British Council and involves exchanges between students and staff of both institutions and South Africa. The other link arises from a collaborative venture with Whitelands College, Roehampton Institute, and the College of St Mark and St John, led by Trinity, under the aegis of the Church Education Foundation. It involves working with a partner college in South Africa to promote the emerging educational system there. The College is to be commended on the supportive links established with developing countries. The audit team noted with interest the international links established by the Centre for Theology and Education with practitioners and researchers at centres across several continents, which have led to collaborative research and co-authored publications.

116 As it continues to encourage and develop its programme of collaborative activities, the College might wish to consider giving greater prominence in its annual monitoring procedures to generic issues relating to collaborative provision, possibly by addressing these issues in separately identified sections of annual course reports, course committee minutes, minutes of the QSC and the Annual Report of the AAU. The College might also wish to gather and reflect on the good practice of its collaborative activities and generalise that experience to all courses and programmes of study it offers with international partners.

THE CHARTER FOR HIGHER EDUCATION

117 The College has produced a Students' Charter which is distributed to all students and staff, with a copy placed in the Library for reference. The document is strongly College-centred and appears to have the dissemination of information as its primary objective. It relies heavily on description, assertion and aspiration and combines a summary briefing with exhortations to good behaviour.

118 Some sections of the College's Students' Charter might benefit from further development; for example, there appears to be no formal complaints procedure and the topic of 'Complaints' is dealt with in an abbreviated form. Not all the students who met the audit team were aware of the Students' Charter, its significance or its relation to the Charter for Higher Education. The College might wish to review the form and content of its Students' Charter and clarify its relationship to the Charter for Higher Education issued by the Secretary of State for Wales.

CONCLUSIONS AND POINTS FOR FURTHER CONSIDERATION

119 The College is at a transitional stage of development. Given its origins, development and size, it was previously able to rely on informal lines of communications between members of staff and between staff and students to assure quality and maintain academic standards. These continue to make an important contribution to the creation of a caring and supportive learning environment for students, and an amenable and stimulating academic environment for staff, notably at departmental level. The College understandably wishes to retain the best of this culture and related modes of operation, whilst introducing systematic and formalised procedures, appropriate to its enhanced size, and consonant with external expectations.

120 The College has introduced a number of formal quality assurance processes over an extended period, but the pace of their introduction has quickened in the past three years. The process of introduction has been led by the College's senior managers, amongst whom the positive contribution of the AP(QA) has been particularly noteworthy. In the judgement of the audit team, some time will be required for these innovations to be understood and effectively operated throughout the College, and the process of adjustment might be assisted if the College was periodically to stand back and review its quality assurance arrangements as a whole. improved understanding and ownership of these arrangements might also be enhanced if the College was to seek critical feedback from its staff and external peers on the structures, standard operating procedures, instruments, documents and reporting conventions it employs. The College might also find it helpful to seek information on good practice from comparable higher education institutions to feed into its own quality assurance processes. The team is confident that on the basis of the progress it has already made, the College is capable of further rapid advances.

121 From its audit enquiries the audit team would particularly commend the College for:

(i) sustaining and enhancing informal processes which it had previously found effective, rather than simply replacing them (paragraph 36);

(ii) its friendly, informative and well-organised interview and admissions arrangements (paragraph 56);

(iii) the level of support it provides for its postgraduate students (paragraph 64);

(iv) the high reputation its support services have established amongst its students, and for the work of all its teaching and support staff in sustaining a caring culture throughout the College (paragraph 66);

(v) the service provided by its Library (paragraph 69);

(vi) the priority it has given to enhancing its information technology provision, and for the range of training and support services for users provided by its Information Technology Centre (paragraph 71);

(vii) the increased attention being given to student representation (paragraph 84);

(viii) the prominence and attention it has given to staff development, and the variety of needs-related development opportunities it has provided for members of its teaching staff (paragraph 102);

(ix) the successful introduction of an appraisal scheme which has won the support of academic staff and for the opportunity being taken to review, upgrade and extend the scheme (paragraph 106).

122 In the light of the findings of the audit team, the College is invited to consider the necessity of:

(i) reviewing and clarifying the guidance it offers its staff on its requirements for the production of reports and minutes (paragraph 32); accelerating the introduction of its plans for periodically reviewing the totality of its formal and informal quality assurance arrangements (paragraphs 37 and 120);

(ii) bringing together and codifying all its assessment regulations in a single document for consideration and approval by the Academic Board, and identifying those principles and practices which it expects all its programme teams to apply to assessments (paragraph 76);

to consider the advisability of:

(iii) satisfying itself that the terms of reference of its committees continue to be appropriate, clarifying them where necessary, and disseminating them to staff (paragraph 31);

(iv) assuring itself that its annual course monitoring arrangements are as effective as it would expect and that Annual Course Reviews are consistently and thoroughly completed (paragraph 54);

(v) clarifying where responsibility lies within its structures for monitoring and reporting on assessment practice (paragraph 74); considering how any gap in the briefing of new external examiners might be closed, and whether a general briefing document might be devised for them (paragraph 78) and reviewing how the QSC discharges its responsibilities for the receipt and evaluation of external examiners' reports, including the monitoring of their consideration by other committees (paragraph 79);

(vi) assigning to the Academic Board a more prominent strategic role in the development of an institutional policy for academic standards (paragraph 81);

(vii) reviewing its arrangements for receiving, analysing, and acting upon the reports of external quality assurance agencies (paragraph 90);

(viii) improving its practice in confirming the appointments of probationary members of the teaching staff (paragraph 99);

(ix) giving greater prominence In its annual monitoring procedures to generic issues relating to collaborative provision; gathering and reflecting on the good practice of its collaborative activities, in order to generalise that experience to all courses and programmes of study it offers with international partners (paragraph 116);

and to consider the desirability of:

(x) reassessing the means by which it can judge the viability of taught postgraduate programmes of study (paragraph 42);

(xi) setting out the criteria used to identify potential members for validation panels in its Handbook (paragraph 45); framing clear guidance on how to record and set out the deliberations, judgements, and conclusions of preliminary and final validation panels for inclusion in its Handbook (paragraph 47); setting out in its Handbook the criteria it uses to determine how a proposal to amend a programme of study or course will be dealt with (paragraph 50);

(xii) establishing means of monitoring its arrangements for the tutorial support of its students (paragraph 58);

(xiii) surveying the form and content of current course handbooks in order to identify good practice in their content and disseminate such good practice throughout the College (paragraph 60);

(xiv) encouraging its academic and student support services to develop, adopt, and publish standards for the delivery of their services for subsequent monitoring by the College against such standards (paragraph 68);

(xv) identifying instances of good practice within the present range of its assessment arrangements and disseminating information about these to its staff (paragraph 76);

(xvi) considering how best it might select and train student representatives (paragraph 84); reviewing the design, content, and methods of analysis of the questionnaires it circulates to students, and the presentation of resulting information (paragraph 85); reviewing the standard form it provides for heads of department to respond to student feedback, and ensuring that the formal responses of heads of department to student feedback information are clearly presented (paragraph 86); considering how best it might inform students that it has received their feedback comments and what it has done to respond to them (paragraph 87);

(xvii) collecting details of the experience gained by its staff in their various external roles, and considering how such information might be disseminated and put to use (paragraph 89);

(xviii) giving greater prominence in its academic policies to innovation in teaching and learning and to using its quality assurance processes more actively to promote quality enhancement (paragraph 92);

(xix) lending greater formality to reviewing the operation of its equal opportunities policy (paragraph 108);

(xx) reviewing the form and content of its Students' Charter and clarifying its relationship to the Charter for Higher Education issued by the Secretary of State for Wales (paragraph 118).

APPENDIX

MANAGEMENT AND ORGANISATIONAL STRUCTURE

Structure of Responsibilities - See Diagram 1.

The Principal is the Chief Executive of the College and is supported at a senior level by a Deputy Principal, three Assistant Principals and a Director of Finance and Support Services. The present administrative structure and allocation of duties are the result of a commission by the College to a firm of consultants (KPMG) to advise on administrative structure, allocation of duties, job descriptions and performance-related pay.

Diagram 1 shows in broad terms the way in which duties have been allocated to the third tier of Management.

There is a close working relationship between members of the Senior Management team, with informal meetings frequently taking place. On a more formal basis, the Senior Management team meet once a week when they are joined by the Registrar, Director of Finance and the Deputy Director of Finance and Support Services.

College Academic Committee Structure - See Diagram 2.

(a) The Assistant Principal (Academic Affairs) is responsible for overseeing Heads of Subjects in the following subject areas:-

Archaeology
Art
Cymraeg (Welsh)
Education
English
Geography
History
Mathematics
Music
Physical
Education
Religious Studies
Technology
Theatre Studies

All Subjects Heads are budget-holders and are responsible for the management of staff within their subject areas.

(b) The Assistant Principal (Academic Affairs) is also responsible for overseeing the Course Directors in:-

Dyniaethau (Humanities)
Health and the Environment
Humanities
The Rural Environment
Theatr, Cerdd a'r Cyfryngau (Theatre, Music & Media)
Heritage Conservation (Archaeology)
Welsh Studies

Most Course Directors are responsible for inter-disciplinary courses within the subject areas in (a) above. Some act as Course Director/Head of Subject. Course Directors are not normally budget-holders although it has been customary, in some cases, to allocate monies when setting up new courses as pump-priming funds.

The Assistant Principal (Academic Affairs) is also responsible for overseeing the:-
Director of Research
Dean of Bilingual Studies
Director of Information Technology.

Top