Higher Education Quality Council

Scottish Agricultural College
Quality Audit Report
October 1996

ISBN 1 85824 311 4

---

CONTENTS

• FOREWORD
• THE REMIT
• THE COLLEGE CONTEXT
• THE AUDIT PROCESS
• SYSTEMS AND ARRANGEMENTS FOR QUALITY ASSURANCE
• THE DESIGN, APPROVAL AND REVIEW OF PROGRAMMES OF STUDY
• TEACHING, LEARNING AND MONITORING THE STUDENTS' EXPERIENCE
• STUDENT ASSESSMENT AND THE CLASSIFICATION OF AWARDS
• FEEDBACK AND ENHANCEMENT PROCESSES
• STAFF APPOINTMENT, DEVELOPMENT, PROMOTION AND REWARD
• CONTENT OF PROMOTIONAL MATERIAL RELATING TO ACADEMIC PROVISION
• CONCLUSIONS AND POINTS FOR FURTHER CONSIDERATION

FOREWORD

1 The Higher Education Quality Council (HEQC) was invited by The Scottish Agricultural College to undertake a quality audit of the College. The Council is grateful to the College for the willing co-operation provided to the members of the audit team.

THE REMIT

2 The Quality Assurance Group of HEQC undertakes quality audits according to the following terms of reference:

(i) to consider and review the mechanisms and structures used by those institutions in membership of the owner bodies of the Higher Education Quality Council to monitor, assure, promote and enhance their academic quality and standards, in the light of their stated aims and objectives; and to undertake a similar consideration and review in respect of other institutions of higher education, at their request;

(ii) to comment on the extent to which such procedures in place in individual institutions reflect appropriate good practice in maintaining and enhancing quality, and are applied effectively;

(iii) to prepare and publish a report on each audit undertaken;

(iv) to prepare and submit an annual report to the Board of Directors of the Higher Education Quality Council;

(v) to liaise with the other groups of the Higher Education Quality Council, drawing their attention to such matters and findings which may be of interest to the higher education system and which may merit further research and development; likewise receiving benefit from the work of the other groups.

THE COLLEGE CONTEXT

3 The Scottish Agricultural College (SAC) was established on 1 April 1990 following the acceptance by the Secretary of State for Scotland of the main recommendation of the Williams Committee Report, A Collegiate System for Agriculture in Scotland (HMSO 1989). The establishment of the SAC brought into a single body the resources and activities of the former North of Scotland College of Agriculture (Aberdeen), the East of Scotland College of Agriculture (Edinburgh), and the West of Scotland College of Agriculture (Auchincruive), together with those of the Scottish Agricultural Colleges Company (Perth), the latter having been established in 1987 to manage and co-ordinate the Advisory and Veterinary Services of the three colleges on an all-Scotland basis. The SAC, which is an independent company limited by guarantee and of charitable status, is in receipt of grant-in-aid funding from the Scottish Office Agriculture, Environment, and Fisheries Department (SOAEFD), and was designated a Central Institution for Higher Education under the Central Institutions (Recognition) (Scotland) Regulations, 1990. The College has three primary functions identified in its mission statement: education and training; research and development; and advisory and consultancy work.

4 The College has three main centres of study located in Aberdeen, Edinburgh and Auchincruive, with associated farms and a network of eight veterinary centres and 24 advisory offices located strategically throughout Scotland. Formal education and training are undertaken mainly at the centres of study, farms and veterinary centres. The advisory offices provide the location for the local advisory services, as well as providing resources for outreach work and for local training. The SAC's work at these various locations includes provision of education courses (mainly at higher education level) and advanced training; the conduct of research funded by the SOAEFD and by other funding sources; advisory and consultancy services for the rural sector and land-based industries; veterinary diagnostic services; and the National Animal Disease Surveillance System for Scotland. These activities are undertaken within an integrated organisational framework which has been a feature of the Scottish system of agricultural education and training since it was first developed during the early part of the century. In each of its separate functional areas, however, the SAC has a formal funding relationship with SOAEFD. This is embodied in a Memorandum of Understanding and a Financial Memorandum which were introduced on 1 April 1993. The annual accounts for the financial year ending in 1995 show that excluding training and unclassified income, 26 per cent of the SAC's income related to formal education and training, 35 per cent to research and development, and 39 per cent to advisory and consultancy work.

The organisational structure

5 When the SAC was established in 1990, its constituent parts had previously been operating as separate companies and this was reflected in the management reporting frameworks established in each part. In 1990, the SAC restructured this management reporting framework across the new college into four operating divisions which were further sub-divided on a department or unit basis. The three main operating divisions - the Academic Services Division, the Veterinary Services Division and the Advisory Services Division - embraced the activities based at the centres of study and related farms, at the veterinary centres and in the advisory offices respectively. A fourth division was established, the Ventures Division, which included only a small number of staff and was, in essence, a reporting framework established to deal with specific new business initiatives which were perceived to have special management requirements and operational needs. To facilitate the integrated development of each of the SAC's three main functional areas, a deputy principal/dean of centre was appointed, in 1990-91, to take a leading responsibility for each centre of study and also for one of the functions of education, research, or specialist advisory work. In addition, three vice-deans with education, research or specialist advisory responsibilities were appointed from the heads of department at each centre. A matrix was thus established which allowed the progressive development of functional management within the Executive Board framework and line management structure. The Executive Board is the most senior executive committee of the SAC. This comprises the Principal and Chief Executive, the Executive Director, the three deputy principals / deans of centre, the Director of the Advisory Service, the Director of the Veterinary Service, and the Secretary and Treasurer. The Executive Board has three functional committees reporting to it, the Advisory and Consultancy Committee, the Research and Development Management Group, and the Academic Board. A statement prepared by the College, describing its constitutional and management arrangements, is attached as an appendix to this report.

COSAR Ltd

6 In 1994, the SAC Foundation was established as a separate charitable trust, designed to administer charitable gifts and other funds for the College. The foundation subsequently established COSAR Ltd as an associate company of the SAC with a remit to facilitate the commercialisation of selected SAC innovations, and to develop commercial ventures emerging from the SAC's science and technology base. The Board of Directors of COSAR Ltd and the SAC Executive Board are linked through overlapping membership, with the SAC Principal and Chief Executive and SAC Executive Director acting, respectively as Vice-Chairman and Managing Director of COSAR Ltd. The Chairman of the SAC Board of Directors is also Chairman of the Board of COSAR Ltd.

Links with other higher education institutions

7 The SAC has strong formal links with other parts of the Scottish higher education sector, offering joint undergraduate and postgraduate courses with the Robert Gordon University in Aberdeen and Napier University in Edinburgh. The SAC also has research and teaching links with Heriot-Watt University in Edinburgh. However, the dominant links, where there are particularly close associations, are with the Universities of Glasgow, Edinburgh, Aberdeen and Strathclyde. In each of these latter cases, there are explicit associate institution arrangements through which the SAC's undergraduate and postgraduate awards are validated. Provision is also made in these arrangements for joint courses; for exchanging teaching contributions between the College and the universities; and for joint research programmes. In these cases there is also collaboration in the sharing of resources, and at Edinburgh and Aberdeen, the SAC and the respective universities jointly occupy buildings and have agreements for shared use of resources within the frameworks of the joint agreements on co-operation.

Undergraduate and taught postgraduate provision

8 In session 1995-96, excluding research postgraduate students, the SAC had a total of around 1200 (FTE) registered students, of whom some 1000 were following degree or sub-degree courses. The SAC courses cover six broad subject areas; Production Industries (392 FTEs); Horticulture (116 FTEs); Business and Resource Management (194 FTEs); Science and Technology (109 FTEs); the Environment (80 FTEs) and Leisure and Tourism (241 FTEs). In these subject areas the College has developed a suite of courses that link HNC, HND and undergraduate provision (see below, paragraph 45). The portfolio of SAC courses includes some 12 HNC, 14 HND, 9 undergraduate courses and 17 taught postgraduate courses.

Postgraduate research provision

9 The SAC has a strong involvement in postgraduate research and at 1 February 1996, a total of 138 students were enrolled for PhD degrees by research. These students are registered mainly with the Universities of Aberdeen, Edinburgh, Glasgow and Strathclyde, with supervision by members of the SAC staff, either on their own, or jointly with university supervisors within the framework of the associate college agreements.

THE AUDIT PROCESS

10 Prior to the audit visit, the College submitted documentation outlining its aims and strategies; its management and committee structures; academic and non-academic staffing arrangements; teaching and learning initiatives; and the mechanisms used to evaluate staff performance. The briefing material included the College Education Manual which the audit team found particularly informative. The briefing material also included a wide range of committee and other papers, and materials to illustrate the College quality assurance mechanisms in operation. Following its reading and analysis of the briefing documentation, the team proposed a programme of meetings for the visit and requested a limited amount of additional contextual materials to help it to fully understand and verify the structure and processes of the College's quality assurance arrangements.

11 The audit visit took place over three days, 8-10 May 1996. During the course of the visit the audit team had wide-ranging discussions with individuals and groups, including:

• the Principal;
• the Deputy Principal and the Systems Manager (Education);
• members of College committees with responsibilities for academic standards, quality and planning;
• members of the College academic and support staff;
• members of the College's Students' Representative Council;
• a broad range of full- and part-time undergraduate students, and postgraduate taught and research students from the College and partner institutions.

In total, the team met approximately 104 members of the College staff drawn from each centre, five staff from associated universities, 50 College students, and some 10 students from partner institutions studying on collaborative programmes offered in conjunction with the College.

12 The audit team wishes to express its gratitude to those members of the College, and the representatives from partner institutions, who attended the various meetings to discuss the College's quality assurance arrangements.

13 The audit team consisted of Mr R Gilmour, Dr E P Maher, Professor G E Taylor, auditors and Ms D Cerqua, audit secretary. The audit was co­ordinated for HEQC by Mr N L Sharp, Head of the HEQC Scottish Office and Assistant Director, Quality Assurance Group.

SYSTEMS AND ARRANGEMENTS FOR QUALITY ASSURANCE

14 The SAC is unusual in comparison with many other institutions of higher education in several ways. It has three main campuses which are a significant distance apart; its higher education provision is validated by several different universities and by the Scottish Vocational Education Council (SCOTVEC); and its taught courses comprise only one of three major activities. The College systems and arrangements for the quality assurance of its educational provision thus have to integrate with those of the different external bodies, as well as seeking to provide unity, consistency and comparability across the three sites.

15 Overall responsibility for quality assurance in the College rests with the Principal and Chief Executive. However, as indicated in paragraph 5 above, each centre of study is managed by a deputy principal/dean of centre who also has responsibility for one of the SAC's three main areas of activity. Each of the three functional areas is also represented at each centre by a vice-dean. Thus, key roles with respect to educational quality assurance are played by the Deputy Principal (Education), who is also Dean of the Auchincruive Centre, and the three vice-deans (education). The central operational roles in the quality assurance process are played by the Systems Manager (Education) and the Academic Registrar. Each centre of study is divided into academic departments, and these departments are grouped, in a relatively informal way, into schools. The latter may cross centre boundaries and, the audit team was informed, exist primarily for the promotion of research. The management of resources is generally located at departmental level. Courses, which frequently cross departmental boundaries, are managed by course tutors who negotiate their staffing requirements with the heads of department concerned. Heads of department thus balance the competing demands of teaching, consultancy, and research on their staff. General matters relating to course and student matters at each centre are the responsibility of a senior tutor who chairs the Centre Education Board (see below, paragraph 21). Although the resulting matrix management structure, with key individuals often dividing their time between several different roles, appeared complex to the audit team, it was assured, by all the staff with whom it met, that the system operated smoothly.

The Academic Board and its committees

16 The Academic Board reports directly to the Executive Board of the SAC (see above, paragraph 5). The Academic Board is a relatively small group, whose membership consists of the Deputy Principal (Education) as Chair, the three vice-deans (education) and the Systems Manager (Education). Its role relates to the strategic development and implementation of College policy with respect to education and training activities. This role is manifested through a number of centre-based and cross-college groups and committees which report, directly or indirectly, to the Academic Board.

17 The Education Quality Standards Committee (EQSC) is appointed by, and reports to, the Academic Board. It is chaired by the Systems Manager (Education), and includes in its membership the Academic Registrar and three additional members appointed by the Academic Board. Its prime function is described by the College, as being 'to promote quality in education in SAC'. It plays a central part in the validation of new courses (see below, paragraphs 27 to 31) and the revision of existing provision (see below, paragraphs 32 to 39). The EQSC is also described, in the Education Manual, as being responsible for overseeing an internal audit function (see below, paragraphs 34 to 35).

18 The Research Degrees Committee also reports to the Academic Board. It has responsibility for the development of procedures relating to the management of research student matters; the monitoring of quality assurance procedures relating to research students; and the scrutinising of progress reports from research students and their supervisors. It is chaired by the Deputy Principal (Research), and includes in its membership the Systems Manager (Education), and the Systems Manager (Research and Development), together with a postgraduate tutor from each centre of study.

19 The Teacher Development and Appraisal Group is chaired by a member of the Academic Board and comprises the SAC Training Officer, three teaching staff nominated by the Teacher Liaison Groups (see below, paragraph 24), and two members of the teaching staff nominated by the Academic Board. Its remit is to advise the Academic Board on matters relating to teaching quality and teaching staff development and appraisal procedures.

Centre Academic Planning Groups

20 Each of the three centres of study has a Centre Academic Planning Group and a Centre Education Board. The Centre Academic Planning Group (CAPG) at each centre is chaired by the Vice-Dean (Education), and includes in its membership, the Dean, the Systems Manager (Education), the Senior Tutor and other nominees of the Dean. The CAPG advises the Dean of Centre on resourcing matters at the centre in relation to education, and is also responsible for submitting, to the Academic Board, proposals for course developments and revision; proposals for inclusion in the SAC Academic Plan; and general proposals relating to educational developments.

Centre Education Boards and the Senior Tutors Group

21 In each centre of study there is a centre education board comprising, the centre's Senior Tutor (in the chair), the Vice-Dean (Education), course tutors from the centre, and the SAC Academic Registrar. The centre education boards receive course reports on a termly and annual basis, and provide a forum for discussion on a wide range of matters relating to the management and operation of courses at the centre, including matters related to teaching and learning, recruitment, industrial liaison, and careers advice.

22 The centre education boards report to the Senior Tutors Group which, in turn, reports to the Academic Board. The Senior Tutors Group includes the Systems Manager (Education and Training) in the chair, the Academic Registrar, the Training Services Officer and the Student Services Manager, in addition to the three senior tutors. The Senior Tutors Group is perceived by the College to have a major role in the development, implementation and monitoring of the SAC quality control and quality assurance procedures for educational activities. It also provides a primary means of ensuring comparability and compatibility across and between the centres, and in ensuring that the SAC is represented as a single educational institution in its interaction with external organisations.

Course management teams

23 Courses are managed by course management teams which report to a centre education board via the relevant course tutor. Course management teams consist of a number of subgroups as follows:

• the core team, comprising the course tutor, year administrators and a representative from each contributing department. The core team is responsible for recruitment, day to day course management, and the conduct of annual review.
• the teaching team, comprising all staff involved in teaching on the course. The remit of the teaching team includes discussion of the annual course review and other quality assurance matters.
• the student liaison group, comprising the core team plus student representatives from each year. The student liaison group is responsible for discussing student concerns.
• the industry liaison group, comprising the core team plus at least two industrial representatives. The industry liaison group is concerned with all matters relating to industrial liaison including, where appropriate, work experience.
• the assessment board, comprising the core team plus other teachers as appropriate. The assessment board considers and approves results of assessments and their implications for student progression, and makes recommendations to the validating boards for academic awards.

Teacher Liaison Groups

24 In addition to the formal structures described above, each centre operates a voluntary teacher liaison group which elects its own officers and determines its own programme of activities. All teachers in a centre are eligible to be members of the teacher liaison group. The audit team met representatives of one of the teacher liaison groups and was impressed by the commitment of its members and the way in which it sought to share and promote good practice in teaching and learning.

The SAC Education Manual

25 Detailed descriptions of the roles of individual staff, the remit and membership of committees, and the various quality assurance procedures are provided in the SAC Education Manual. At the time of the audit visit, copies of this document were available in hard copy, in each department, but were not provided individually to each member of staff. The audit team was informed that it was planned to make the manual available electronically at some future date. Those staff whom the team met were clearly aware of the existence and role of the Education Manual and how to access it in their own departments. It was clearly regarded by these staff as a useful and practical set of guidelines.

Overview

26 The College's formal procedures for quality assurance appeared to the audit team to be detailed and complex for what is a relatively small institution in terms of total student numbers. There is a large number of committees, and some members of staff carry out several different formal roles within that complex structure. Discussions with staff, however, suggested to the team that both the structure, and individual roles within that structure, were, in general, clearly understood. In addition, it rapidly became clear to the team, in the course of the audit, that the formal structure was accompanied by frequent cross-centre and cross-college informal contact.

THE DESIGN, APPROVAL AND REVIEW OF PROGRAMMES OF STUDY

Design and approval of new courses

27 Detailed and clearly written guidance notes on course development and approvals processes are given in the SAC Education Manual. New course development requires the approval of the Scottish Office Agriculture, Environment and Fisheries Department (SOAEFD) as the funding body of the SAC. The process for approval is that outline approval is given through acceptance of the SAC Strategic Plan, with final approval following on from the validation process and the submission of definitive course documentation to the SOAEFD (see below, paragraph 31).

28 Any individual or group within the College may propose a new course which is then discussed at the relevant Centre Academic Planning Group (CAPG) (or groups where more than one centre is involved). If the proposal is agreed at the CAPG, it is submitted, by the Vice-Dean (Education), to the Academic Board. At this stage no formal documentation is required, but, if the Academic Board agrees to further development, a course planning document must be completed by the proposers. This is a brief (three to four pages) document in two standard sections designed to establish the feasibility of the proposed programme. The first section deals with the curriculum design of the proposal in terms of the aims and objectives, outline curriculum, major electives, and use of existing units. The second section deals with resource issues, including market niche, competition, funding, predicted income and costs. Section one is considered by the EQSC in relation to academic credibility, and section two by the CAPG primarily in relation to resource matters. The outcomes of these considerations are then submitted to the Academic Board which, in turn, makes a recommendation to Executive Board as to whether the proposed course should receive approval in principle and should therefore progress through full development to validation. At this stage, a successful proposal would be included in the SAC Strategic Plan and, where appropriate, the relevant University Joint Board would also be notified.

29 Following approval in principle, the Academic Board appoints a course development team leader who is then responsible for selecting course development team members in consultation with appropriate vice-deans and heads of department. The course development team is responsible for carrying out appropriate market research, and developing the course submission document. This is expected to be largely a 'stand alone' document, not relying on reference to other SAC documentation. It is in three parts. The first of these, which is common to all course submission documents, summarises the SAC academic and management structures, mission and objectives, and quality control and assurance mechanisms. The second section, which is course specific, provides general information on the proposal, including course title, duration, course rationale, course structure and content, teaching and learning strategies, assessment strategies, award regulations, course management structures and processes, course organisation, and resources. Section three contains a full specification of all units to be included in the proposed course.

30 When complete, the course submission document is forwarded to the EQSC for internal scrutiny, the outcome of which is a recommendation to the Academic Board. Once the approval of the Academic Board has been secured, the proposal progresses to a validation event, the precise form of which is determined by the validating authority. For SCOTVEC courses, the Academic Board appoints the chair of the validation panel, and arrangements for the validation event are then made by the SAC Academic Registrar in consultation with the panel chair and development team leader. The panel chair then submits a report of the event, through the Deputy Principal (Education), to the Executive Board. For university validated courses, documentation is submitted to the appropriate university and further processed under that university's regulations. The university reports back to Executive Board on the outcome.

31 Following a successful validation event, final course documentation is produced in the form of a definitive course document and a course handbook. Within the definitive course document all pages must show the date of production and version number, and any amendments agreed following course validation or review must be incorporated. The SAC Academic Registrar is responsible for maintaining master copies of all the definitive course documents and for lodging copies with the validating authorities. Before final implementation, the Deputy Principal (Education) submits a request, on behalf of the Executive Board, to SOAEFD for confirmation of funding approval.

Annual course review

32 All SAC courses are required to conduct an annual course review exercise involving a designated meeting of the full course team. A formal minute of this meeting must be kept, and this forms one component of the documentation used by the College for the ongoing monitoring of its provision. The process to be followed in annual course review is clearly documented in the Education Manual and requires consideration of the performance of the course in relation to its aims and objectives. Factors to be considered include resource matters, student recruitment performance and satisfaction, industrial liaison, and staff development. Performance indicators such as progression rates and first destination statistics are required to be explicitly considered. The formal record of the annual course review must include a summary of the course team's proposals for development and action, indicating the staff responsible for particular actions and the agreed timescale for action. A summary of the reports of external examiners must also be included, together with, where appropriate, a report of any meetings with SCOTVEC external verifiers (see below, paragraph 60). The review record must also include the report of the course industrial liaison group (see below, paragraph 76).

33 When the College was initially created, a system of internal scrutiny of reports of annual course reviews was implemented which involved annual scrutiny of all reports by a sub-committee of the EQSC. This scrutiny panel had as its remit, consideration of both the annual course review documents submitted by course tutors and also the consideration of the overall effectiveness of the College's quality system. The audit team received copies of the reports of the scrutiny panel for 1992-93 and 1993-94. In the latter report, the panel focused particularly on the operation of internal procedures and recommended to the Academic Board a change to what they considered would be a more effective and efficient procedure of internal audit.

Internal audit

34 New procedures for internal audit were implemented in 1995-96 to review the provision for 1994-95. In the new system, the Academic Registry became responsible for compiling, on an annual basis, a profile representing each course's performance. Each profile incorporates the course's stated aims and objectives, together with data relating to recruitment targets, number of applications, number of entrants, student opinion surveys, progression rates, first destination statistics and minutes of course committees. The course tutor for each course is provided with the draft profile for discussion at their annual course review meeting. Following this meeting, the course tutor is required to provide a commentary on the profile and submit both to the EQSC. All course profiles and commentaries are scrutinised by the EQSC which has been enlarged through the addition of co-opted members from the SAC Council in order to provide added objectivity and breadth. Any evidence of non-conformity with SAC policy requirements or procedures is summarised and reported to the Academic Board. The Education Man al indicates that the operation of these processes will be subject to a new process of internal quality audit. The Education Manual indicates that this will be undertaken by specifically trained SAC staff with sufficient teaching and management experience, and that, on average, each course might expect to be audited once every five years.

35 At the time of the audit visit, the College staff would have had only a short time to consider the new procedures, and no member of staff would have had direct experience of all elements of these new procedures. Nonetheless, the audit team was concerned to note, from its discussions with academic staff, a lack of a uniform understanding of the new procedures. The team was also informed of the 'atypical' timing of the annual review process for the academic year 1994-95, the first year within the new arrangements. At the time of the audit visit, the training of internal auditors had not taken place, although the team was subsequently informed that arrangements for such training had been made with the University of Stirling Management School which would allow the internal audit process to commence in session 1996-97.

Course revision

36 The need for a course to be amended by, for example, replacing units or altering the core or elective status of units, may become apparent through the annual course review processes at any time during the period of validation of that course. The desire, or requirement, for change may be identified by the core team, the EQSC, or the Academic Board. Any proposal for change is developed by the course team and submitted, via the Systems Manager (Education), for decision by the Academic Board. Where the revision relates to a degree course, both the proposal and the Academic Board's decision are notified to the relevant Joint Board.

37 If the proposed change is approved by the Academic Board, the subsequent procedure depends on whether the proposal is considered to be a minor or a major amendment. The distinction is based on the overall scope of the change, and advice is provided by the Systems Manager (Education) on this topic. In the case of minor amendments, documentation including, where appropriate, new unit descriptors, is submitted by the course tutor, through the Systems Manager (Education), to the EQSC and the Academic Board for approval. Communication to SCOTVEC of the change is through the SAC Academic Registrar, and to universities through the relevant Joint Board. A major amendment requires revalidation of the course.

Course revalidation

38 Each course is validated for a period determined by the validation panel, the maximum being five years. During this period, there is regular monitoring through the annual course review process. Periodically, at the end of the initial validation period or when major amendments are proposed, a more formal review is conducted. This is described, in the Education Manual, as being a hybrid between the initial validation and annual review processes, and is referred to by the SAC as revalidation. It follows a similar pattern to the validation process, but, in addition, is expected to analyse in detail the student experience of the existing course, together with a fundamental re­appraisal of the purpose, curriculum design and learning strategy of the course in the light of the developing institutional mission and policy framework.

39 The revalidation process requires each course management team to compile a self-critical revalidation document, utilising appropriate qualitative and quantitative data. The Education Manual describes in some detail the areas to be covered by revalidation documents, including the re-appraisal and review of aims and objectives; recruitment and employment issues; teaching and learning strategies; and the overall resourcing of the course.

Design, approval, and review mechanisms: an overview

40 The audit team discussed the College processes for course design, review, and revalidation with a range of staff, and, in addition, analysed documentation relating to the various elements of the course approval and review processes. It appeared to the team confusion referred to earlier (see above, paragraph 35), procedures were well known to staff and were carefully followed.

TEACHING, LEARNING AND MONITORING THE STUDENTS' EXPERIENCE

41 Although education accounts for only about one quarter of the SAC's current activities by revenue (see above, paragraph 4), it is viewed by the College as being very much central to its core function. The other main activities, of research and consultancy, are seen as making a substantial contribution to teaching and learning and providing a distinctive character to the College's course provision.

Formal and informal mechanisms for monitoring the students' experience

42 The formal College processes for monitoring the students' experience include meetings of course management teams and their student liaison groups (see above, paragraph 23); annual review procedures; revalidation procedures; and the use of student evaluation questionnaires in relation both to individual units and to complete years of courses (see below, paragraph 75). The EQSC has a central role in monitoring teaching and learning strategies, and receives all reports of annual reviews (see above, paragraph 33), as well as the minutes of the meetings of student liaison groups and the data gathered through the student evaluation questionnaires.

43 In addition to the formal processes of monitoring and review, the audit team was informed that the small size of the College, together with the close-knit nature of the staff and student communities at each centre of study, created an informal context within which it was relatively easy for students to raise with tutors any concerns which they might have in relation to their courses. The team was informed that this extensive contact between students and tutors acted as an effective informal monitoring system which supported the formal mechanisms described above. From its discussions with staff and students, the team shared the College's view of the value of these informal mechanisms.

Education liaison officers

44 Each centre of study has an education liaison officer, working under the general direction of the SAC Academic Registrar, responsible for implementing recruitment and schools liaison policy at the centre. The education liaison officers engage in a wide range of promotional and advisory functions through schools, careers conventions and exhibitions. The education liaison officers are also responsible for identifying any special needs that applicants may have in relation to their chosen courses, and for informing the relevant course tutors and senior tutors of these needs. The role of the education liaison officers extends to contributing to the formal induction programmes for new students. Students whom the audit team met indicated that they had found the induction process to be very helpful and supportive and, in particular, they commented favourably on the role of the education liaison officers. The team found the extended role of the education liaison officers, in providing a supportive link between schools and the College, commendable.

Links between HND and undergraduate courses

45 The College has developed a structure of course provision, in particular through its Bachelor of Technology degrees, which allows full articulation between HNC, HND, and degree courses. Through the documentation and discussion with staff, the audit team learnt that this allowed the College to offer a range of nationally recognised awards at different levels which were related to the needs of employers, and which reflected the admissions and access policy of the College. The HNC and HND stages of programmes comprise units designed to conform to SCOTVEC's specifications. This involves units described in terms of competencies, associated range statements, and assessment criteria. The degree stages of programmes are described in terms of more traditional approaches to learning and assessment.

46 The College was aware of the potential difficulties that could arise for students in making this transition from the competency based approach of HND courses to the more traditional approaches to undergraduate learning and assessment (see below, paragraph 70). The audit team learnt that, partly as a result of feedback from students, a bridging study skills programme had been developed to support students in making this transition. The students whom the team met, and who had experienced this course, indicated that they had found it to be very helpful. However, it did appear to the team that take-up of the course was variable across the College and, in the view of the team, the College might wish to consider how it might ensure that all students benefit from an appropriate bridging course.

Innovations in teaching and learning

47 Centre education boards (see above, paragraph 21) have a specified role in identifying and promoting innovative approaches to teaching and learning. The centre education boards report to the Senior Tutors Group which also has an identified role in disseminating good practice. The EQSC is also charged with the task of identifying and disseminating good practice through its scrutiny of course submissions. The teacher liaison group at each centre of study is also a key group in stimulating interest in, and developing, innovative approaches to teaching and learning (see above, paragraph 24). The audit team learnt that the College had provided a further stimulus to innovation in approaches to teaching and learning through the provision of funds to support research and development within the College in this area. The team also heard that the College had recently held a workshop for staff on research in teaching and learning.

48 During its visit, the audit team was told that in practice, as in other areas of College activity, much is done to promote innovation in teaching and learning through informal networks and also through the regular activities of course management teams. The team also learnt of a range of interesting initiatives undertaken by the teacher liaison groups, such as a series of workshops on teaching innovations. The team formed the view that the teacher liaison groups appeared to operate very effectively in linking formal and informal systems, and the team would wish to commend both the work of these groups, and the College's support of them.

Library and IT provision

49 Each centre has a library committee which meets under the chairmanship of a vice-dean, with those at Aberdeen and Edinburgh centres of study including university representation. The College has computing/IT units at each centre, their activities being co-ordinated through the SAC Information Technology Group, chaired by the Assistant Principal. In Auchincruive, the Library Committee, whose remit includes advising on electronic and computer assisted systems of learning, includes an IT representative.

50 The annual course review and revalidation processes are the formal mechanisms by which the College monitors the adequacy of its library and IT provision (see above, paragraphs 32 to 40). The audit team was informed that, because of the intimate nature of the College, any problems would be readily identified, communicated, and dealt with through the frequent contact between service providers and users, both staff and students. In the view of the team, this interaction of informal and formal mechanisms appeared to work effectively, and students reported in generally favourable terms on both the level of IT and library provision, and their opportunities for commenting on the adequacy, or otherwise, of that provision.

Student advisory services

51 All students are assigned to a personal tutor whose role is to provide general educational advice and guidance. In addition, there is a College Welfare Officer who can provide more specialised forms of student support, and arrange access to professional counselling services outwith the College. The counselling and welfare support services are co-ordinated by the College Student Services Manager. The Student Services Manager works closely with the senior tutors and the education liaison officers, and also with the President of the Students' Representative Council. Careers advice and guidance is viewed by the College as the province of course tutors in the first instance, since they are assumed to have most direct specialised knowledge in relation to particular vocational areas. Students at Aberdeen and Edinburgh centres of study have access to the careers services of the associated universities.

52 In general, students whom the audit team met appeared to find the support services satisfactory. The team heard that previous complaints about the quality of careers advice had been responded to positively. The team was advised repeatedly that, given the close-knit character of the College community in the various study centres, students found much of the advice and support they needed in direct informal contact with tutors. Students with whom the team met reported considerable appreciation of the knowledge and approachability of staff on academic-related and other matters.

Research students

53 The College has developed a substantial research student population, with 138 PhD students registered at the time of the audit visit. These students are supervised by SAC staff, either alone or jointly with a university-based supervisor. The SAC staff involved frequently have appointments, as associate members of staff, at the validating university. At the time of the audit visit, the College had recently introduced a training programme for supervisors of research students. Under this programme, staff are required to attend a training workshop before spending at least one year as a second supervisor under the mentorship of an experienced primary supervisor.

54 Each centre of study has a postgraduate tutor who is responsible for co­ordinating postgraduate student matters at the centre. The centre postgraduate tutor chairs the centre postgraduate committee which involves both postgraduate student and supervisor membership. The postgraduate committees are intended to provide a forum for discussion of all matters relating to the education and welfare of postgraduate research students at each centre. The Research Degrees Committee of the College is responsible for formally monitoring the progress of postgraduate research students (see above, paragraph 18).

55 Research supervisors and students both complete annual reports which are submitted to the relevant centre postgraduate tutor who reports annually to the Research Degrees Committee. In addition the Research Degrees Committee receives data generated by the questionnaires which all research students complete annually. At the end of their second year, research students are normally required to give a presentation at a SAC postgraduate workshop. At the time of the audit visit, a generic research training programme for students had recently been introduced.

56 From the analysis of the documentation, and from discussion with staff and students, the audit team formed the impression that structures and processes to support postgraduate students at SAC were well organised and effective, with the main components of an effective quality assurance system in place. The team formed the view that, while some mechanisms were still evolving at the time of the audit visit, the process as a whole was being effectively driven and monitored by the Research Degrees Committee. The team would wish to commend the energetic and active role played by the Research Degrees Committee.

Equal opportunities in teaching and learning

57 The College has an equal opportunities policy, stat d in its Education Manual, which includes a commitment to monitor the composition and progress of the student body. In consequence, the Academic Registrar collects, analyses and distributes annual data on the characteristics of students, and is expected to alert the appropriate vice-dean (Education) where it is felt that there are grounds for concern.

58 Staff and students whom the audit team met were of the opinion that equal opportunities issues did not currently raise concerns at the College, and a SCOTVEC audit report in 1993 had reflected a similar view. While the team found no reason to doubt this view, in practice it was not clear to it that the College's formal system provided adequate data to allow for effective monitoring. For example, although the College generated data on the ethnic origin of its students, it did not seem, to the team, to have available appropriate population information to enable this data to be appropriately interpreted. The College might wish to reconsider the adequacy of its formal monitoring system in this area.

STUDENT ASSESSMENT AND THE CLASSIFICATION OF AWARDS

Academic standards

59 The Strategic Plan describes the College's corporate policy as being, to 'strive for quality and client satisfaction in the provision of education, conduct of research and provision of advisory services'. The inclusion of the College's three primary functions in this statement was viewed by the audit team as being of particular significance. The team was made aware, through the documentation and discussion with staff, that the interplay with practically-based research and advisory work was significant in enhancing the College's educational provision, and made it particularly relevant to the needs of the industrial and commercial sectors which it serves. These sectors influence the design and delivery of courses through contributions to market research, course validation and annual course monitoring. However, the major direct influences on the College's academic standards appeared to the team to be its external validating bodies, SCOTVEC and the four validating universities.

SCOTVEC

60 SCOTVEC is the validating body for those courses which are composed entirely of national certificate modules or higher national units. In 1990, SCOTVEC issued a consultation paper on future arrangements for quality assurance in which a key principle was the devolution of authority for the management of quality to providing centres. The extent of devolved authority would depend on the demonstrated robustness of the providing centres' structures for quality assurance. Following a SCOTVEC quality audit in 1993, the SAC became the first higher education institution to obtain devolved authority, from SCOTVEC, to validate its own courses. Under these arrangements, the senior tutors at the three centres of study act as internal verifiers (members of College staff who verify that approved assessment processes are being followed). SCOTVEC makes periodic independent checks on the standards of assessment by appointing independent external verifiers who visit the centres of study to sample assessments and meet teaching staff and students involved in the course. From the documentation provided to the audit team, it was clear that the SCOTVEC approach to the specification of learning outcomes and performance criteria was now very firmly embedded in the College. This was confirmed by the team's discussion with staff who commented favourably on the role of competence-based learning and assessment in the explicit setting of standards. Indeed, the team was told that if the College had its own degree awarding powers, it would wish to extend this approach to the specification of standards at degree and honours degree level.

University partnerships

61 The second major external influence on the College's academic standards is the Scottish university sector in general, and its validating universities in particular. The College has a variety of relationships with universities, but these are formalised in associated institution status with the Universities of Aberdeen, Edinburgh. Glasgow, and Strathclyde (see above, paragraph 7). As the College's validating bodies, these universities have ultimate responsibility for ensuring the standards of the College's undergraduate and postgraduate programmes of study. In each case, a Joint Board provides the formal link between the College and the respective validating university. The audit team was told that there were differences between the universities in the details of certain procedures, such as those for appointing and considering the reports of external examiners. Senior managers of the College told the team, however, that they viewed this as no more than a management problem, and one that was outweighed by the benefits to the College of its range of university links in different parts of Scotland. Some of these links are of long standing, and involve joint teaching arrangements and the recognition of College staff as associate staff of the universities. Staff whom the team met expressed the view that the interchange in teaching and research, between the College and the universities, helped to ensure that standards were constantly being compared within a vibrant academic community.

Links with employers

62 The audit team was informed that the College's extensive and close links with industry also influenced the standards of its awards. In addition to the formal inputs of representatives of industry and commerce to the governing body of SAC and to course validation and monitoring, the team was told that College staff maintained close links with their local industrial and commercial contacts, who would quickly inform them if courses were not fulfilling their vocational functions. The team was informed, on several occasions, that the College's blend of education, research, and advisory and consultancy work provided a unique learning milieu for its students and ensured that their courses were grounded in the needs of the industries and employers which the College served (see below, paragraph 76).

Overview of academic standards

63 The audit team was told that these external influences had an impact on academic standards initially through external contributions as part of the College course approval procedures. Subsequently, standards, translated into unit outcomes and assessment criteria by members of the teaching team, are scrutinised during the course monitoring and review processes. While ultimate responsibility within the College for academic standards rests with the Principal, in practice this responsibility is devolved to the Deputy Principal (Education) as chair of the Academic Board, with the scrutiny function being performed by the EQSC, and in particular by its chair, the Systems Manager (Education).

64 So far as the audit team was able to ascertain, the College takes a responsible and careful approach to the specification of its academic standards through its course approval and monitoring procedures within the frameworks provided by its partnerships with external validating bodies. The team noted with interest the range and variety of these partnerships, and the opportunities which they provided for the College to compare the requirements and practice of different bodies, and to develop its own policy on standards, which would reflect its distinctive mission. It was not clear to the team, however, that these opportunities had yet been fully exploited, either by the Academic Board or the EQSC. The team was told that the College had its own standards which it would seek to achieve, even if these exceeded those required by SCOTVEC and the validating universities, but the team vas not made aware of any explicit statement of these standards, or of obvious examples of where they were being applied in addition to those of the validating bodies.

Assessment

65 The assessment strategy for each course is described in the definitive course document. Course tutors have responsibility for the implementation of the defined strategy, and for ensuring that students are fully informed about assessment procedures and regulations. The course documents and handbooks provided to the audit team contained clear descriptions of arrangements for assessment and award classifications, including the award of 'merit' in higher national units and the requirements for progression from the second year of a HND to the third year of the associated BSc or BTechnol course. Students from a range of courses confirmed, in discussion with the team, that the criteria for assessments and award classifications had been carefully and fully explained to them. The team would wish to commend the College for its clear identification and communication to students of the full range of assessment criteria.

66 However, some students commented to the audit team on what they regarded as lack of consistency in the requirements for a 'merit' award. Variability between units in the proportion of students achieving merit had been identified by one of the scrutiny panels of the EQSC in its 1993-94 report, and this is a matter that the College may wish to reflect upon as it develops further its new internal audit arrangements. Notwithstanding this possible problem, staff members, including those recently appointed, appeared to the team to be conversant and comfortable with competence-based approaches to learning and assessment. This positive response indicated to the team the success of the College's provision of staff training and development opportunities in the theory and practice of assessment. Students also seemed to understand and appreciate the value of competence-based approaches, although a number of them commented on the burdens of continuous assessment and on the difficulties encountered in moving from this system on higher national units to the more traditional approaches of assessment at degree level (see above, paragraphs 45 and 46).

67 An aspect of continuous assessment, which was brought to the audit team's attention by most of the groups of students which it met, was the variation in the length of time taken for marked assessments to be returned. This had also been identified as a problem in the EQSC's Scrutiny Panel Reports for both 1992-93 and 1993-94. Some students and members of staff whom the team met volunteered the opinion that courses were currently over-assessed, and that some teachers, who also had research or advisory duties, were over-burdened with marking. In view of the long running nature of this problem, and the fact that this concern was expressed several times to the team by different groups of students, the College may wish to develop means of assuring itself that students' assessments are returned in time for the feedback to provide learning benefits appropriate to the stage of the course and the timing of subsequent assessments.

68 Nearly all of the College's courses are offered at only one centre of study, the exceptions being some courses in Agriculture which are offered at all three centres. The audit team was interested to note that the assessments associated with these courses were not required to be the same at each centre. It was explained to the team that course teams were expected to use the learning resources available to them at a particular centre and to adapt learning and assessment strategies to local circumstances. The team was informed that, while courses might not be identical at different centres, the cross-centre Agriculture Committee was responsible for ensuring their comparability. The team noted that 'marked differences in success rates' between the three centres had been referred to in the most recent (1993-94) report available from the EQSC's scrutiny panel. Given the importance of comparability of provision and standards within and between courses in the further development of the College's identity as a single institution, the College may wish to consider how its internal audit procedures may be developed in order to deject any significant differences in standards and levels of student achievement between its centres, and to take any necessary remedial action.

Grievance and appeals procedures

69 The College has a formal appeals procedure which is stated in the Education Manual. This procedure involves a two stage process. At the first stage, the appeal is to the core team of the relevant course management team. The second stage appeal involves the appeals procedure of the validating university through the relevant vice-dean (Education). The audit team noted in the documentation an example of the second stage appeals mechanisms in operation, involving a Joint Appeals Board with the University of Glasgow. Students whom the team met appeared to show a variable knowledge of the formal appeals system and, in general, stressed the likelihood of a personal approach to an appropriate member of staff resolving the problem in the first instance.

Articulation

70 The College is, in the view of the audit team, understandably proud of the potential it offers for students entering at one level to progress to higher level courses. In particular, its Bachelor of Technology degrees have been specifically designed to facilitate wider access and allow full articulation between HNC, HND and degree courses. Several members of staff told the team of students who had taken this route and had ultimately obtained good honours degree classifications. The team also met students who had progressed from the second year of the HND to the third year of a degree course, and who commented on the difficulty of moving from the structure of continuous and competence-based assessment on the HNC course, to the more traditional forms of course work and assessment associated with degree courses. Some described the change as a 'shock', but there were also positive comments m relation to those courses which had mediated the transition with a week long induction to the third year of the degree. It appeared to the team that teaching staff recognised the need to help students in adjusting to different teaching and assessment regimes and that the induction week, while not yet considered necessary by all courses, was becoming a welcome and more pervasive feature of the College's provision (see above, paragraph 46).

External examiners

71 Each validating university specifies its own arrangements for the selection, appointment, and briefing of external examiners, and for the manner in which examiners' reports are to be considered. However, course tutors whom the audit team met maintained that, within the constraints of the formal procedures of a particular university, they had a large amount of discretion in selecting external examiners and in dealing with examiners' recommendations. The latter was the responsibility, within the College, of individual course tutors, though the team was told that external examiners' reports would form part of the documentation produced for the new annual scrutiny process (see above, paragraph 34). While the team was reassured that individual course tutors would act upon the recommendations of external examiners and that the university systems were also likely to ensure that appropriate action had been taken, it was less clear about the ways in which the College was able to detect, extract and disseminate any aspects of external examiners' reports which had institution-wide application. The team noted, for example, that the College did not produce its own digest of external examiners' recommendations on all its courses, but was told that all external examiner reports were read by the chairs of the EQSC and of the Academic Board, and that any relevant issues would be raised in the appropriate forum or would be dealt with through informal networks. The team took the view that this was an example of how the efficiency of the quality assurance and enhancement processes depended on the judgement and performance of one or two individuals rather than the formal procedures of an annual monitoring and reporting exercise.

FEEDBACK AND ENHANCEMENT PROCESSES

Student liaison groups and the Students' Representative Council

72 The audit team learnt from the documentation that the SAC regards feedback and responsiveness as key principles in developing its quality assurance system. Although the relatively intimate nature of the College community means that there is a considerable use made of direct communication with tutors, there is also a range of other feedback mechanisms. In addition to informal contacts, courses have a forum for the presentation of student views in student liaison groups. Each course has a student liaison group which includes one student representative from each year of the course in addition to the core team. The minutes of the meetings of the student liaison groups feed directly into the core team and are also forwarded to the EQSC. Students whom the team met commented favourably on the way the student liaison groups worked, and argued, with some evidence, that they provided a useful feedback into course provision and management and that they could produce desired changes.

73 At the Auchincruive centre of study there is a Students' Representative Council whose Executive and President have regular meetings with members of the College's senior management. However, although the Students' Representative Council at Auchincruive does pick up some welfare and academic issues, at the moment it is mostly concerned with social and recreational activities, and is perceived largely in this role by the students. Discussion between the audit team and the Students' Representative Council members revealed that they would like to extend their general welfare and academic related roles in the College. The team noted in this context that they had begun involvement with the National Union of Students, which offered them training in a number of areas such as matters of welfare and student representation. At the Aberdeen and Edinburgh centres of study, SAC students do not have their own students' union, but do have access to the universities' student organisations.

74 It appeared to the audit team that, since student liaison groups operate at course level, and the Students' Representative Council is limited to the Auchincruive centre and has a largely non-academic role, one consequence is that students are not really academically represented at other levels in the College system. It also appeared to the team that a further consequence is that there is no effective student overview of issues, either within a centre of study, or across the whole College. Absence of the latter would in turn tend to create and reinforce the local, rather than College, identity reported by most students to the team. The College may wish to consider the fuller involvement of students on a College-wide basis.

Student surveys

75 The College has introduced the systematic use of student evaluation surveys for new students, who are surveyed in the middle of term one; for all students at the end of each academic year; and for each unit. Students confirmed the use of these questionnaires, and the audit team also received examples, both of the questionnaires used, and of the processed data. In the view of the team, the College is to be complimented upon its attempts to gather student evaluation systematically at different stages in the learning process. However, examination of the documentation provided by the College and discussions with members of staff raised a number of concerns for the team about the effectiveness of this feedback system. It was not clear to the team that the questionnaires were carefully enough designed to elicit satisfactory and appropriate information. For example, despite an EQSC scrutiny panel report (1992-93) raising questions about whether students were adequately consulted about the quality of courses, an item in previous editions of the questionnaire about quality of teaching no longer appeared in feedback questionnaires in use at the time of the audit visit, although there was an opportunity for open-ended comment. It also emerged, in meetings the team had with staff, that some larger courses produced their own questionnaires and data analysis, while other courses relied more on informally gathered information. This suggested to the team that serious limitations were perceived in the value and use of the College-wide system. There appeared to the team also to have been problems in responsiveness to feedback. In one example submitted to the team, of a review of a course following poor student feedback, it was noted that some time had elapsed before corrective action had been taken. In another instance, a scrutiny panel reported that student dissatisfaction was not always being picked up and represented in course reviews. The lack of a procedure for monitoring the performance of the formal data gathering system, together with heavy reliance in practice on informal contacts for feedback information, may well, in the view of the team, have obscured the shortcomings of the formal system. The College may wish to review the formal feedback mechanisms to ensure that they are effective in practice and usefully complement other sources of information.

External involvement

76 The College has developed strong vocational and industrial links as a result of its educational, research and consultancy work. The nature of its activities makes it particularly sensitive to the need to remain responsive to the employment needs of its graduates. New course developments and course revisions are informed by market research. Centre based industry/career liaison groups co-ordinate these industrial and employer links. These groups meet quarterly and report via the Senior Tutor to the Centre Education Board. The membership includes the Senior Tutor, the industry/careers representative of each course team, and the centre education liaison officer. In addition, each course management team has an industrial liaison group which includes at least two industrial representatives. Industrial liaison groups meet annually to review the extent to which courses are meeting industry needs, and their report is included in the course review process.

77 It was clear to the audit team, both from documentation and from discussion with College staff, that courses benefited from external input in a number of ways which impacted on courses, from recruitment, through course delivery, to the quality of student output. The team heard that industry representatives took their College roles seriously and actively.

78 In the view of the audit team, the considerable involvement of College staff in research and consultancy, and the way this informs teaching, was a further means of bringing outside influences to bear on educational activity with considerable benefit to students. Together with external contributions to course development and review, as indicated above, it seemed to the team that the College was making effective use of its extensive network of outside contacts, and this was to be commended.

STAFF APPOINTMENT, DEVELOPMENT, PROMOTION AND REWARD

79 The audit team learnt that the College has been undergoing a prolonged period of change and development, and was seeking 'to develop a culture in which flexibility in the face of change is a core value to be embodied in personal training and development'. At the time of the audit visit, the process of change was extending to the staffing structure itself. The staff of the College are at present employed on terms and conditions of service analogous to those of the Civil Service, but the Memorandum of Understanding between the SOAEFD and the College allows for the College to take on full responsibility for all matters relating to the pay and grading of staff. The team was informed that the College intended to make this change during the financial year beginning April 1996. Given the extent of change in the College, the team was impressed by the buoyancy and enthusiasm of staff.

Arrangements for staff appointment, induction and probation

80 There are standard College procedures and processes laid down for selection and appointment to lecturer posts. These include a defined composition for interview boards. A programme of staff training in interviewing skills for potential board members was under way in the College at the time of the audit visit. In its discussions with recently appointed lecturers, the audit team was made aware of differences between departments in the arrangements for selection of staff. In some cases, applicants had been asked to make a presentation before their formal interview, while in others no presentation had been required. Once appointed, new members of staff are subject to a two-year probationary period, which is monitored and reported on annually by the relevant head of department. None of the staff members whom the team met appeared to be finding any difficulty with the probationary process, and all were confident that any problems which might threaten confirmation of probation would be identified and discussed with their line managers as they arose.

81 Newly appointed teaching staff are all invited to attend a cross-College teacher induction conference. Staff whom the audit team met reported that this had been a valuable introduction to the College and to their roles and, in addition, provided a valuable a forum for meeting staff from other departments and centres of study. The team noted the constructive way in which the College formally and systematically evaluated its induction conference and acted on the feedback from participants. The conference is followed by a series of seminars and workshops, under the 'Probationary Staff Development Programme', which is offered at each of the three centres of study. Some staff also reported to the team that, on appointment, they had each been allocated to an experienced member of staff from another department as their mentor. The team heard that the mentoring arrangement, which was reported as being very helpful, was to be extended to staff changing roles within the College. For example, the team met a recently appointed course tutor who was being mentored in that role by the Academic Registrar. The team was also informed that staff undertaking their first experience of supervising a research degree student would have a second, experienced supervisor as a mentor (see above, paragraph 53).

82 The audit team noted, from its discussion with staff, the generally high level of awareness and satisfaction of new members of staff with the processes of their appointment, induction, mentoring, initial training and ongoing development, and of the monitoring of their performance during the probationary period. The team would wish to commend the College on these arrangements.

Staff development policy and its implementation

83 The College indicates, in its Strategic Plan 1995-99, that it seeks to 'encourage achievement and provide an environment for the continued professional development of staff'. The Strategic Plan indicates that, while there has been significant staff development activity in the past, there is a need for this to be increasingly focused on 'the needs of the organisation and of individual members of staff'. The staff development strategy includes a compulsory induction course for all new staff; individual training and development programmes for all staff; training for new entrants to teaching, both initial and extended; and a commitment to the Investors in People programme.

84 Parts of the plan, including the compulsory induction and training for all new teachers in their first year, had been implemented at the time of the audit visit. The audit team was told that plans for introducing the extended training, with certification for all teachers after two years of service, would be subject to slippage from the planning date of April 1997 because of continuing discussion on the most effective way to deliver the training. The implementation of the structured programme of training and career development through a Personal Training and Development Plan, would depend on the implementation of the new staffing structure (see above, paragraph 79). The team was informed that final consideration of the College training strategy would be influenced by the decision on whether or not the College should seek Investors in People status, and that this in turn awaited, at the time of the audit visit, the results of a staff survey. While the College's reluctance to commit itself to a definite policy statement in advance of a decision on Investors in People is understandable, the team noted the College's obvious commitment to the full development of its staff. Given that so much activity is already under way, the College may wish to consider the desirability of progressing the development of an institution-wide policy for staff development in line with identified institutional needs as soon as is practicable.

85 While the Staff Training Group has ultimate responsibility for identifying institutional training needs, the audit team was told that the main impetus in identifying and meeting the development requirements of teaching staff came from the Senior Tutors Group. This group may have issues referred to it by the Staff Training Group, or it may take on matters identified at the individual centre education boards, which senior tutors chair, or arising from the teacher liaison groups. The team was aware, both from documentation and discussion with staff, that the College provides a wide range of opportunities for staff to undertake training and development for their roles as teachers and personal tutors. Particularly noteworthy, in the view of the team, was the annual education conference which brings together teaching staff from the three centres of study, and which was referred to warmly by many members of staff in discussion with the team. From the documentation, the 1995 Education Conference appeared to the team to have been a wide-ranging and lively mixture of sessions involving a wide range of College staff. The team wishes to commend the College on the annual education conference and its part in developing teaching and counselling roles, disseminating good practice, and fostering communication between staff from different centres of study, and between teachers and managers.

86 The audit team was informed of the wide range of training and development opportunities that were available for all College staff. For example, the team met several members of the technical and support staff who had obtained degrees with the College's support, and one who had undertaken the induction and training programme for teaching staff. The team wishes to commend the College on its breadth of provision of staff development opportunities, particularly in relation to technical and support staff.

87 The College's commitment to developing its staff in support of their teaching function was also evident in its programme of training for the supervisors of postgraduate research students (see above, paragraph 53). Although none of the supervisors whom the team met had undergone the training, the team was assured that this was because the programme had initially been targeted at more junior staff, and that all but a few supervisors would have completed the training by the end of 1996. The team was also aware of the mentoring structure which supports staff new to the role of supervising postgraduate research students. The team wishes to commend the College's commitment to training its research degree supervisors and, in particular, for extending that training to potential supervisors.

Staff reporting and appraisal procedures

88 The College has an Annual Staff Reporting (ASR) procedure in place which is designed to evaluate the performance, and determine the objectives, of each member of staff. The ASR performance criteria are also used in determining awards of performance related pay. The scheme is currently being reviewed as the College prepares to take on responsibility for the pay and grading structures for its staff (see above, paragraph 79). The current arrangements involve completion of a Staff Report, which comprises a job description written by the individual member of staff, and a performance assessment made by a reporting officer (usually the individual's immediate line manager), and endorsed by a higher grade countersigning officer. At least once every three years, and more often on request, the member of staff is given an appraisal interview, normally by the countersigning officer. Staff acting as appraisers are expected to undertake appropriate training. All appraisal forms are held by the Personnel Department which is expected to identify generic training needs from the completed forms. The audit team asked various members of staff about their experience of the ASR, and all confirmed that they were subject to the annual process. No staff identified any problems with the process; and the team heard some reports of how the procedure allowed workload problems or development needs to be effectively raised and addressed. The team wishes to commend the College on its application, on a cross-College basis, of its staff reporting and appraisal procedures and its commitment to develop that system in line with its new responsibility for the pay and grading scales for its staff.

Policy, procedures and criteria for staff promotion

89 Promotion appraisal is an optional part of the ASR procedure but, in recent years, career progression at the College has depended almost exclusively on applying for more senior posts which are, normally, also open to external applicants. The audit team was informed that, prior to the audit visit, a round of applications for 'merit promotions' had been invited, with recommendations sought from heads of department. This had led to three promotions which, the team was told on more than one occasion, had all been awarded to research staff. While the team would find it difficult to draw any general conclusions from such a small sample, it was evident that the absence of teachers from those gaining merit promotion was considered to be significant by some staff. The team was informed that the College was aware of the need to develop a more effective framework for appraising the performance of teaching staff, and had established a Teacher Development and Appraisal Group (TDAG) which, at the time of the audit visit, was undertaking this task as part of the review of the ASR procedures. The team formed the view that this development potentially offered the College a means for developing explicit criteria for merit promotion which recognised merit in teaching, as well as in research and consultancy.

CONTENT OF PROMOTIONAL MATERIAL RELATING TO ACADEMIC PROVISION

90 The SAC produces a range of promotional material, including an undergraduate and postgraduate prospectus; general and individual course leaflets; an educational video; and various posters and other promotional material. Discussions between the audit team and course tutors confirmed that they produced and checked course material for publications. Some monitoring role was also undertaken by the Senior Tutors Group. Material in relation to postgraduate provision was viewed by the Chair of the Research Degrees Committee as well as by the Systems Manager (Education). Final responsibility, at College level, for promotional material rests with the Systems Manager (Education).

91 The audit team formed the view that these mechanisms appeared to work effectively, and resulted in the production of accurate materials.

CONCLUSIONS AND POINTS FOR FURTHER CONSIDERATION

92 The Scottish Agricultural College has undergone very substantial change as its constituent parts have come together into a single unit. The audit team formed the view that, in general, change processes had been effectively managed, and that staff were well informed, committed, and enthusiastic. The academic staff with whom the team met clearly viewed the SAC as a single, multi-site organisation. It appeared to the team that this feeling of unity was strengthened by the various committees and initiatives, formal and informal, which involved cross-site meetings and discussion, and the consequent spread of good practice. It also appeared to the team, however, that the students tended to see themselves rather more in terms of the particular site on which they were based.

93 A central issue in the College appeared to the audit team to derive from the way that formal and informal quality assurance systems related to one another in practice. Informal communication among staff appeared to the team to be good, with senior management regarded as approachable and responsive. In the past, the College size and collegial style, particularly in individual centres, was conducive to a strong and extensive informal system operating on the basis of personal contact. At the time of the audit visit, this system still appeared to the team to be largely in operation and to work quite effectively. However, at least partly in response to perceived external pressures, the College has recently developed an elaborate set of formal procedures, which did not appear to the team to be fully integrated yet with existing modes of operation.

94 The development of the formal system appeared to the audit team to have been largely driven from the top down, with the result that knowledge and ownership of procedures at other levels in the College appeared to the team to be variable. A good example of this appeared to the team to be the process of annual course review and the transition to an internal audit process.

95 Another, and it appeared to the audit team, more general problem, has been that some staff seemed to rely on informal contact and processes to deal with issues. This is well exemplified, in the view of the team, in the annual monitoring process, where large amounts of data are routinely produced and distributed without obvious attempts to check on whether they are useful or usable. For instance, student feedback questionnaires are formally an important part of the course review process, but the one in use at the time of the audit visit seemed to the team to provide course teams with rather limited information. This situation has resulted in some course teams devising and using their own questionnaires rather than trying to change the formal, College-wide, questionnaire which had been in use for three years. This exemplified to the team the potential problem that, despite the existence of a formal system, many staff rely in practice on informal contact with others and parallel, informal, approaches to do much of the work of a quality assurance system. Whilst this in itself is not necessarily a shortcoming, one consequence has been that the formal system is not, in practice, fully utilised and, partly because of this, has not been reviewed and developed as purposefully and effectively as it might. In general, in the view of the team, it might be constructive to consider how the formal and informal systems might more effectively be co-ordinated.

96 It appeared to the audit team that one aspect of the formal structures which contributed to the above situation, was the fact that they seemed to depend heavily on a small number of individuals and, in particular, the Systems Manager (Education). In the view of the team, this might potentially give rise to limited feelings of ownership of the formal structures. As the College continues to develop its systems and arrangements for quality assurance, it may wish to consider how more pervasive feelings of ownership might be developed.

97 The audit team wishes to commend the College in particular for:

(i) the active role of the educational liaison officers in providing a bridging link between the community and the College (paragraph 44);

(ii) the role of the teacher liaison groups in providing a constructive means of bridging the gap between formal and informal systems (paragraph 48);

(iii) the enthusiasm, expertise and accessibility of its staff, as perceived by students (paragraph 52);

(iv) the development of an active and effective Research Degrees Committee and its commitment to training research degree supervisors and potential supervisors (paragraphs 53, 56 and 87);

(v) its evident care in ensuring that arrangements and criteria for assessments and award classifications are effectively communicated to students (paragraph 65);

(vi) the extensive use made of strong industry links to enhance its educational provision (paragraphs 76 to 78);

(vii) its arrangements for the appointment, induction, and probation of new members of teaching staff, including the establishment and development of its mentoring scheme (paragraphs 80 to 82);

(viii) its annual education conference which facilitates the development of teaching and counselling roles, disseminates good practice, and fosters communication between staff from different centres, and between teachers and managers (paragraph 85);

(ix) its provision of training and development opportunities for all categories of staff (paragraph 86);

(x) its application, on a cross-institutional basis, of its current staff reporting and appraisal system, and its commitment to develop that system in line with its new responsibility for the pay and grading scales of its staff (paragraph 88).

98 As it continues to develop its quality assurance arrangements, the College may wish to consider the necessity of:

(i) reviewing its formal quality assurance mechanisms and their operation to ensure that they most effectively integrate with, and complement, those informal systems based on direct personal contact (paragraphs 35, 71, 75, 93, 94);

to consider the advisability of:

(ii) developing means of assuring itself that students' marked assessments are returned in time for them to gain learning benefits appropriate to the stage of the course and the timing of subsequent assessments (paragraph 67);

(iii) extending student representation in the College system beyond the course committee level and, more generally, encouraging cross-centre student contact as a means of fostering among students the view of SAC as a single and cohesive entity (paragraph 74);

(iv) reviewing and revising its formal methods for obtaining student feedback, and the way in which the data so generated is used for enhancement purposes (paragraph 75);

and to consider the desirability of:

(v) making an appropriate induction programme available and mandatory for all students embarking on courses, or stages of courses, which employ unfamiliar teaching and learning strategies (paragraphs 46 and 70);

(vi) ensuring comparability of similar courses where they are offered across different centres of study (paragraph 68);

(vii) the development and publication of the revised staff development policy as soon as is practicable (paragraph 84);

(viii) developing and communicating the criteria for merit promotion in such a way as to demonstrate the value of teaching as well as research and consultancy (paragraph 89).

APPENDIX

SAC

ORGANISATION AND MANAGEMENT ARRANGEMENTS

Constitution

SAC is a private company limited by guarantee without share capital and having charitable status. The company was created as a Scottish Central Institution by Statutory Instrument under the Central Institutions (Recognition) (Scotland) 1990 Act on 26 July 1990 following the adoption by the Secretary of State of recommendations made by the Committee of enquiry into the future of the Scottish Agricultural Colleges (the Williams Committee).

The Board of Directors

The SAC Group Board of Directors comprising twelve non-executive and two executive directors is responsible for setting major policy and strategy and for monitoring the performance of the company. The Board has established the SAC Council of some 41 members to provide advice drawn from a range of sectors of interest and local communities throughout Scotland.

Executive Management

The executive management of the SAC group of companies is the responsibility of the SAC Executive Board which comprises the Principal and Chief Executive (chairman), Executive Director (managing director of COSAR, SACs commercial company), the Directors of the Advisory and the Veterinary Services, the company Secretary and Treasurer and three Deputy Principals (who each have dual responsibilities, as executive managers of the SAC principal functions of Consultancy, Education and Research, and as Deans of the Academic Centres of Study at Aberdeen, Ayr and Edinburgh),

Administration

The company Secretary and Treasurer heads an SAC Administration Division which is responsible for all aspects of financial, legal and personnel management and which operates through departments located at each of the three Centres of Study and within the SAC Central Office at Edinburgh. These departments provide local administrative support to executive managers and, in addition, SAC-wide support services.

Functional Management

The Executive Board has established sub-committees to support the Executive in the management of its principal functions: the Academic Board, the Research and Development Management Group and the Advisory and Consultancy Committee, each chaired by the appropriate member of the Executive Board.

Technical Commication and Advice

The Executive Board have established 6 Schools of Study (Animal Sciences, Food Science, Plant and Environmental Science, Building Design, Engineering and Mechanisation and Economics and Resource Management) to provide fora for communication between departments with technical interests in common and to provide advice to the executive and their committees.

The Academic Board

The Academic Board advises the Executive Board on strategy and policy for the education and training function and is responsible for setting targets and monitoring performance. It comprises the Deputy Principal (Education) (chairman), the Systems Manager(Education) and three Vice-Deans (Education) who are the senior education managers at the Centres of Study.

Academic Board Sub-committees

The Academic Board has established sub-committees to take responsibility for: education standards (Education Quality Standards Committee), Centre of Study strategic developments and resources (Centre Academic Planning Groups), education delivery and related policy developments (Centre Education Boards) postgraduate research student affairs (Research Degrees Committee) education support services (Senior Tutors Group) and training support services (Training Services Committee).

Education and Training Management

The Deputy Principal (Education) has executive responsibility for education and training within SAC and is assisted by the Systems Manager (Education), and three Vice Deans (Education), responsible for local education management and strategic development at each Centre of Study.

Quality Standards and Education and Training Support

The Systems Manager (Education) exercises overall responsibility for quality standards through chairmanship of the Education Quality Standards Committee and heads the Education and Training Support Unit which supplies student support services including recruitment, registration, recreational provision, careers advice, counselling and welfare and also provides for the staff development programme specifically related to education.

May 1996

Top