INTRODUCTION

1 This is a report of an audit, carried out by the Higher Education Quality Council (HEQC), of the quality assurance arrangements for a collaborative partnership between the University of Sheffield and Taylor's College (hereafter referred to as 'the College'), leading to first-year courses in Law and in Management which enable students to enter the second year of the appropriate degree courses at the University in Sheffield. It forms part of a series of pilot audits of overseas collaborative partnerships undertaken in 1996. The audit included a visit to the College in May 1996. Further information about these audits is contained on the inside cover of this report.

2 An audit visit to the University of Sheffield took place in June 1991. The present report is the first in which the University's partnership arrangements have been the subject of an HEQC quality audit, since there has not been an audit of the University's general collaborative provision.

3 The Council is grateful to the University of Sheffield and the College for the willing co­operation provided to the members of the audit team.

THE AUDIT PROCESS

4 Prior to the audit team's visit to Malaysia, the University provided information relating to all phases of its relationship with the College and agreed a programme of meetings with the team. The team which visited the College comprised Professor D J Murray, Professor J Rear and Professor A J Reynolds, auditors, and Dr D S Law and Ms C Webb for HEQC.

5 The audit team met the Principal, senior officers of the College, all full-time staff who teach on the programme, two part-time staff with experience of the programme, several graduates who began their degree programmes at the College and concluded their studies at the University, a number of students currently registered on the programme in Malaysia, and a representative of the University. On its return to the UK, members of the team met senior staff from the University to verify a number of matters.

6 The representative of the University whom the audit team met was a senior administrative officer who had recently assumed the post of the University's Head of Administration, South East Asia. He told the team that part of his role was to facilitate wider collaboration with the MBf Education Group, owners of the College.

BACKGROUND TO THE PARTNERSHIP

7 At the time of the HEQC 1991 audit of the University's quality assurance arrangements, the franchise relationship with the College had not been established. It was the subject of an agreement for the Licensing of Courses dated November 1992.

8 Other UK universities (at present Bradford and East Anglia) have agreed to take students from the two programmes which the University of Sheffield has established and monitored; the three universities act as a 'consortium'. Sheffield consulted its UK partners during the establishment of the link, and the audit team learnt that some members of their staff have visited the College.

9 The University's briefing documentation describes the relationship as a 'franchise'. The

1992 agreement, in accordance with preferred usage in Malaysia, employs the term 'licence' which it has granted to the College to teach first-year courses in Law and in Management. The programme in Law provides entry into the second year of the University's LLB or BA in Law, and differs from that provided in Sheffield only in an interchange of a first-year and a second-year element. The programme in Management can provide admission to the second year of several BA programmes, and the pattern of studies is identical to that provided at Sheffield. The agreement specified the staffing required for both programmes, described the expected modes of delivery, and outlined the library and computing resources which the University deemed to be necessary. Students enrolled at the College are not registered as students of the University; no award, save for a certificate of attendance, is available to those who complete the programme at the College but do not proceed to a university in the UK.

Initial contacts

10 Although the briefing documentation did not describe the origins of the relationship, the audit team learnt from staff of the College that the College had sought the support of the University in setting up twinning programmes, having first identified it as a suitable partner. The College was also evaluated by the University as a suitable partner.

Evolution of the relationship

11 The first students were admitted to the College's programmes in October 1992; virtually all of them transferred to the University in 1993, and the first cohorts of students enrolled in this way graduated in 1995. In preliminary discussions with the College, the University indicated that it did not feel able to admit more than 25 in each of Law and Management, but the definitive agreement provided for recruitment of up to 110 students to each stream. These numbers were necessary for viability for the College and led to the need to find acceptable partners to which successful students could transfer.

12 In 1993 the University set up an Office of Collegiate Studies and established a Board for Collegiate Studies (BCS), in order to achieve standardised practices in dealing with validation and moderation. All the University's franchised courses pre-date the establishment of the Board of Collegiate Studies, and were set up on the basis of individual agreements. The University document recommending the creation of the Board states that 'things were slightly easier for franchising', in respect of which departments were thought to have been very careful in their approval and quality-control processes. Consistent with this view of the University's arrangements for franchised courses, The Collegiate Studies Handbook 1995-96 states that it was written particularly with validated rather than franchised courses in mind. Subsequent to its visit to Malaysia, the audit team was informed by the University that consideration was being given to drawing up a revised statement on franchises, setting out procedures and expectations of both partners. The team had concluded that this would be desirable and noted that the University had also concluded that guidelines would be helpful, both for use within the University and with partner institutions.

Partnership objectives

13 The Collegiate Studies Handbook does not state the University's motivation in establishing overseas links, but states that the Corporate Plan of 1993 committed the University to continuing expansion of off-campus delivery, primarily through validation, and to ensuring an equivalent experience for off-campus students. The University's representative in Malaysia told the audit team that the University had a particular commitment to developing collaborative links with that country. The prospectus of the College states that its aim is 'to open the doors of renowned universities all over the world to students from Malaysia and the region'.

Taylor's College

14 Taylor's College, founded in 1969, has at present over 3,500 students. Initially it prepared students for entry into Australian universities, but it now also prepares students for entry into UK and Canadian universities. Its collaborations with the University of Sheffield and its other UK partners are described by the College as twinning programmes, in keeping with usage common in Malaysia. The MBf Education Group, of which the College is a member, operates several other private educational establishments in Malaysia.

THE UNIVERSITY'S QUALITY ASSURANCE ARRANGEMENTS

15 The 1992 agreement made provision for monitoring of the programme and the experience of students through two visits each year by staff from the University's Faculty of Law and School of Management. Reports seen by the audit team showed that this pattern of visits had been maintained until the 1995-96 session, and had included participation in examiners' meetings held at the College. The team was informed that, when appropriately qualified Sheffield staff had not visited during a particular semester, scripts marked by College staff had been sent to Sheffield for second marking.

16 The University's Board of Collegiate Studies, set up since the link with the College was established, has responsibility for all franchising and validation and for the operation of related quality-control procedures. It has the power to approve the composition of boards of examiners, and to recommend to the Senate the appointment of examiners and moderators and the recognition of staff of partner institutions. In the papers provided by the University the audit team was not able to identify precisely how these powers had been used in relation to the College. However, visiting Sheffield staff were now referred to as 'moderators', in keeping with the terminology used by the Board and the University informed the team that all College staff had been approved by the moderators and by the University through the Board. The team welcomed the introduction of the Board of Collegiate Studies as a means of co-ordinating the University's arrangements for its collaborative teaching provision, and in the expectation that its processes would apply routinely to franchised programmes.

17 University staff who visit the College bear the major responsibility for supporting the College in the maintenance and enhancement of quality. In addition to monitoring the development of resources supporting teaching, they have observed the teaching of College staff and have gathered the views of the College's staff and students. Although meetings with staff and students have sometimes been described as informal, they have given rise to written reports to the University and to specific recommendations to College managers. The audit team wishes to commend the continued attention given by the University to the enhancement of the quality of the teaching provision at the College.

18 The University's papers state that by 1995 it perceived the relationship to be 'mature' and that it had confidence in the marking standards applied by College staff. This has given rise to a trial of new visiting arrangements during the 1995-96 session, in which members of the University would make a single ten-day visit, instead of the two envisaged in the 1992 agreement. During this longer visit they would not only observe teaching and meet College staff and students, as before, but would have additional opportunities to participate in teaching.

19 Members of the College's teaching staff told the audit team that they had taken an active part in discussions leading to the longer visits by Sheffield staff, and that five of the University's teaching staff had stayed for the longer period during the 1995-96 session. The new arrangement was thought to be beneficial by allowing students to experience significantly more teaching by University staff, and thus to gain a better appreciation of the styles of teaching they would encounter on transferring to a UK university. The team was told that the extended stay also allowed University staff to acquaint students with practical features of the transfer, including the choice of options and arrangements for finding accommodation, and allowed them to meet prospective students and their parents. It appeared to the team that the new arrangements were in effect a shift from the University's prior emphasis on routine monitoring of established processes to a more developmental role in which visiting staff were more closely involved in the programme and with students. This refocusing of the University's contribution seemed to the team to have benefits both for current students and for the evolving institutional partnership.

THE ESTABLISHMENT OF THE LINK AND INITIAL APPROVAL PROCESSES

20 The 1992 agreement specified the structure of the first-year courses to be provided at the College and indicated the staff and resources necessary to teach to the required standards. It also set out the arrangements for the transfer of students to the UK, the nature of the University's involvement in the appointment of the teaching staff, and requirements for their training and induction at the University. The agreement was drawn up in the light of a four-day academic inspection of the College in July 1992, by seven members of the University's staff. The detailed nature of the agreement and the report of the visit demonstrated to the audit team that the University had carefully considered both the processes necessary in establishing the link and the subsequent distribution of responsibilities between the partners.

21 Annexed to the agreement were extracts from the University's Calendar setting out general regulations governing examinations and certain regulations specific to the faculties of Law and Social Sciences.

22 In the process of establishing the link, the University offered to the College two 'consultancies', one covering assistance to the College during the launch period, the other offering comprehensive support in developing library facilities. The audit team was informed that the first of the proposed consultancies was taken up and given effect through visits by University staff to the College, during which they participated in the selection of teaching staff to provide the planned programme, and by visits to the University by the newly appointed College staff. The team commends the University for the support it provided to the College in the recruitment and training of staff who were to teach the courses defined by the University.

STAFFING AND STAFF DEVELOPMENT

Recognition

23 In discussion with both full-time and part-time teaching staff at the College, the audit team learnt that in all cases the University, including staff in Law and Management, had been consulted and given approval prior to the confirmation of their appointment. Subsequent to its visit to Malaysia, the team was told by the University that there had been no changes to the full-time staff teaching on the programmes.

Staff development

24 As envisaged in the agreement, the full-time staff initially recruited by the College to provide the franchised programmes did participate in special induction programmes at the University, extending over nearly three weeks. Since then, visiting Sheffield staff have collaborated with College staff in assessment and, more recently, in teaching at the College. In 1995 the Management School provided a four-day programme at the University for two further members of College staff. The audit team was informed by the representative of the University in Malaysia that a three-week period of familiarisation was no longer felt to be necessary as the programme had become established and the staff had gained in experience.

25 The audit team saw reports by visiting University staff in both Management and Law, and noted that they had given considerable attention to issues of staff development, including comment on teaching sessions provided by College staff. In particular, the team learnt that a University moderator had observed a tutorial class given by a part-time member of teaching staff, albeit without making subsequent comment. From the reports of visiting staff it was evident to the team that they had sought improvements in means of communication between the staff of the two institutions, and the team was informed by College teaching staff that they were able to contact directly their opposite numbers in the University staff, who were known to them through the two-way visiting process. Moderators' reports showed that visitors from Sheffield had encouraged the College to provide time for its staff to carry out some research, and had given thought to the effects of proposed changes in time-tabling arrangements. The audit team was pleased to note these consistent and well conceived efforts by University staff to enhance the experience of those providing parts of its programmes in another country.

STUDENT ASSESSMENT

Setting of examinations and other assessments

26 Under the agreement, the University has the right to set examinations and to approve papers set by College staff. Some examinations were initially set in Sheffield, but the audit team learnt that all are now prepared by College staff. The team noted that this is consistent with, but a significant extension of, the University's earlier reported statement that it is now confident of the marking standards of College staff. College staff told the team that University staff to whom draft papers were sent made detailed comments on every paper, and that University staff had discretion in deciding whether to accept the suggested changes.

27 The audit team was told that the University had provided general guidance to College
staff on their marking of coursework and that, when visiting the College, Sheffield staff
commented on coursework assignments and their marking. The team concluded that these arrangements did offer the University sufficient insight into the practice and standard of coursework assessment on the programmes that lead to admission to its degree course.

Administration of examinations

28 Consistent with the annexing of examination regulations to the 1992 agreement, the audit team was informed that in most respects the arrangements for examinations at the College follow the University's regulations, although this was not a requirement of the 1992 agreement.

Marking of assessments

29 Examinations are now marked by College staff, with samples of scripts sent to Sheffield for second marking, accompanied by marking schemes prepared by College staff. The audit team was informed that all failed scripts were subject to second marking and that the University relied upon the knowledge and experience of its moderators to select other scripts for second marking. It seemed to the team that adjustments in marks could be made following the second marking, but only for scripts in the sample selected. The team suggests that, if this is the case, the University may wish to consider whether its procedures provide for circumstances in which moderators might need to look at the marks on all scripts if the marking was judged overall not to be consistent with University standards.

External examiners

30 Since no University award is made in recognition of success at this level of the programme, examiners external to the University are not called upon to participate in assessment.

Examiners' meetings and decisions

31 The audit team was shown the minutes of what were termed examiners' boards held at the College, with Sheffield staff in attendance, and in this context referred to as 'external examiners'. The status of these boards was not clear to the team, since the students whose results were considered were not at that time working towards a University award. Under the new arrangements which involve a single annual visit, such boards are no longer held at the College. Instead, individual examiners send marks and sample scripts direct to Sheffield, where they are considered by the Board of Collegiate Studies. The team learnt that results of College examinations are not announced until they have been ratified by a Sheffield board. This ratification has the effect of admitting successful students to the second year of the University's degree programmes in Law and Management. The team suggests that, when its agreement with the College is revised, the University may see benefit in specifying more explicitly the roles of its staff and examining boards in relation to assessments of students' success in the programmes it has franchised.

ACADEMIC STANDARDS

32 Even though students enrolled at the College are not registered with the University, the terms of the 1992 agreement give the University the right to 'approve' students and to 'formally admit' them to the courses. The audit team was told that most of the students
admitted by the College satisfied the requirements set out clearly within the agreement, in respect of both subjects and grades. Information on candidates with different qualifications was routinely referred to the University, and some were not approved. The team noted that this thorough specification of entry conditions supplemented the University's ability to determine the standards of advanced-standing entry to its programmes which is provided by moderation of the College's examination papers and sampled examination scripts.

MONITORING ARRANGEMENTS AND THE STUDENT EXPERIENCE

Feedback from students

33 Visiting staff from Sheffield have pointed out to College staff that good practice requires student appraisal of each module, and have provided copies of forms used at Sheffield for this purpose. The team was informed that the questionnaire used within the College to obtain feedback on individual modules was a modified version of that used by the University, and students reported making use of such forms to express their views on the teaching of individual modules. The team noted that a staff/student committee was instituted at the College in response to suggestions from visiting staff from the University.

It was evident from moderators' reports seen by the audit team that Sheffield staff continue to take an interest in these aspects of student experience, and the team commends their consistent attention to issues relating to student feedback.

Annual review

34 This is achieved through the reports of visiting Sheffield staff which are used to compile the overall report, following consultation with College staff. The University may wish to consider ways in which College staff might be more directly involved in the process of producing the written annual review, as a means of encouraging all those associated with the programmes, within both College and University, to assemble experience of the operation of the programmes and ideas for improvement. The University and its prospective students may also benefit from a process that would engage the College's teaching staff in a collaborative team effort, involving also some of the University's own staff.

35 In the early years of the collaboration, the separate reports of visiting staff in Law and Management described their duties as one or both of 'external examining' and 'validation'. With the establishment of the Board of Collegiate Studies, visiting staff are now described as 'moderators'. The intended role of staff appointed as moderators is to act as objective assessors on behalf of the University, and not as representatives of particular faculties. It further provided that their reports should go direct to the Board. It appeared to the audit team that some of those who had earlier visited the College as 'external examiners' for Law and Management had simply been redesignated 'moderators'. Three moderators for Law and six for Management had now been involved in the programme at the College and the University is of the view that their contribution has ensured that fresh critical perspectives are brought to bear on the programme.

36 Variations in the structure of reports by visiting staff suggested to the audit team that until recently the University had defined only in general terms the nature of the required 'moderation' of franchised courses. Nevertheless, the reports showed that Sheffield staff, whether called external examiners or moderators, have consistently dealt with a wide variety of matters relevant to the quality of teaching provision and student experience. They have regularly commented on the general learning environment and on developments in library and if facilities, and have made clear their expectation for the continuing enhancement of these services, to match developments within the University.

37 Minutes of the BCS appeared to the audit team to show that the Board gave brief consideration to the reports of the moderators of the programmes at the College. From the University's documentation, the team could not determine how and where in the University's structures the numerous comments and recommendations contained in those reports were addressed in detail. The University informed the team that since moderators' reports are seen by College staff before they are submitted to the Board, and that the Board does not comment in detail on their content if it deems them to be a satisfactory record, there is an expectation that any issues needing to be addressed with College staff will already have been raised with them. If any matters of concern are raised in the moderators' reports, the Board would draw these to the attention of the College 'automatically'. The University may wish to consider whether the Board of Collegiate Studies' consideration of the moderators' reports might not provide an opportunity for comment on the process of critical evaluation itself, which may be of interest to the College, to the participating departments, and contribute to further improvement and the dissemination of good practice.

Periodic review

38 The 1992 agreement granted 'full licensing' to the College for an initial period of five years, but no process of review prior to extension was defined. The University's representative in Malaysia stated that the BCS had not yet reached decisions on these matters and that he expected a renewal of the licence to be associated with a redrafting of the agreement to represent more accurately the way in which the relationship has developed since 1992. The audit team welcomed these indications that the University would soon address these areas.

Provision of information to students

39 In discussions with current students and with graduates who had been admitted to the University via the franchised programmes, the audit team learnt that, while at the College, students received detailed information, emanating from the University, on the content and assessment of individual modules taught at the College. However, students at the College did not appear to receive, in any systematic way, written information on matters relating to the programme as a whole and to their progression through it. The team learnt from the College's Director of Programmes that she was, at the time of the audit visit, engaged in drawing up a Student Handbook, based on the University's regulations. She envisaged that the Handbook would contain information on such matters as plagiarism, personal behaviour and discipline, and conduct during examinations. The team was pleased to note this indication that the University had reached agreement with the College on a matter left unresolved in the 1992 agreement.

Learning facilities

40 Visiting Sheffield staff have given particular attention to library facilities at the College. While the book stock was initially thought not to be satisfactory, later reports have indicated a much higher level of satisfaction in those visiting from Sheffield. University staff have also considered the provision of computing facilities, sometimes creating detailed listings of the equipment available. The team commends the University on the continuing interest taken by its staff in these facilities and by the encouragement given to the College to enhance and update them.

Support

41 In their reports of meetings with the College's students, visiting staff have demonstrated that they have regularly sought to inform themselves about the support provided by the College staff.

Introduction to the UK

42 Moderators have reported that students at the College repeatedly raise matters relating to their transfer to the UK, and that during their visits they have given increasing attention to the practical problems experienced by transferring students. The audit team was told that the longer visit that took place just before its own visit provided additional opportunities for Sheffield staff to deal directly with student concerns; in particular, those associated with life and study in the UK.

Feedback to students

43 Although the University has not set out formal requirements or clear expectations, the reports of visiting staff show that they do monitor the ways in which College staff comment on assessed work. The audit team learnt, as had University's moderators, that students had opportunities to discuss their work, on an individual basis, with the College's teaching staff.

Publicity and promotional material

44 College staff informed the audit team that they routinely obtained prior approval, from a specified member of the University's staff, for publicity material and for the content of the College prospectus, in so far as they related to the Sheffield franchise. In respect of the approval of matter relating to other universities participating in the consortium, the team was told that the University of Sheffield did not accept responsibility for such statements, but passed them to the relevant institution for vetting. Having seen samples of publicity

material, the team observed that the wording of references to the outcomes of external quality assessments of provision offered in the United Kingdom were capable of being taken out of context. The University may wish to consider how such potential misinterpretations might be avoided in its scrutiny of information deemed appropriate and subsequently used for promotional purposes.

CONCLUSIONS AND POINTS FOR CONSIDERATION

45 Students engaged in the courses franchised to the College are not formally registered with the University. Nevertheless, the University has taken a close interest in the operation of the courses that provide admission with advanced standing to its degree programmes. Members of its staff have established effective working relations with the College's teaching staff and managers, and increasingly involve themselves in the delivery of the franchised programmes. This closer contact with the College and its students supplements the University's continuing comprehensive monitoring of the College's entry standards and assessment processes. The conduct of the relationship between the two institutions is in general consistent with the HEQC's Code of Practice. Although effective links already exist between College and University staff at several levels, the team considered that the University might fruitfully explore further modes of collaboration, for instance, through joint preparation of critical reports on the programmes offered at the College, and, where possible given the students' subsequent transfer to Sheffield-based teaching, discussion of ways in which details of the curriculum might take some account of the needs of students in Malaysia.

46 It seemed to the audit team that the effectiveness of the links between the College and the University owed much to the committed and active endeavours of particular members of staff, rather than to comprehensive procedures laid down by the University. The establishment of the Board of Studies provides a more formal and systematic framework for linking the University's quality assurance responsibilities with the monitoring and control arrangements of participating departments. Such a framework is likely to assist the continuing development of the teaching which is franchised to the College, and which provides an increasingly significant stream of entrants to degree programmes in two of the University's faculties.

47 In a number of respects, current practice has developed beyond the present agreement between the University and the College. The audit team noted several ways in which the developing partnership had assisted the College, and the University may find it beneficial to consider with the College how the present agreement might be modified in order to incorporate and fully reflect the nature of their current arrangements.