Preface
Quality Assurance of Overseas Collaborative Provision
The Quality Assurance Agency for Higher Education (QAA) was established in 1997 to promote public confidence that the quality of provision and standards of awards in higher education offered by UK institutions are being safeguarded and enhanced. As part of its activity, QAA undertakes regular academic quality audits of individual institutions to review the operation and effectiveness of arrangements for assuring quality and standards. A brief guide to the various quality assurance processes operating in UK higher education is available from QAA on request.
Quality audits also cover the arrangements which institutions use to assure the quality and standards of their awards and programmes offered in collaboration with other partners, both within and outside the UK. As part of this process, QAA audit teams visit overseas partners of UK institutions so that the same enquiries can be made about how the quality and standards of UK awards and programmes offered to students outside the UK are safeguarded, as are made of UK-based provision. This initiative is designed to help provide enhanced confidence in the work of British universities and colleges operating outside the UK.
QAA's audits of overseas partnerships use as their main reference point the Code of Practice for Overseas Collaborative Provision in Higher Education (2nd edition, 1996) which was published by the former Higher Education Quality Council (HEQC) and which QAA has endorsed for the purpose of the current programme of visits. This offers guidance on good practice, and a framework within which institutions can review and consider their current and future activities. The Code of Practice has been widely welcomed by universities and colleges. While UK institutions participating in the programme have not been narrowly 'measured' against the Code, (which is not intended to be a definitive check list), their experience of using it, and the findings from the overseas visits in general, will contribute to its revision and further development
This report is published following an overseas audit programme to Israel undertaken in the spring of 1998 which looked at collaborative arrangements between several UK institutions and their overseas partners. It should be read in conjunction with QAA's (and, where appropriate, the former HEQC's) published audit report(s) on the UK university or college concerned, details of which can be found in this report.
The UK universities and colleges which participated in this overseas audit programme did so voluntarily and with the agreement of their overseas partners. Their collaborative links cover between them a range of programmes and subjects, levels of award and different forms of institutional partnership, involving a mix of partners from small, privately funded organisations to large, publicly funded universities.
Introduction
1 This is the report of an audit carried out by the Quality Assurance
Agency for Higher Education (QAA), of the quality assurance arrangements
for a partnership between the University of East Anglia (also referred
to as 'the University' or 'UEA') and David Yellin Teachers' College (also
referred to as 'the College') for the purpose of offering a programme of
study in Israel leading to an award of the University of the MA in Applied
Research in Education.
2 The audit of this partnership was one of seven undertaken in the spring of 1998, covering a range of collaborative partnerships which UK higher education institutions have established in Israel. In every case, visits were made to both the UK and partner institutions. Further information about quality audit and QAA is contained in the preface at the front of this report.
3 The general quality assurance arrangements of the University of East Anglia were the subject of an academic quality audit undertaken by the Higher Education Quality Council (HEQC) and published in a report in 1993. A continuation audit was carried out by QAA in 1997 which included collaborative provision, and the report was published in 1998.
4 The Agency is grateful to the staff of the University and to staff and students of the College for their assistance and their co-operation with the audit team.
The audit process
5 Prior to the audit visit, the University provided QAA with briefing
documents, comprising a description of the University's procedures for
the validation and delivery of award-bearing courses overseas. The audit
team requested a number of supplementary documents concerned with the relationship,
including the Course Annual Report, external examiners' reports,
the Student Handbook, Validation Committee minutes and reports from
the Academic Adviser.
6 A member of the audit team and the QAA Assistant Director who co-ordinated the audit visited the University of East Anglia on 3 February 1998. Discussions took place between members of this team and a number of University staff, including the Pro Vice-Chancellor and Dean of the School of Education, the Chair of the Validation Committee, the University Adviser and Deputy Director of the Centre for Applied Research in Education. The University made helpful arrangements for the meetings with its partner institution and the audit team visited David Yellin Teachers' College on 26 February 1998. Discussions were held with more than a dozen people, including the Dean of the College; the external examiner for the course; the MA Course Directors, past and present; Academic Co-ordinators; teaching staff; and MA students.
7 The audit team comprised Dr S Jackson, Professor D J Murray, Professor D R Webb, auditors, and Mr D C Attwood, audit secretary. The audit was co-ordinated for QAA by Miss A R Hynes, Assistant Director in the Institutional Review Directorate, who accompanied the team to Israel.
The audit context - Israel
8 The period of preparation for the audit and subsequent visit to
Israel coincided with discussion of changes to the law governing higher
education in Israel. An amendment to the law was enacted in February 1998
and this altered the licensing and accrediting powers of the Israel Council
for Higher Education (CHE). QAA understood that the amendment made to the
law further empowered the CHE, and altered what had been an advisory relationship
with regard to some educational activities of foreign universities in Israel
into one of a regulatory authority. It was explained that these changes
were designed to address internal concerns about the activities of non-Israeli
universities which were offering higher educational provision within Israel.
The period to which this audit was addressed was prior to the implementation
of this amending legislation.
9 Prior to February 1998, institutions from outside Israel conferring degrees on the basis of study undertaken at a partner institution in Israel could, at their discretion, and in respect of certain forms of provision, seek a form of approval from the CHE. In the group of institutions considered, practice on this matter varied. Therefore, reflecting on paragraph 10 of the HEQC Code of Practice for Overseas Collaborative Provision, (2nd edition 1996), which states that 'students and applicants should be clearly informed whether or not the award for which they wish to register has been given official recognition...' the audit team based its enquiries around the position taken by each institution.
Matters given consideration in the audit
The University context
10 The University's Strategic Plan, which is reviewed and updated
annually, is its central planning document. The plan expresses the University's
commitment to a role in the East Anglian region working on a collaborative
basis with other educational institutions. In relation to collaborative
partnerships, the Strategic Plan expresses the University's commitment
to maintain best practice in validation and franchising policies and procedures.
The University's policies and procedures are described in a Validation
Handbook. The first edition of this appeared in February 1995 and a
second edition in 1997. The University's practice is stated to be informed
by the HEQC Guidelines on Quality Assurance, by the points of good
practice highlighted in the reports of audits undertaken by HEQC, and by
the Council of Validating Universities' Code of Practice. This latter Code forms
a part of the Validation Handbook. The University informed the audit
team that it did not seek out overseas partnerships and that the arrangement
with the College was the only one it was operating at present.
The background to the partnership
11 The College is a privately funded teacher training college in Jerusalem. Founded in 1913, it is one of the oldest educational institutions in Israel and the first to establish Hebrew as the language of instruction. It has a faculty of 120 full-time and 340 part-time staff and there are approximately 2,500 students. The College offers Bachelor of Education degrees, and courses in professional development in aspects of education in both the Hebrew and Arabic languages. Associated with the College are three demonstration schools. It describes itself as a fully accredited teachers' college and for this purpose is recognised by the Ministry of Education and the Council for Higher Education in Israel. Support for the MA in Applied Research in Education is administratively located in the College's Institute of Applied Research in Education.
12 The audit team was told that the partnership between the University
and the College arose out of the University's commitment to applied educational
research, which is focused in the University's Centre for Applied Research
in Education within the School of Education (also referred to as 'the School').
The University's prominence in applied educational research had led to
contacts with educationalists generally, and specifically with the College,
as a progressive and prestigious institution in the field of professional
education in Israel. The Dean of the College was personally known to the
University through his participation in a research network, and three graduates
of the University's master’s degree in applied research in education
were associated with the College. One of these, who continued to be active
in educational research, maintained close contact with the School.
Initial approval and validation process
13 Primary responsibility for collaborative partnerships lies with the University's Validation Committee, which is one of the committees of Senate. In accordance with its terms of reference, the Committee considers and advises the Committee of Deans, and the University Senate, on policy issues concerning validation and quality assurance in monitoring the academic aspects of validation. It advises the Senate on detailed proposals for validation and revalidation and on substantial amendments to validated courses. It also appoints validation panels. The Validation Committee keeps under review the teaching arrangements and academic facilities available; it receives external examiners' reports for validated programmes; and it also receives annual reports on validated programmes from partner institutions. In addition, the audit team was told that the Committee is responsible for approving external examiners and the examination boards for validated programmes.
14 The University's policy governing the establishment of collaborative partnerships has been introduced primarily to provide for partnerships in the UK. The relationship with David Yellin Teachers' College is its only overseas partnership. The approval process involves: first, receiving from a prospective partner institution a request for a validation; secondly, the Validation Committee recommending for decision by the Committee of Deans and the Senate whether, in principle, to proceed to a validation; thirdly, appointing a pre-validation adviser to work with the potential partner institution; fourthly, the Validation Committee appointing a validation panel which undertakes a validation visit and exercises a delegated authority to decide on the proposal; and fifthly, the University and the partner institution entering into a formal agreement which incorporates a definitive course document setting out the specifications and requirements of the validated course.
15 In the instance of the partnership with the College, the request for validation of a part-time MA in Applied Research in Education, came through the School of Education. Following contacts made through research and academic conferences, a senior member of the School had visited the College and reviewed the facilities and academic environment. On the basis of the resulting report, the School initiated exploratory discussions with the Dean of the College. A former UEA graduate, who was actively involved in research, was identified as the prospective director of the programme in the College. In the course of the discussions the Dean of the College visited the University. The outcome of these discussions was a jointly formulated proposal for a part-time Master of Arts degree in Applied Research in Education, the content of which was to parallel a part-time programme leading to an award with the same title offered in the School of Education. The programme, as proposed, was to involve a combination of course work and a research-based dissertation. It was to be assessed 60 per cent on the basis of the course work and 40 per cent on the basis of the dissertation. In the paper covering the jointly agreed description of the course, which the School put forward for initial consideration in the University, the School proposed that the main language of instruction should be Hebrew, and that students should be permitted to submit course work and dissertations for assessment in Hebrew. Among other statements in the School's paper was one that the formal agreement with the College would state that the College would provide, as required, English translations of a sample of dissertations and course work, and interpretation as necessary to assist in oral examinations. The proposal came to the Validation Committee with the approval of the Dean of the School of Education acting on behalf of the School Board. The proposal was approved in principle on behalf of the Validation Committee by Chair's action, and the recommendation from the Validation Committee was approved on behalf of both the Committee of Deans and the Senate also by Chair's action.
16 Further stages in the established process followed. In November 1994
a Validation Panel, chaired by one of the University's Pro Vice-Chancellors,
and consisting of two other members drawn from the University, supported
by the University's Validation Officer, made a formal visit to the College.
On the basis of its assessment, the panel exercised its delegated authority
to approve the course, subject to conditions which the audit team was told
were subsequently met. The University provided the team with a copy of
the report of the Panel.
17 Having taken a policy decision on the proposal, the University entered into a formal agreement with the College; the Vice-Chancellor and the Dean of the College signing for their respective institutions. The formal agreement between the University and the College has wide coverage and includes an annex on the regulations governing the degree. These include instructions to examiners, a statement of the role, requirements and reports of external examiners and a definitive course document. The definitive course document, among other matters, specifies the entrance requirements and recruitment policy, the course design, the means of assessment (including the weightings between components) and certain other quality assurance provisions. The audit team saw the general clarity of the formal agreement as an example of good practice. The agreement also stipulates much that is important for the safeguarding of standards in the partnership.
18 The audit team also noted, however, that there were no provisions in the agreement relating to the language of instruction and the language, or languages, in which course work and dissertations had to be, or were permitted to be, submitted. Similarly, provision is made for documents to be provided to the University, but there is no stipulation concerning the language in which such documents are to be made available, where responsibility lies for making translations and what protocols govern translations, including those of course work and dissertations. Neither is there provision for interpretation at oral examinations. The agreement does not confer authority explicitly to permit the work submitted to be in a language other than English, nor, conversely, does it confer authority to restrict assessment to any specific language or languages. While there may have been a clear understanding about the languages to be used, or allowed, for instruction, assessment, documentation and monitoring, nothing is provided on this in the agreement. In the light of the obligation stated in the agreement to ensure no racial, religious or political test in order for a person to be registered or to graduate, and in the light, also, of the availability in the College of programmes in the Arabic language, the absence of provisions relating to the language of instruction and assessment may have additional relevance. The University may wish to consider the advisability of an early review of what terms should be stated relating to language, translation (including the protocols governing this) and interpretation in a revision to the agreement.
19 There are two further matters that the audit team noted about the content
of the agreement. The first concerns appointments and the rules governing
assessment and these are discussed below in paragraphs 39 and 41. The second
concerns the process of approval within the University's regulatory structure.
The team was told that the proposal for a partnership with the College
was the first proposal for an overseas collaboration considered by the
Validation Committee. It was also the first proposal for a collaboration
specifically on a master’s degree, and the first involving teaching
in a foreign language and assessment which was in practice to be permitted
in that language. Despite these features the consideration given to the
proposal within the University's regulatory structure prior to its validation
was handled by Chair's action in respect of all the four bodies which hold
responsibility for considering it. According to the minutes provided to
the team, the chairs of committees, having taken Chair's action in approving
the proposal, did not draw attention to any policy issues raised. Similarly,
the report of the Validation Panel did not identify policy issues to which
attention had been given, or which merited subsequent consideration as
a general policy question for the University. Whilst recognising that the
approval processes took place in 1994, and that much has happened since
then, both within the University and in Israel, the University may nonetheless
wish to reflect upon the policy issues associated with collaborative provision
which this partnership raises; most notably, perhaps, the implications
of providing or administering a programme offered in practice substantially
in a foreign language and with the option, largely taken up, of being assessed
in a language other than English.
Arrangements for the admission of students
20 The University has made explicit in the formal agreement that students registered on the programme are students of the College. They are registered as candidates for the University's award, not as students of the University. The University does not communicate directly with the College's students.
Arrangements for programme monitoring and review
21 The pre-validation proposal for the programme from the School of Education had stated that validation would be a developmental process, taking the form of annual review visits, during which there would be both continuing monitoring and staff development. The framework for this was set out in the formal agreement. The process of pre-validation and validation which was followed reflected this approach. The decision on the pre-validation proposal had been taken by the Chair of the Validation Committee, which was reported to the Committee in October 1994. The normal process, according to the subsequent (1995) Validation Handbook, was that 'once approval in principle has been granted to proceed with validation a member of UEA faculty is nominated to work for an agreed period with the course team to provide expert and general advice and act as a "critical friend" to the course team during its preparation for validation'. On this occasion the Validation Panel proceeded directly with a visit, and approval of the partnership took place in November 1994.
22 The agreement governing the partnership between the University and the College provides for continuing monitoring of the programme in the four year period governed by the initial agreement. Monitoring is the responsibility, in the first place, of a University adviser for the programme. The adviser acts in a liaison capacity. The adviser is an ex-officio member of the programme committee which was to be appointed in the College with responsibility for the programme. The adviser is responsible for receiving samples of student work and other relevant documents as agreed between the adviser and the College, and for reviewing all documentation. Two members of the University's staff, one of whom is normally the adviser, are responsible for visiting the College each year to undertake an evaluation of the programme. Having reviewed all documentation, the adviser is required to submit an annual report on operational matters, both to the University and to the College.
23 The University's Validation Handbook contains further guidance on the office of adviser. When teaching under this partnership started, the Handbook in use was the February 1995 edition. A second, revised edition was issued in 1997 with some changes of emphasis and clarification in the function of adviser. The Handbooks indicate possible activities and areas of involvement for advisers but they also state that, within the broad terms of the role, the objectives of what the adviser did were to be agreed by the adviser in consultation with the course or programme team. According to the 1997 edition of the Handbook, the adviser is expected to provide for the sharing of good practice and experience but it also states explicitly that 'it is not an auditing role'. In relation to the adviser's expected involvement in the examination board for the programme, the Handbook provides that normally the procedure should be for the adviser to attend the meetings of boards of examiners. However, it also gives advice to advisers that 'you will need to discuss with the course team how the best use of your time may be made - this may involve only occasional attendance at formal meetings and Board of Examiners meetings...'. Advisers are directly responsible to the University's Validation Committee to which an annual report has to be made. The 1995 edition of the Handbook specified that the report should be factual rather than discursive or evaluative, the second edition of the Handbook states that the annual report is expected to review the previous year's objectives, record activity over the year and set out objectives for the forthcoming year.
24 In respect of the partnership with the College, the adviser's functions had, therefore, been outlined in different ways. The University informed the audit team that in the period since the approval of the programme - and excluding the visit from the University in association with the re-validation of the programme - there had been four visits to the College by staff from the University. The University provided to the team reports on three of these visits. Staff and students associated with the programme also commented on the more recent visits, including that connected with the re-validation of the programme. It was clear from these comments that the visits of staff from the University were important and played an influential, and indeed inspirational, academic function in communicating an understanding of the thinking behind, and justification for, the approaches to educational research on which the programme placed particular emphasis. The team was also informed that the value of these visits was reinforced by contact maintained at a distance. However, less emphasis had been placed on monitoring the delivery of the programme in accordance with what was set out in the agreement. The expert and general advice needed to support institutional and staff development to ensure that the management of the programme, changes in the curriculum, assessment and the determination of results accorded with what was intended under the agreement with the University, had also not been explicitly addressed.
25 The University exercises its overall responsibility for monitoring its collaborative partnerships through the Validation Committee. For the partnerships with institutions in the UK the Validation Committee delegates parts of this responsibility to joint boards of study which separately link the University with each partner institution. This is to allow the Validation Committee to concentrate on policy issues and the validation of new courses. There is no joint board of study with the College, and the Validation Committee has not, in consequence, delegated its responsibility for the general monitoring of this degree. The University also told the audit team that there was no committee in the School of Education exercising oversight over the partnership on behalf of the School or by delegation from the Validation Committee, although, on an informal basis, an unofficial meeting in the Centre for Applied Research in Education was informed of developments on the programme. Nor was there any formal link between the programme offered collaboratively and the parallel internal master’s degree with the same title.
26 The Validation Handbook provides for the Validation Committee, in exercising its monitoring function, to receive annual reports from the University's adviser, from the course director in the partner institution and from the external examiner appointed to the programme. The audit team was given two reports presented to the Validation Committee, which were described as the adviser's annual reports. Both were the reports on the visits to the College referred to above. One document was in the form of a report to the British Council which, the team was informed, the Chair of the Validation Committee had accepted in place of an annual report. The second formed part of a more extended report, because the visit of a member of the University staff to the College had been associated with attendance at a conference and with leading a seminar elsewhere in Israel.
27 The audit team was also given two reports from the course director in the College. The University informed the team that through an oversight the University had not sought, and had not received, a report relating to the second year of the programme. The team was provided with two reports from the external examiner.
28 The audit team recognised that recent visits from the University had left an impression on staff of the College and on students which had helped to sustain the academic endeavour associated with the programme. However, the University had intended to offer developmental support as the programme was delivered, because the recruitment of staff in the College was dependent on the approval of the programme. Some misunderstandings about the University's expectations could have been avoided, had there been the intended developmental support. As it was, the College had taken the initiative both to assign more staff to support the management of the programme prior to the admission of the third cohort of students, and had begun to formulate what it recognised to be essential policy to govern important decisions.
29 The audit team recognised an ambiguity in the role of the University's adviser. On the one hand the University was heavily dependent on the adviser for monitoring the programme so that, in its delivery, it accorded with what the University intended under the agreement with the College and, also, in serving as the means by which the Validation Committee was able to exercise its stated responsibility. On the other hand, it was explicitly stated in the Validation Handbook that the Adviser did not have an audit function and had considerable discretion in negotiating a role with the programme committee in the partner institution. In this situation it was not clear to the team who was expected to monitor the operation of this programme so that the University could assure itself that requirements, specified as being important to the academic standards of the University, were being maintained and how, if there were such a post, the University was intending to secure the information to allow the responsibility to be carried out.
30 The audit team noted that the reports received by the Validation Committee did not permit that Committee to assess, on behalf of the University, whether many of the provisions set out in the agreement covering the partnership were being adhered to or not. The reports from advisers made after visits to the College conveyed a broad appreciation of what was being achieved and of the commitment of the staff in the College, but these did not cover the topics indicated in the agreement or in the second edition of the Validation Handbook. Neither through these reports, nor the others, was the University able to confirm that provisions covering entry requirements; equal opportunities in respect of the registration of students; the curriculum of the programme offered; the structure of assessment; or the method of assessment and determining results conformed with what had been agreed, despite the emphasis placed in the Validation Handbook on these matters as being important for ensuring academic standards. Two reports received from the College did indicate that important provisions in the agreement were not being observed. The team was informed that the Validation Committee had deferred action on these because a re-validation visit was pending three months later.
31 It was also apparent to the audit team that the Validation Committee
had not performed as a proactive body in carrying out the requirement to
keep teaching arrangements under review between validation and re-validation
in the absence of a joint board of study. The University may wish to consider
the necessity of reviewing both the assignment of responsibilities for
monitoring the programme and the sources of information needed for those
responsibilities to be undertaken with the thoroughness that the University's
stated policy requires.
32 University policy provides for the re-validation of programmes, the procedure for which is broadly the same as for initial validation. The programme at the College was subject to a re-validation visit in 1997. The panel comprised a Pro Vice-Chancellor and the Dean of the School of Education.
33 The re-validation report confirmed the strength of the programme at the College and commented on the enthusiasm and commitment of the staff and students. The programme was re-validated for a further three years, subject to action on a list of points. This list reinforced the general requirement that the programme should accord with what had been specified in the formal agreement governing the partnership. Where the College wished to propose changes, these were required to be formally agreed by the University. The audit team was provided with documents to show the care which the University was taking to pursue the action points. The focus of re-validation, as the Validation Handbook indicates, and as the team was advised, is on the course as offered in the partner institution, though the re-validation of the programme at the College did point to certain action required by the University.
34 It was clear to the audit team that the re-validation of the programme
in the College had been conducted with care and rigour and that appropriate
follow-up action was in progress. It was also apparent to the team that
the re-validation had served to remedy matters that should not have arisen
as problems if the University had engaged in its intended developmental
activity. Furthermore, departures from what the University expected should,
in the normal way, have been identified by the University through its process
of regular monitoring. The University may wish to consider the advisability
of extending the process of review associated with re-validation so that,
in addition to reviewing the programme in the partner institution, it undertakes
a dispassionate assessment of how far, and how adequately, the University
itself has carried out the responsibilities falling to it.
35 As indicated in paragraph 20 above, students registered on the programme belong to the College. The registered candidates have no right of access to University resources. Before entering into the partnership, it is the University's policy to establish whether a potential partner institution can provide necessary facilities and an appropriate environment for those registered on the proposed programme. This is one consideration in deciding on institutional approval in principle and the question is addressed in both the initial validation and the re-validation of the programme. The audit team noted that these considerations were apparent in the papers it saw relating to the approval in principle, validation and re-validation of the programme at the College.
36 The University has recognised that, while in most respects the formulation
under which those registered on the programme at the College are both students
of the College and candidates registered for the University's award, in
some areas the University's obligations to these students has required
clarification. Following the re-validation of the course, and in accordance
with the decision of the panel, the College is drawing up its own complaints
and appeals procedures after studying those of the University. The University
specifies that there can be no appeal against the academic judgement of
examiners. It was evident to the audit team that the University had thought
through what limit it would place on its obligations towards those studying
for its award and had followed through this policy consistently.
Student assessment and academic standards
37 The University has delegated to the College the authority to set all student assessments, provided these accord with the programme as validated. The validated programme specifies the number and weighting of items of course work and the dissertation. The agreement also provides that the College is responsible for making the arrangements for the assessment and examination of students in accordance with the University's regulations and conventions, and an annex contains instructions to examiners. Under the agreement the University is entitled to appoint one or more external examiners nominated by the College and it is required to approve the membership of an examination board nominated by the College. The agreement stipulates how these internal and external examiners are to examine candidates. The external examiner is required to be one of two or more examiners examining each candidate.
38 An annex to the Agreement records instructions to examiners for the degree. These set out the requirements for the award and state decisions to be reached by the examination board. For those unfamiliar with the established practices of the University in assessing course work and dissertation, the instructions would offer only limited help. Some process of co-ordinating marking standards might have been expected to have formed part of staff development. The School of Education's proposal, which formed the basis for the University's approval in principle of the partnership, had specified that the University would review a sample of dissertations and course work which would be translated, and it implied that the University would observe, or take part in, oral examinations. The audit team was told that course work is not, in practice, monitored and that the two dissertations referred to the University for monitoring were the only two so far submitted in English. The team was also told that the adviser has not attended a meeting of the examination board for the programme.
39 Both from the documents made available to the audit team and from what it was told, it was apparent that the provisions in the agreement relating to the assessment and examination of candidates had not been fully observed. Some of the departures would be regarded as significant, indeed fundamental, in the context of United Kingdom practice. Following the re-validation visit, the University has already introduced changes. The University may wish to consider whether the option of the University adviser attending meetings of the examination board should be exercised so that the College can gain a clearer understanding of the University's expectations in the assessment of course work and dissertations, and in the determination of results. The University may wish to consider the advisability both of implementing the formal instructions governing the assessment of course work and dissertations in the appendix to the agreement, and also of clarifying the University's own expectations regarding the monitoring of course work and dissertations, and the manner in which translations are to be made.
40 The University has been in the advantageous position of having, as external
examiner, a person of standing in Israeli higher education with directly
relevant subject expertise. The person does not have experience of examining
taught degrees in the UK or of externally examining for them. At present
the full and substantial burden of external examining under the regulations
falls to that one person. The external examiner was, as the re-validation
panel noted, handicapped by the University's failure to provide her with
the briefing documents which would normally be sent to external examiners.
The audit team learnt that this omission has been put right. The University
may wish to consider, however, the advisability of further strengthening
the examining arrangements by reference to the CVU Code of Practice paragraph
7.3 where it states 'those appointed for overseas collaborative programmes
should, between them, have a combination of appropriate UK and relevant
overseas experience'.
Staffing and staff development
41 The University states in its Validation Handbook that its aims are 'to ensure that the programmes validated by UEA are of such high academic standard by paying regard for each programme to...the qualifications and experience of the lecturing staff'. Staff available in the College was a matter to which the Validation Panel gave attention in initially validating the programme. However, the agreement does not provide for the College either to inform the University of the staff teaching the programme, or to seek the University's approval for whoever is to teach. The College informed the audit team that its normal practice was to inform the University of the staff teaching on the programme and to supply curricula vitae. It was not standard practice to receive confirmation from the University. In what the University told the team there was support for this account of existing practice. The University may wish to consider the advantage of clarifying these aspects of its partnership.
42 The need for staff development was recognised both by the University
and by the College in the initial consideration of the proposed partnership.
The re-validation panel reviewed the staff development being provided by
the College and commented favourably on that provided both for new and
for established staff. The audit team was told that both the Dean of the
College and the first Course Director had spent periods at the University
in order to observe and learn about the University's approach to managing
its programmes and assuring the quality of them. The team was also told
that the first Course Director had, on his initiative, made other visits
in order to maintain contact with relevant staff in the University.
Claims made for approval in Israel
43 The formal agreement between the University and the College stipulated that the College was to provide the University with confirmation that the programme had the approval of the Israel Council for Higher Education. The University informed the audit team that this permission had not been forthcoming, but that the Ministry of Education had given a form of approval on the basis of which the University decided to proceed with the partnership with the College. The team did not inquire into the specific terms of official Israeli responses to requests for approval of the degree programme. Subsequent to the audit visits to the University and the College, the University provided QAA with an extract from a document used by the College when admitting students to the programme. This document included the passage:
‘…This two-year program operates within the framework of the Institute for Applied Research in Education which was founded in the College and which is patterned after the MA degree offered by the University of East Anglia in Norwich, England.
At the end of the period of study at the Institute, students will receive a transcript and a certificate of completion of the course of studies (which includes a research project in a field relevant to the student’s work). This is the only certificate we are entitled to give at this point; signify your agreement to this in writing on the designated space below. This certificate does give accredited hours of in-service training for courses completed.
In order to remove all doubt I am clarifying this point and ask to receive each student's signature confirming that they have received this announcement and are aware of its contents: until the Council for Higher Education permits the University of East Anglia to open a branch in the College, all that you will receive from the College at the end of the program is a certificate of completion from the Institute of Applied Research in Education in the College…’
QAA was not provided with information on what was stated to students about
the degree which the University confers on students who successfully complete
the programme followed at the College as distinct from any award from the
College. The University may wish to satisfy itself that such statements
comply with what is prescribed in the HEQC Code of Practice on Overseas
Collaborative Provision, (2nd edition, 1996), concerning the information
that should be provided to applicants and students on the official recognition
accorded to the award made by the University.
Publicity and promotional material
44 The University has made no stipulations concerning publicity in its
agreement with the College. Since adopting a policy on publicity in September
1996 the University has made an annual request to the College for publicity
information. As part of the re-validation of the course, the panel reviewed
publicity by the College and was satisfied that it was fair and accurate.
45 The degree certificate of the University awarded to graduates of the programme records the place of study. Nothing is stated about the language in which the programme was assessed. The audit team was told by the University that this was because the programme could be assessed either in English or in Hebrew. It was also told that there is no requirement for students to state formally in which language they wish to be assessed. As explained above there are no provisions in the agreement covering the programme about the language of assessment.
46 Besides reviewing this omission in the agreement and from the regulations for the award, the University may wish to consider the advisability of stating on the certificates of its awards the language or languages of assessment. In so doing the University may wish to give due weight to the relevant provision in the HEQC Code of Practice for Overseas Collaborative Provision, (2nd edition, 1996), and to local circumstances in Israel.
Conclusion
47 The University is committed to maintaining high standards in its
awards and, to this end, it is committed to principles and procedures that
are designed to ensure that programmes validated by the University lead
to awards which are consistent in standard and are comparable with other
awards conferred by the University.
48 The University has been consistently impressed by the quality of the staff and students on the programme at the College, by the environment created by them and by the quality of work produced on the programme which leads to the University's degree of Master of Arts in Applied Research in Education. The University itself makes a contribution to the academic environment in the College, notably on the occasion of visits by persons from the University.
49 The University aims to assure itself of the standards of its award offered through its partnership with the David Yellin Teachers' College and to this end the University adopted procedures which it decided should largely accord with those applying generally to its validated programmes. The University also approved specific arrangements for the degree, in application and elaboration of its procedures. Broadly, these procedures and arrangements, and the manner in which relevant ones are set out in the agreement with the College, should put the University in a position where it can convince its peers and others that it is able to demonstrate the basis of the claim it makes for its standards. There are, nevertheless, some important limitations to these procedures and to the agreement between the University and the College. Among points noted in this report, the University may wish to consider the assignment of responsibilities in the University for monitoring the programme, and to gaps in the design of the collaboration in respect of the language of assessment and programme management. The University may also wish to consider provisions relating to what is to be translated and the protocols governing the translation of specific documents - notably those forming part of assessment. The audit team has additionally reviewed the manner in which the University has operated its own procedures and assisted its partner institution to understand and operate them in the manner required by the University. In some respects the procedures have been operated with thoroughness, but the University's approved procedures and arrangements relating to the assessment and examination of students have not been implemented with the full rigour that might have been expected and, given their significance for the maintenance of academic standards, the University will wish to ensure their diligent application henceforward. In addition, the manner in which the University has used its monitoring processes has restricted their value as instruments of quality assurance; it will wish to ensure that due attention is given to following the procedures and approved arrangements relating to the monitoring of this programme.
Annex
Commentary on the audit report supplied by the University of East Anglia
The University of East Anglia has been proud to work in partnership with David Yellin Teachers' College. An innovative and challenging Masters programme has developed which has grown in reputation and brought together educators from all sections of Israeli society. The MA in Applied Research in Education has been enthusiastically received by a distinguished group of students and established pioneering work in a neglected field of qualitative research in Israel.
The academic focus of the programme and procedural framework have evolved within a context of increasing, and occasionally divergent, regulatory and quality assurance frameworks in both Israel and the UK. QAA's audit highlights the particular challenge to the development of a common understanding of such frameworks, but nothing that QAA has observed or reported challenges our evaluation of the academic quality of the programme.
QAA's report on the partnership between the University of East Anglia and David Yellin Teachers' College concerns a relationship which, having developed from shared research interests in a specific area of educational research, focused upon the development of a specific programme of study to lead to the award of the MA in Applied Research in Education. This is the University's only validated overseas programme.
The ethos of the partnership has been strongly developmental in an academic context. The reputation of the programme and the innovative methods that were being introduced have grown in the local environment, despite working in an often volatile regulatory framework for higher education provision in Israel. The latest changes to the Israeli regulatory framework have led to the decision that the University will not be seeking a new licence to renew the validation arrangements, as the law does not permit a programme which is specifically tailored to reflect the Israeli educational system. However, it is intended that the collaborative research partnership will continue.
The University is aware that in some areas of its quality assurance procedures for collaborative provision, additional measures for overseas provision are needed to reflect the particular nature of quality assurance at a distance and the significant differences in culture and context. In particular, measures are being taken to:
- standardise and clarify the format of contractual arrangements;
- clarify the implications and procedures for providing or administering programmes offered in practice substantially in a foreign language;
- clarify the role of the University Adviser;
- clarify and enhance the arrangements for examination procedures and the
appointment of external examiners for programmes delivered overseas.
Appropriate action will be taken on the final point as soon as possible.
The University was pleased to note the comments made by auditors in paragraph 48 concerning the quality of staff and students' work at the College and the recognition that the University had made a contribution to the academic environment at the College.
Top