Annual Review Trends Report 2004-05

in partnership with

   

PDF version (297KB)

Contents

• Executive summary
• Chapter 1 - Introduction
• Chapter 2 - Outcomes
• Chapter 3 - Processes
• Chapter 4 - Structures
• Chapter 5 - Conclusions
• Appendices
• Appendix 1 - Summaries of the key themes arising from the analysis of the strengths, good practice and weaknesses listed in the published major review reports
• Appendix 2 - Major review - changes to processes
• Appendix 3 - Acronyms

Executive summary

The Quality Assurance Agency for Higher Education (QAA) is committed to produce and publish an annual report of emerging trends at the completion of each annual schedule of reviews (Handbook for major review of healthcare programmes, paragraph 96). The main purposes of these reports are to record the findings of review teams; to promote good practice, focusing on learning gained about academic and practitioner standards, and quality of learning opportunities; to log the developments in the review processes and procedures, and the changing context in which the review method operates; and to suggest possible changes to any future methodology.

This second report considers the 28 major reviews across 15 disciplines that took place between spring 2004 and spring 2005, approximately one-third of the total number of reviews to take place during the contracted major review cycle 2003 to 2006. The report is structured to consider the outcomes, process and structures relating to major review.

The major review process is an effective periodic review system built on existing internal quality assurance processes in higher education institutions (HEIs) and strategic health authorities (SHAs), as well as considering other external quality assurance arrangements. It has successfully acted as a vehicle for the Nursing and Midwifery Council's (NMC) annual monitoring (see section 3.1.5 of the main report for further details).

The findings of the review teams have been generally positive with only one limited confidence judgement made in relation to the academic and practitioner standards for one programme. The quality of learning opportunities has also been positive, with the majority receiving commendable judgements and only a few programmes the approved judgement. The opportunity to be more specific in the judgements made through differentiation by programme and/or level and/or mode has been welcomed by all involved in the reviews, as has the use of action plans which enable the HEIs, SHAs and partner placement providers to respond to the strengths, good practice and weaknesses identified by reviewers (see section 2.2). The plans are published in the review reports, thereby providing a more holistic view of the provision. Initially a little verbose and over-enthusiastic in the identification of strengths and good practice, the bullet points and hence the action plans have become more streamlined and meaningful (see sections 2.2 and 3.3.2).

The positive features of the provision reviewed so far include the:

• beneficial role of the clinical assessor (including practice or clinical educators, clinical practice facilitators, practice placement facilitators, supervisors and mentors) in enhancing student learning, supporting mentors and assessors, and improving standards and quality (see sections 2.1, 2.2.1 and 2.2.2)
• effective partnerships that exist between most HEIs, SHAs and practice placement providers further enhanced by preparation for the major review process itself and the work of the practice review facilitator (see sections 2.2, 2.2.1, 2.2.2 and 3.2)
• generally high academic and practitioner standards seen, supported by appropriate learning outcomes, curricula and assessment methods, and reflected in high student achievement in most programmes (see sections 2.2.1 and 2.3.1)
• consistently positive scrutiny of the quality of learning opportunities, particularly initiatives to widen access and retain students, the support provided to students in both academic and clinical settings, the learning resources available in many areas, and the innovative use of new information technology and communication products (see section 2.2.2)
• generally strong placement learning and effective use of placements, despite the national issue of shortages in some disciplines (see section 2.2.2)
• effective arrangements for the monitoring and enhancement of standards and quality by most providers (see section 2.2.3)
• absence of any statistically significant differences on the basis of level or discipline for student achievement, employment, recruitment and attrition, although there are some interesting patterns emerging (see sections 2.3.1-2.3.4)
• high student employment, with allied health profession students studying at degree level being slightly more mobile in their employment than health visiting, midwifery and nursing students (see section 2.3.2)
• overall positive picture of recruitment and retention in the majority of programmes (see section 2.3.3).

The common weaknesses in the provision identified include:

• a lack of consistency in the use and understanding of marking criteria, and poor quality of feedback provided to students (see section 2.2.1)
• the absence of standard processes for returning work to students (see section 2.2.1)
• minimal planned and organised opportunities for interprofessional learning/education particularly in the allied health professions (see section 2.2.1)
• limited service user and carer involvement in programme development or implementation (see sections 2.2.1, 2.2.2 and 3.2)
• the lack of opportunity for clinical assessor updates and continuing professional development, usually due to workload (see sections 2.2.2 and 2.4.3)
• the failure to maintain fully the live mentor register - relevant to health visiting, midwifery and nursing only (see section 2.2.3)
• inconsistency at the faculty, school or department level in the implementation of some quality assurance procedures, and a lack of clarity about the processes for updating modules supported by documentary evidence (see section 2.2.3)
• not keeping formal records of student evaluations and a lack of systematic evaluation between the HEI and practice placements (see section 2.2.3)
• high attrition rates in a small number of disciplines (see section 2.3.3).

A point to note for future debate:

• there does not appear to be consistency (in the pre-registration disciplines being reviewed) as to the appropriateness or otherwise of grading practice assessments, and whether practice assessments should contribute directly to the classification of the award. This is an important area for debate when considering the value attached to learning in practice (see section 2.2.1).

Major review - positive features include:

• the 41 events run by QAA to train the reviewers, brief facilitators and brief subject staff on preparing the self-evaluation document (SED). Some 770 people have attended these events, the overwhelming majority (at least 92 per cent) evaluating them positively, welcoming the opportunity to work with colleagues from different disciplines or providers, and acknowledging the quality, skills and approachability of those delivering the events (see sections 3.1.2 and 3.1.4)
• the preparatory meeting as an effective mechanism enabling discussion about the process, clarification of scope, identification of practice placements to be visited for day two and, more latterly, day three, and establishing a positive working relationship between the providers and the Review Coordinator (see section 3.1.3)
• the significant work undertaken by practice review facilitators (PRFs) in engaging clinical staff in the reviews, as well as deftly organising often complex and geographically spread practice placements into a workable, two-day programme of visits for the reviewers (see sections 3.1.3 and 3.2.2)
• the review team supported by the small group of review coordinators. Both have undertaken a significant amount of work to the benefit of the reviews and personal development (see section 3.1.5)
• the integration of NMC annual monitoring with the major review process (see section 3.1.5)
• the responsiveness of QAA's Health Team in responding to queries, issues and concerns of participants (see section 3.2)
• enabling the student voice to be heard through a variety of mechanisms with scope to identify specific groups/kinds of students to follow-up themes and queries (see section 3.2)
• the 2 days + 2 days + 1 day model of major review visits in enabling reflection and further preparation between the visits for all concerned (see section 3.2.1)
• the use of a draft 0 of the major review report at judgements meetings to discuss and test the evidence again, thereby ensuring rigorous and robustly-evidenced judgements (see section 3.2.1)
• the Major Review Facilitator (MRF) and PRF roles which have promoted partnership working, enabled clinical and academic staff to have an equal input into the SED and action plan, and have ensured that feedback from the reviewers, through the Review Coordinator, is disseminated to all those involved in the provision (see section 3.2.2)
• the use of non-health specialists as MRFs in order to maintain a high level of objectivity and to bring increased knowledge of institutional policy and procedures (see section 3.2.2)
• the refined process for the publication of action plans that provides the plans in an electronic format to HEIs/SHAs earlier in the process and for an extended period, to combat the difficulties experienced by the MRF and PRF in securing sign-off by suitable senior staff of the completed plans (see section 3.3.2).

Major review - challenges include:

• the common intensity of approach that has meant that HEIs/SHAs with small healthcare provision have received the same level of scrutiny as the larger, more complex provisions; it is suggested that, in any future cycle of major review, the intensity of scrutiny is proportionate to the size of the provision and the level of risk attached (see section 3.1.1)
• the time gap between training the reviewers and their first major review. For some this has been an extended period which has, in spite of regular update newsletters, caused some anxiety (see section 3.1.4)
• recruiting reviewers from the smaller disciplines and certain larger disciplines; a significant amount of time has been spent by the Health Team, colleagues in SHAs, professional bodies, Health Professions Council and Skills for Health in generating interest and nominations (see section 3.1.5)
• the lack of service-user involvement directly in the major review process (section 3.2)
• the continually and rapidly changing context in which major review operates (see chapter 4).

 

Chapter 1 Introduction

1 This is the second report in the series of annual review trends reports for the major review of healthcare education: the periodic peer review of healthcare programmes in England in the allied health professions, health visiting, midwifery and nursing. The Quality Assurance Agency for Higher Education (QAA) is committed to produce and publish an annual report of emerging trends at the completion of each annual schedule of reviews (Handbook for major review of healthcare programmes (the Handbook), paragraph 96). The main purposes of these reports are to record the findings of review teams; to promote good practice, focusing on learning gained about academic and practitioner standards, and quality of learning opportunities; to log the developments in the review processes and procedures, and the changing context in which the review method operates; and to suggest changes to any possible future methodology. To facilitate this, the report considers the outcomes, processes and structures of the major reviews that have taken place between spring 2004 and spring 2005.

2 A range of data sources has been used:

• the 28 published major review reports (including action plans) from January 2003 to June 2005, including the statistical analysis of the quantitative data tables, analysis of the bullet points listing strengths, weaknesses and good practice, recorded in the reports
• the analysis of the questionnaires sent to the participants in each of the major reviews that have taken place: review coordinators, reviewers, subject staff (both academic and clinical), and Strategic Health Authority (SHA) staff. The evaluations by the latter two groups are coordinated by the major review facilitators (MRFs) and practice review facilitators (PRFs) respectively
• the six focus groups held during 2003-04 and 2004-05, attended by 110 participants in the reviews, including reviewers, review coordinators, MRFs and PRFs, SHA staff, clinical staff and academic staff involved in major review
• the evaluations for each of the 41 reviewer training events, facilitator briefings and self-evaluation document (SED) workshops
• the reflections from the QAA officers recorded through 45 days of visit support for preparatory meetings (20 days), day five/judgement meetings (23 days), and two 'call outs'.

3 It is important that this report is read in the context of the development of the Partnership Quality Assurance Framework for Healthcare Education in England (PQAF) by the Department of Health (DH) (now Skills for Health¹ (SfH)) with the Nursing and Midwifery Council (NMC), the Health Professions Council (HPC) and the SHAs. The purpose of the PQAF is to develop a more streamlined and integrated quality assurance system for healthcare education in England that is owned by the partners (above) and stakeholders (the higher education institutions (HEIs) and practice placement providers). There are five proposed elements of the PQAF, of which three processes - major review, approval, ongoing quality monitoring and enhancement (OQME) - are supported by two further elements: the shared-evidence base, and benchmark and quality standards. During the academic year 2004-05, the approval and OQME processes were prototyped in seven HEIs; three partner organisations undertook the approval process and four undertook the OQME process; one institution/SHA participated in both the approval and OQME processes. Two organisations had undertaken major review prior to the prototype and were able to test the value of the major review report and action plan as a source of evidence for OQME.

¹ Responsibility for the PQAF was transferred to Skills for Health, a sector skills council, under a service-level agreement from the DH from 1 October 2004.

4 Major review, in the context of streamlining quality assurance and reducing the burden for HEIs, builds on existing internal as well as external quality assurance arrangements; for example, the links with NMC annual monitoring where, for the year that major review takes place in an HEI in conjunction with its placement providers, major review acts as the vehicle for the NMC annual monitoring. An NMC visitor/reviewer is a member of the review team and utilises the evidence base gathered by the team. Major review also replaces the quality element of fundamental/contract review of the HEI undertaken by the SHA. As noted earlier, HPC has been actively involved in the development of major review but, until recently, has not been in the position in its own development to consider linking its quality assurance mechanisms with major review.

5 The purpose of major review is to provide the public with the assurance and confidence that the students and trainees who successfully complete healthcare programmes are competent and safe practitioners. The review and subsequent reports consider, with equal emphasis, practice and campus-based learning. Disciplines, as selected by the partners for inclusion in major review are:

• Audiology
• Clinical psychology
• Diagnostic and therapeutic radiography
• Dietetics
• Health visiting
• Midwifery
• Nursing
• Occupational therapy
• Operating department practice
• Orthoptics
• Paramedic science
• Physiotherapy
• Podiatry
• Prosthetics and orthotics
• Speech and language therapy.

A number of disciplines that are within the professional regulation of the HPC, for example, arts therapy, are not included in major review. Similarly, professions such as medicine, dentistry and pharmacy are not within the auspices of major review in this cycle.

6 Audiology, clinical psychology, operating department practice and paramedic science were added to the original list of disciplines included in major review after initial scoping had taken place. This had a significant impact on the planning and management of the process by QAA. Major review considers NHS-funded (mostly) programmes at a range of levels from certificate of higher education to professional doctorate, including those regulated by statutory regulatory bodies. Continuing professional development (CPD) modules and courses are also part of major review. In each major review, the exact nature and scope of the current provision to be included is agreed before the start of the review.

7 QAA was awarded the contract to develop, implement and manage a cycle of major reviews for 2003 to 2006. The major review method has been developed in partnership with the DH, NMC, HPC, SHAs/workforce development confederations (WDCs) and, additionally, with input (through the major review working group and steering group) from HEIs, Trusts, the voluntary sector and the independent sector. The early methodology was tested by six prototype reviews carried out in 2001-02 and evaluated by QAA and an external evaluator commissioned by the DH (England). The method was refined in light of these experiences and the first review occurred in January 2004. The prototype reports were considered in the first annual trends report; therefore, this report will only consider the reviews that took place from January 2004 onwards.

Chapter 2 Outcomes

8 The outcomes of major review are judgements about the academic and practitioner standards and the quality of learning opportunities of an HEI/SHA's healthcare provision, and the identification of key strengths, weaknesses and good practice within the provision that form the basis of an action plan which is completed by the HEI/SHA and published as part of the major review report.

2.1 Judgements

9 The Handbook notes (paragraph 77) that 'reviewers will make a judgement about the academic and practitioner standards set and demonstrated in the programme for each of the health professions offered by a provider'. In coming to these judgements, the review team considers four elements: learning outcomes, curriculum, assessment and student achievement. In contrast, single judgements are made across the totality of the provision for each of the elements making up the quality of learning opportunities: learning and teaching, student progression, and learning resources and their effective utilisation. In all instances, review teams were able to make judgements in accordance with the Handbook, although additional challenges were faced where self-evaluation documents (SEDs) did not follow the guidance in Annex D of the Handbook to provide a separate section for each discipline in academic and practitioner standards, or were not written evaluatively in the areas listed.

10 One feature of major review is the ability to differentiate judgements (in both academic and practitioner standards, and quality of learning opportunities) by programme and/or mode and/or level. The reviewers welcomed the opportunity to be more specific in their judgements and not penalise the whole provision when only one area, for example, was in difficulty. One limited confidence judgement for one programme was made in academic and practitioner standards; and 13 differentiated judgements were made in the quality of learning opportunities, all by programme.

11 The maintenance and enhancement of standards and quality (MESQ) is the only element of major review where, although considered thoroughly by the reviewers and made the subject of a narrative commentary in the report, no judgement is made. The reviewers consider the effectiveness of the partnership between the HEI and practice placement providers; fitness for purpose of the quality assurance processes at the subject level; the partnerships between the HEI and its SHA(s); and, finally, the effectiveness of the relationship between the HEI and its partner placement providers with the professional statutory regulatory bodies (the Handbook, paragraph 93). The reviewers note that, such is the importance of MESQ in contributing to public confidence, the review method in any future cycle would benefit from a judgement being made.

12 There are two judgement categories for academic and practitioner standards: confidence and no confidence (a subcategory of a confidence judgement is limited confidence); and three categories for the quality of learning of opportunities: commendable, approved and failing, as defined by the Handbook (paragraphs 79 and 85). The judgements for the 28 reviews that took place between spring 2004 and spring 2005, for which there are published reports, are reported in Table 1.

13 In evaluating whether the judgements made by the review team were consistent with the dialogue during the review, 99 per cent of respondents to the post-review evaluation questionnaires agreed that they were consistent. On the whole, participants considered the categories of judgements appropriate; however, teams commented in the questionnaires and the focus groups that the definition of the term 'commendable' in the Handbook is not that used in everyday language and this can create dissonance.

2.2 Strengths, good practices, and weaknesses

14 For each major review, the reviewers identify key strengths, good practices and weaknesses that are discussed in the text and listed as bullet points in each section of the review report. These bullet points are then transferred verbatim into the action plan, for completion by the providers, prior to publication as part of the major review report. The nature and number of the strengths, good practices and weaknesses have altered during the review period. Initially, the reviewers were generous in the number of bullet points identified for the action plan; for example, the provision was commended for statutory regulatory body requirements such as fitness for practice on pre-registration programmes. This was proving unhelpful in enabling HEIs and SHAs to complete the action plan in a meaningful way or within the timescales. In response to feedback, additional guidance (QAA guidance regarding action plans, 2004-05) was provided by QAA to the reviewers and review coordinators that the review reports should only include key strengths, good practice and weaknesses in the bulleted lists and, therefore, the action plan. However, this did not preclude mention of other notable points in the text of the report. Analysis of the action plans for this report (see Appendix 1) highlighted that a number of the bullet points identified under MESQ would be better placed in the relevant elements within academic and practitioner standards or the quality of learning opportunities.

Table 1: Judgments reported in the published reports between spring 2004 and spring 2005

[Note in Table 1 below judgements relate to all the provision being reviewed. In some instances a separate judgement is made for specific programmes - this is considered to be a differentiated judgement]

	Academic and practitioner standards	Quality of learning opportunities
		Learning and teaching	Student progression	Learning resources
Allied Health Professions	Confidence x 9 reviews	Commendable x 9 reviews Approved x 2 programmes in diagnostic radiography and therapeutic radiography	Commendable x 9 reviews	Commendable x 6 reviews Approved x 3
Mixed - AHP and Nursing and/or Midwifery	Confidence x 8 reviews Limited confidence x 1 programme in clinical psychology	Commendable x 8 reviews	Commendable x 8 reviews	Commendable x 7 reviews Approved x 1 Approved x 2 programmes in clinical psychology
Nursing, Midwifery and Health Visiting	Confidence in 11 reviews	Commendable x 11 reviews	Commendable x 10 reviews Approved x 1 review Approved x 4 programmes in community nursing, nursing (all branches) and midwifery	Commendable x 11 reviews
Total	28 Confidence judgements	28 Commendable judgments Approved judgements in 2 programmes	27 Commendable judgements 1 Approved judgement Approved judgments in 4 programmes	24 Commendable judgements 4 Approved judgements Approved judgements in 2 programmes

15 Through analysis of the published reports a series of key trends is identifiable. One strength that spans all disciplines and elements of major review, and is reinforced by the data from the evaluation activities, is the effective partnership that exists between the HEI, placement providers and SHAs in developing and delivering the programmes. However, many providers acknowledge the crucial impact that major review activity, such as writing the SED, has in developing and facilitating such partnership. Another strength is the important role of the clinical assessor (used for the purposes of this report synonymously with practice or clinical educators, clinical practice facilitators, practice placement facilitators, supervisors and mentors) in supporting and assessing students in practice placements either directly or indirectly.

16 The following three sections identify other key trends specific to academic and practitioner standards, the quality of learning opportunities and MESQ. Summaries of the key themes for each section disaggregated, where applicable, by the type of programme area (allied health professions, or health visiting, midwifery and nursing) are presented in Appendix 1.

2.2.1 Key trends - Academic and practitioner standards

17 Partnership links between HEIs, SHAs and placement providers appear to be well developed and working effectively. Examples are given in the majority of reports of all stakeholders working together to develop curricula that ensure learning outcomes and assessments represent educational, professional, regulatory and service needs, and integrate fully theory and practice. Two reports note, as a feature of good practice, user involvement in the development of the curriculum although this is not widespread. Few weaknesses are cited in the reports, but one notes the limited contribution of clinical assessors to curriculum development; three others note a difficulty in achieving an effective consolidation of practice in specialist community public health nurse programmes (SCPHN); and one that employers of students on a BSc Health Care Practice and MSc Advanced Clinical Practice programmes are not working fully with the HEI to promote student achievement.

18 In all reports, the curricula appear to provide comprehensive coverage of key cognitive, professional and clinical skills and competencies, and reflect The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ), relevant subject benchmark statements and regulatory requirements, with the exception of two return to practice programmes in nursing and midwifery and one programme in prosthetics and orthotics, which do not explicitly map the learning outcomes to the benchmark statements. The review reports note frequently, as a strength, that staff research and scholarship, where it takes place, informs curriculum development, ensures currency and enhances teaching quality.

19 In the majority of programmes, learning outcomes are appropriate, take account of statutory regulatory and professional body requirements, and are communicated clearly to students and staff. One report notes that the clinical psychology programme had explicitly aligned its learning outcomes to the NHS modernisation agenda.

20 In terms of assessment, the majority of reports note that the handbooks, guidelines and marking criteria are clear and comprehensible for both students and clinical staff. One report notes a specific weakness within an allied health profession programme where there is some inconsistency in the way assessment criteria are written across modules, resulting in some confusion among clinical assessors in marking assignments, and among students in determining word limits and associated penalties. A further five reports record a variable quality of assessment guidelines and strategies, leading to some inconsistency in marking, and a lack of clarity in the process for dealing with student failure. Another report records a lack of marking criteria for marks above 70 per cent, making it difficult for nursing students to know how to achieve standards beyond this mark.

21 The majority of reports acknowledge the use of a variety of assessment methods, in academic and clinical settings. This is listed either as a specific strength, as in the early reports, or more latterly in the text. Other strengths identified include the arrangements for student support in assessment (all reports); the use of practice portfolios affording a good opportunity to integrate theory and practice; the high quality of student work as identified by the external examiners (seven reports, both allied health professions and nursing); and the contribution of an external moderator across a range of clinical placements (one allied health profession report).

22 Good practice is recorded in one health visiting programme where a panel is used to scrutinise student assessments, and in physiotherapy where an e-learning environment is used to provide feedback on assessments of practice.

23 The most frequently identified weakness is in relation to the feedback provided to students, which is considered to lack detail, be of poor quality, difficult to read and often delayed to such an extent that the feedback received is not able to inform future assessments. These weaknesses relate to both academic and clinical assessment and are slightly more prevalent in the allied health professions. An issue identified in one report, in occupational therapy and physiotherapy programmes, is where the placement assessment does not contribute to the final award. There does not appear to be consistency (in the pre-registration disciplines being reviewed) as to the appropriateness or otherwise of grading practice assessments, and whether practice assessments should contribute directly to the classification of the award. This is an important area for debate when considering the value attached to learning in practice.

24 Good arrangements for student support in assessment feature in all reports. Students value the support they receive from clinical assessors and academic staff. Student achievement appears to be good, with completion rates of 100 per cent normally noted in clinical psychology and speech and language therapy, and a high take-up of employment in the NHS. Non-achievement of funded students who enrol on programmes but do not go forward to assessment is only noted explicitly as a concern in one nursing programme. This issue is apparently only applicable/relevant to heath visiting, midwifery and nursing post-registration, CPD programmes/courses.

25 In eight reports, the reviewers noted and endorsed explicitly in the bullet points where employers and/or external examiners have confirmed student achievement of fitness for purpose, practice and award. This visible confirmation is an issue for some SHA staff who would welcome more explicit commentary in review reports on student fitness for purpose in order to meet their contract monitoring requirements. QAA gave additional guidance to the review coordinators through the CPD events provided for them. However, it is suggested that, in any future cycle of reviews, there is stronger guidance on ensuring an equitable balance between practice and academic issues.

26 Clinical assessors are noted as a particular strength in supporting student achievement in placements. Many examples of good practice and effective working are identified, including the tripartite meetings between student, clinical assessor and link tutor (where a lecturer has a formal relationship with that clinical area/staff), which supports both parties and helps the student to integrate theory and practice. Interassessor reliability monitoring also provides further clinical assessor support opportunities. Weaknesses include a lack of clinical assessors in some placements, or clinical assessors who are not sufficiently updated or aware of the required learning outcomes, and a lack of continuity in one placement system and the fragmented nature of midwifery placements from another provider causing difficulties for students in maintaining relationships with clinical assessors.

27 Interprofessional learning/education (IPL/E) appears to be well established in some disciplines, but there is considerable room for improvement in others. Good practice in IPL/E is noted in radiography and speech and language therapy, and plans for the further development of IPL/E programmes and opportunities are noted in clinical psychology and nursing in particular. Full embedding of IPL/E in programme structure is noted in midwifery, health visiting and physiotherapy, although the quality of IPL/E provision can frequently vary across placements and programmes offered by the same providers. Particular difficulties noted are: problems in achieving IPL/E in busy clinical placements; limited or insufficiently exploited opportunities for developing IPL/E within the curriculum; a lack of sharing good practice within or across schools or faculties within the same institution; and, in some programmes (paramedic science, podiatry and therapeutic radiography), students felt that IPL/E is introduced too early in the programme, before students are established in their own discipline. The reviewers have found that there is little differentiation made between interprofessional working, learning and education, and there are assumptions that working in multidisciplinary settings or teams is synonymous with interprofessional working and learning. The extent to which the emerging health professions framework (part of the subject benchmarks) is used and addressed varies considerably across the reviews and the disciplines.

28 A higher proportion of CPD programmes is delivered in the health visiting, midwifery and nursing disciplines, and the number of comments in the reports reflected this. On the whole, the programmes are considered to develop practice, meet service needs and enable theoretical learning to be applied to professional practice. There is a wide and varied suite of postgraduate programmes across all reviews, many of which are specifically designed to offer IPL/E opportunities for qualified professionals.

2.2.2 Key trends - Quality of learning opportunities

29 Strong partnership working is evident throughout the quality of learning opportunities sections of the reports, irrespective of discipline. Examples of strengths identified include: the use of link tutors to strengthen student and clinical assessor support, and assist in developing an optimum learning environment in placements; clinical assessors maintain good relationships with academic staff and facilitate integration of theory and practice; and the implementation of a coherent approach to the development of teaching and dissemination of good practice (three reports) across academic and practice settings. Only one example is given of an HEI and NHS Trusts not working fully in partnership in the dissemination of programme information at the point of recruitment to a programme.

30 Five reports indicated that students from a range of disciplines and academic levels considered the quality of programmes, teaching and support to be positive and responsive. Two reports note particularly the existence of well-structured mechanisms for recording and responding to feedback from students. A number of good practices are also identified in this area, including the provision of daily feedback to students by clinical assessors, considerably enhancing the learning experience, and the effective dissemination and implementation of actions agreed as a result of student feedback. Patient-centred learning is clearly evident in many programmes, although user and carer involvement in learning and teaching is not always used widely. Common weaknesses documented in several reports are the absence of standard processes for return of work to students, and little or no feedback to placements.

31 A variety of routes to registration is noted in several reports, in addition to effective selection, interview, admission and induction procedures. Effective recruitment from a diverse range of backgrounds is noted (three reports), along with new access opportunities in occupational therapy and physiotherapy programmes, and effective access and widening participation strategies and specialist initiatives to address the needs of students with diverse cultural backgrounds in nursing and midwifery programmes. Also noted are high application rates and the development of mechanisms to manage growth in student numbers and diversity supported by varied progression opportunities for students with financial or personal difficulties and effective pastoral support. Seven reports acknowledge high or significantly improved retention rates across all disciplines. Three features of good practice cited are collaboration between an SHA and further education colleges to widen access opportunities, support systems to manage the transition to higher education, and an example of an HEI working closely with the SHA to address attrition and recruitment difficulties. Five reports note high attrition rates, three of which relate to nursing, and two to allied health professions. One report notes limited tracking in areas of student attendance and failure.

32 The reports document a high standard of placement learning which is well supported by the HEIs, placements and SHAs. Placements are mostly utilised effectively; placement staff respond to available feedback in developing and improving the learning environment and make a significant contribution to the programmes, particularly in health visiting, midwifery and nursing. This is supported by excellent placement monitoring and clear guidance documentation for students. More weaknesses are noted across a range of allied health professions, including a lack of placement opportunities (in three reports), a limited range of placement experience available on several programmes in one report, and poor-quality or limited accommodation within placements in another four reports.

33 Clinical assessors are again identified as an important resource in facilitating and supporting students effectively in placements. Two points of good practice are an SHA clinical assessor award scheme which enhances and adds value to the role, and the maintenance of an audited professional portfolio detailing the clinical assessor's clinical, teaching and assessing skills, updates and CPD. However, areas of weakness noted particularly in health visiting, midwifery and nursing, include a shortage of clinical assessors, difficulties in keeping clinical assessors up-to-date with curriculum developments due to clinical workloads, inconsistent clinical assessor training and/or briefing, and a heavy burden on clinical assessors due to large numbers of students in practice placements.

34 Several reports acknowledge the use of experienced and expert academic staff to benefit teaching and research. Two weaknesses are noted: some clinical psychology students did not feel that the experience, skills and availability of staff met their doctoral research needs, and in one recorded instance an NHS Trust's research ethics committee is deemed to hinder research efforts with bureaucratic procedures and restricted research topic options for students.

35 Strengths in other areas of learning resources are evident across all disciplines, but more so in health visiting, midwifery and nursing. Information technology (IT) facilities and virtual learning environments such as WebCT, Blackboard and other e-learning environments are noted as effective learning and teaching tools, and helpful in developing interprofessional learning/education. Several examples of good practice are noted in relation to the development of facilities for interprofessional learning.

36 Well-equipped skills laboratories and teaching rooms are a common strength across many programmes. Library facilities are generally considered excellent, although some students found difficulty in accessing library and IT facilities on placements. Good practice cited includes a book delivery service to students on placement, interactive CD-ROMs, and a resource centre available for individual study outside teaching time. Very few reports note a general lack of appropriate resources. However, four reports note that, for return to practice nursing students, weaknesses included a lack of open learning methods, minimal departmental response to problems on the programme, and limited opportunities for students to use skills laboratories.

37 There is an excellent provision of support from clinical and academic staff for students, including specific provision for students with disability or English language needs. A wide variety of effective support systems and services is in place. Good practice includes clear guidelines which outline available support mechanisms, special needs appropriately assessed and supported, and an innovative scheme to support practice learning which is being promoted in collaboration with local providers. Of the few weaknesses identified, one report notes an increasing pressure on staff to maintain support due to a shortage of qualified clinical assessors, and another that activating formal support arrangements for students with a learning disability can take time.

2.2.3 Key trends - Maintenance and enhancement of standards and quality

38 A key strength identified in the reports is the good, well-established and effective working relationships between HEIs, SHAs and practice placement providers, in this case, in relation to ensuring that programmes meet the needs of local and national stakeholders, including the recruitment and retention of students, and that monitoring and evaluation processes are undertaken appropriately and effectively.

39 Effective arrangements for MESQ are evident from the reports, although more so in the allied health professions than in health visiting, midwifery and nursing. The arrangements include systematic, coordinated and effective procedures for monitoring, reviewing and enhancing programmes and processes, including annual monitoring and external examiner reports; a range of mechanisms for responding to issues identified through evaluation to improve quality of provision including, in one report, external commissioning of course evaluation; flexible committee structures able to bring about change quickly; programme teams that are responsive to the needs of students and trainees, practitioners, employers and commissioners; and support from the purchasers in developing the programmes and funding enhancement activities. The reports note several examples of good practice, including a quality assurance maintenance and enhancement unit which has developed a website detailing their approach to quality assurance and arrangements for supporting its procedures, high-quality documentation and effective arrangements for ensuring that quality assurance procedures are followed, and joint structures and processes established with all partners concerned. Weaknesses identified include some lapses at school level in the implementation of otherwise rigorous procedures, a lack of clarity about the procedures for updating modules and subsequent documentary evidence, the need to introduce consistent and routine monitoring of contracts, programmes and clinical educator training where this is not already in place, and the failure to maintain fully the live mentor register in relation to health visiting, midwifery and nursing.

40 Generally the provision is open and responsive to evaluation and feedback (slightly more so in health visiting, midwifery and nursing than the allied health professions) but the following weaknesses are noted in several reports across all disciplines: not keeping formal records of student evaluations; a lack of effective mechanisms for obtaining feedback from clinical assessors; and a lack of systematic sharing of evaluation outcomes between some schools and practice placements.

2.3 Data arising from major review

41 Major review reports contain three tables providing data: Table 1 provides data on completion and achievement statistics for all award-bearing programmes for 17,674 students; Table 2 provides employment statistics for pre-registration programmes and post-qualifying programmes (by exception reporting only) - 8,408 students; and Table 3 presents recruitment and attrition statistics for pre-registration and NMC recordable qualifications - 19,620 students. Due to the different nature of each table, the programmes it relates to, and the way in which the data are presented, direct comparison between the tables is not possible.

42 In 2002-03 there was an expectation that there would be an agreed national minimum dataset in relation to WDC/SHA commissioned programmes. This dataset would enable comparison of data between organisations, as it included definitions of the terms used, such as attrition. In order for the major review reports to contribute to the published data, the major review steering group agreed the data that should be published in the reports. At the time of publication, there is still no agreed published minimum dataset. The fundamental requirements for the reports have not changed since January 2003; however, additional guidance has been developed as providers work to provide the data required for the report which has gone some way to address the issues raised in the first annual review trends report of inaccurate or incomplete data. However, the amount of work required is highly dependent upon the systems agreed between the SHA and the HEI(s) for that part of England; there is little apparent consistency between SHAs and HEIs.

43 The following sections explore in more detail the analysis of the data and, where appropriate, makes reference to sector-wide data for England drawn from the Higher Education Statistics Agency (HESA). However, this contains a note of caution: the time period that the data covered varies by programme but ranged from 1998 to 2004 across all programmes. It is also not always clear from the data if the year given refers to the start date of the cohort, or if the data relate to all students studying in that year.

44 Providers were asked to submit the data for the three tables by discipline and, within each of these, by programme, in order to be meaningful against the judgements made in academic and practitioner standards which are by discipline. However, this causes a tension for the increasing number of interprofessional programmes developed in line with DH policy and service initiatives. The emphasis on discipline-focused judgements in academic and practitioner standards means that the reviewers are required to allocate interprofessional programmes to one or more discipline. The criterion for such allocation is the discipline from which the majority of students on the programme are drawn. The quantitative data guidance given to the providers and the reviewers are not able to give due regard to the extensive and innovative range of interprofessional learning courses/programmes now being delivered in England. Any future cycles of review must be structured in such a way that interprofessional learning programmes/courses are recognised and celebrated. Where it has been possible to identify interprofessional learning programmes from the data tables, these have been clustered into a separate group.

2.3.1 Completion and achievement statistics

45 Analysis of the completion and achievement statistics indicated that the predominance of students were registered on diploma (mostly nursing and a few midwifery) programmes (6,122) followed by (allied health professions, midwifery and nursing) degree programmes (9,043). There were 792 students on postgraduate programmes, including 213 studying clinical psychology and 33 on nursing professional doctorates; 444 students were recorded on 'interprofessional' programmes and 209 on conversion courses. In nursing and midwifery, 917 students were registered on return to practice programmes.

46 Around 70 per cent of all students had enrolled on nursing programmes, with the majority on advanced diplomas, diplomas, conversion programmes and return to practice programmes. Midwifery had the second highest number of students registered, with approximately one-half registered on diploma programmes and one-half on degree programmes. Only a very small percentage were registered on a postgraduate award. It is also interesting to note that there is significant variation in the number of nursing students by institution. One HEI/SHA accounts for one-fifth (19 per cent) of all nursing students and two HEI/SHAs both recorded in excess of 1,000 students, together accounting for 28 per cent of all nursing students. The HESA data also confirm the variation in the number of students enrolled on nursing degree programmes within England, with six HEIs each registering over 1,000 students enrolled on nursing programmes.

47 The allied health professions programmes had 19 per cent (3,332 students) registered largely on degree programmes. The highest number of students completing programmes is in physiotherapy (868, 26 per cent), radiography (757, 23 per cent) and occupational therapy (683, 21 per cent). The smallest professions were audiology (0.1 per cent) and operating department practice (0.3 per cent).

48 This pattern in the size of disciplines for the reviews, in terms of student numbers, mirrors the data from HESA (2003-04) for the higher education sector as a whole, with the preponderance of nursing and midwifery students on sub-degree programmes. Apart from clinical psychology, which is only studied at doctoral level, the discipline with the greatest proportion of postgraduate students was speech and language therapy which had 36 per cent of its students registered at postgraduate level. Although nursing had the largest number of students studying at postgraduate level, these only represented 3 per cent of the students registered on a nursing programme.

2.3.1.1 Diplomas and advanced diplomas

49 There were a total of 9,043 students recorded for advanced diploma and diploma programmes, of which 90 per cent were recorded against nursing programmes and a further 10 per cent for midwifery programmes. Advanced diplomas generally refer to the programmes that enable diplomates to gain honours level credit, as well as the intermediate award. This is commonly because the students of pre-registration nursing study for three years to gain a diploma (a set number of curricular hours is a requirement for some professions). On allied health professions programmes, 22 students (0.2 per cent) were awarded diplomas. HEIs are very careful to provide recognition of academic achievement even if students have not been able to meet the criteria for registration.

50 Over 8,000 students were enrolled on diplomas and advanced diplomas within nursing (excluding those on interprofessional programmes). Just under one-quarter of these were registered at one institution. Overall, there was a pass rate of 95 per cent for nursing students and 93 per cent for midwifery students. Although, for the majority of nursing programmes, the pass rate was in excess of 98 per cent, one institution noted a low pass rate, in relative terms, of 78 per cent (note that there were just under 400 students enrolled on this programme). This one institution accounted for 22 per cent of all nursing diploma failures.

Table 2 Number of students studying by discipline and qualification aim

	Diplomas		Degree classification							Post-graduate		Inter-professional		Return to practice		Conversion
	Pass	Fail	First	2i	2ii	Third	Pass	Fail	Total	Pass	Fail	Pass	Fail	Pass	Fail	Pass	Fail
Audiology			0	1	2	0	0	0	3
Clinical psychology									0	209	4
Diagnostic and therapeutic radiography			69	263	277	53	3	0	665	92
Dietetics			11	38	5				54	26
Occupational therapy	11		48	262	308	49	2	2	671	1
Operating department practice	10								0
Orthoptics			11	55	58	12	0	0	136
Paramedic science			8	52	19	3	9	24	115	50
Physiotherapy			97	516	239	9	2	0	863	5
Podiatry			23	65	55	10	0	1	154			2
Prosthetics and orthotics	1		16	37	10	2	0	0	65			1
Speech and language therapy			10	79	34	1	0	0	124	70
Health visiting			35	213	176	17	5	0	446			8
Midwifery	843	66	100	332	223	49	5	20	729	5
Nursing	7,722	390	204	902	720	116	109	46	2,097	321	9	421	12	824	93	206	3
Total	8,587	456	632	2,815	2,126	321	135	93	6,122	779	13	432	12	824	93	206	3

Note: Certificate nursing students (147) have been excluded from the data.

Table 3 Diploma outcomes (including Diploma, Diploma of Higher Education and Advanced Diploma)

Diplomas	Pass	%	Fail	%
Midwifery	843	93	66	7
Nursing	7,722	95	390	5
Total	8,587	95	456	5

 

Figure 1 Degree classifications across disciplines in major review

 

Source: Major review of healthcare programmes reports

2.3.1.2 Degree outcomes

51 Students were recorded as studying at degree level in all discipline areas except operating department practice (diploma) and clinical psychology (professional doctorate). Across the 13 disciplines with degree programmes, the highest proportion of students, 46 per cent (2,815), achieved an Upper Second class, with slightly fewer than 35 per cent achieving a Lower Second class. One-tenth of all students studying for a degree achieved a First class degree (see Figure 1). The proportion of failures recorded was very low, at only 2 per cent (93 students). Students failing were only recorded in five of the 13 disciplines.

52 If this is compared to the HESA data (2003-04) in respect of all higher education students studying on a degree programme, the profile of degree classifications is very similar.

Table 4 Comparison of data from major review and HESA on degree classifications

Degree classification	Data arising from major review	HESA data the all higher education students (2003-04)
	%	%
First	10	10
Upper Second (2i)	46	44
Lower Second (2ii)	35	31
Third or Pass	7	7
Unclassified or Fail	2	8

53 The 'best performing' disciplines, in terms of the percentage of First and Upper Second class degree classifications received were: dietetics (91 per cent), prosthetics and orthotics (82 per cent), speech and language therapy (72 per cent) and physiotherapy (71 per cent) (see Table 5 below). However, this does not take any account of entry qualifications and value added to students of undertaking a programme.

Table 5 Proportion of students by degree classification and discipline

	Degree classification
	Total	First	2i	2ii	Third	Pass	Fail
	%	%	%	%	%	%	%
Dietetics	54	20	70	9	0	0	0
Prosthetics and orthotics	65	25	57	15	3	0	0
Physiotherapy	863	11	60	28	1	0	0
Speech and language therapy	124	8	64	27	1	0	0
Midwifery	729	14	46	31	7	1	3
Health visiting	446	8	48	40	4	1	0
Nursing	2,097	10	43	34	6	5	2
Podiatry	154	15	42	36	7	0	1
Paramedic science	115	7	45	17	3	8	21
Diagnostic and therapeutic radiography	665	10	40	42	8	1	0
Orthoptics	136	8	40	43	9	0	0
Occupational therapy	671	7	39	46	7	0	0
Audiology	3	0	33	67	0	0	0

Note: Due to the small numbers of students in audiology they have been excluded from the commentary below.

Note: Where totals do not add up to 100 per cent, this is a result of rounding to the nearest integer.

Figure 2 Degree classification by discipline

Source: Major review of healthcare programmes reports

54 The discipline area that attained the highest number of graduates awarded a First class degree was podiatry, with 15 per cent (23 students) achieving this level compared to the average of 10 per cent across all disciplines. Paramedic science, which was reviewed at only one institution, recorded the lowest number of students awarded a First class degree, only 7 per cent (eight students). Paramedic science also recorded the highest number of students who failed their degree, at 21 per cent (24 students) compared to the average of 2 per cent. Paramedic science also performed below average in the categories of awards for Upper Second class, Lower Second and Thirds. However, this is a relatively new discipline and the provider reviewed so far is one of the leading HEI/SHA partnerships in establishing the provision at degree level and developing a sufficiently-sized pool of practice staff who are conversant with curriculum development, assessment and supporting students.

55 Although speech and language therapy had slightly fewer students gaining a First class degree, a larger proportion than average attained an Upper Second, with nearly 64 per cent (79 students) gaining this qualification compared to the average of 46 per cent.

56 Midwifery achieved a higher than average number of students attaining a First class degree - 14 per cent (100 students), and slightly below average numbers of students gaining a Upper Second or Lower Second. The number of students who failed the midwifery degree was slightly above average at 3 per cent (20 students, 17 of whom were recorded from two institutions).

57 Nursing programmes recorded slightly lower than average proportions of students gaining a First, Upper Second and Lower Second class degree. However, more than double the average proportion of students were awarded a Pass, 5 per cent (109 students) compared to the average of 2.2 per cent. It should be noted that many of the initial pre-registration degree programmes in nursing were not at honours level and, therefore, only had Pass or Fail as an outcome, whereas later programmes were at honours degree allowing for degree classifications. This will inevitably have skewed the dataset with regard to the number (and proportion) of students obtaining a Pass, but it is not possible to determine the extent of this impact. A higher proportion of students studying nursing at degree level failed compared to the average across all programmes.

2.3.1.3 Postgraduate programmes

58 Students studying postgraduate programmes were recorded in nine discipline areas. The total number of students studying for a postgraduate qualification was 792, with the highest proportion of these (42 per cent) studying nursing, but this represents only 3 per cent of all nursing students. The professional doctorate in clinical psychology is only available at the postgraduate level.

59 Of all postgraduate students, only 13 were recorded as failing, a pass rate of 98.4 per cent. This may be more a reflection of the flexible nature of postgraduate programmes, both in delivery and completion, and does not take account of the number of students withdrawing from programmes or those accessing modules only (see below).

2.3.1.4 Other programmes

60 Within nursing, there was an array of programmes available. Just over 200 students undertook Enrolled Nurse to Registered Nurse conversion programmes. The pass rate for conversion was 99 per cent.

61 Just fewer than 1,000 students undertook return to practice programmes. The pass rate for these programmes was 90 per cent. However, 61 per cent of students were enrolled at one institution, which had the lowest pass rate for the return to practice programmes (84.2 per cent); this significantly skewed the overall pass rate. All other nursing return to practice programmes recorded pass rates over 98 per cent.

2.3.2 Employment statistics

2.3.2.1 Employment

62 Data provided in Table 2 of the major review reports describe the employment following attainment of their qualification for students on pre-registration and post-qualifying programmes (by exception reporting only). The data are based on a total of 8,408 students and 14 discipline areas.

63 The largest proportion of students across all discipline areas, 53 per cent (4,446 students), was recorded as being employed by local employers (Figure 3), with almost 60 per cent of these having undertaken a nursing qualification. No clear definition of 'local' had been provided, and there may have been scope for different interpretations of this category by respondents. The proportion of students who were recorded as unemployed across all disciplines was 3 per cent (270 students), with 20 per cent assigned to the 'other' destination (including voluntary work and travelling abroad) or 'unknown'.

Figure 3 Employment status following qualification attainment across all disciplines

Source: Major review of healthcare programmes reports

 

64 There was an apparent variation in student mobility dependent on the level of programme studied and the discipline area (see Table 6). It is possible that students already being employed within local Trusts and being released part-time to study may account for such variability. Other influencing factors may be local workforce needs and individual students' personal circumstances. For some disciplines, for example, pre-qualifying physiotherapy, applicants have to be prepared to move from their locality to study. On graduation, they may well return to their home area.

65 From analysis of discipline areas where more than 100 students were recorded (to make the data meaningful), over 93 per cent of health visiting students were employed locally following qualification. Similarly, in midwifery, the majority of students are employed locally. In contrast, only 11 per cent (11 students) of students studying orthoptics, and just slightly less than one-quarter (47 students) of students studying speech and language therapy were employed locally following graduation, with the majority being employed elsewhere. Although paramedic science had fewer than 100 students recorded, all the students were employed locally. It is interesting to note that only just over 50 per cent of nursing students are employed locally, irrespective of the type or level of programmes.

Table 6 Proportion of students by destination and discipline

Total	Further study	Local employers	Employers elsewhere	Unemployed	Other/unknown
	%	%	%	%	%
Audiology	Not applicable
Clinical psychology	226	0	62	33	0	5
Diagnostic and therapeutic radiography	507	1	58	30	1	10
Dietetics	21	0	48	14	10	29
Occupational therapy	507	1	62	21	1	15
Operating dept practice	10	0	100	0	0	0
Orthoptics	98	8	11	72	0	8
Paramedic science	72	0	100	0	0	0
Physiotherapy	772	0	33	50	4	13
Podiatry	147	2	54	20	9	15
Prosthetics and orthotics	49	0	6	80	6	8
Speech and language therapy	191	0	25	45	1	29
Health visiting	91	0	93	6	0	1
Midwifery	705	0	67	14	2	18
Nursing	5,012	2	53	17	4	24

2.3.2.2 Unemployment

66 Across all disciplines, the overall average for unemployment was 3 per cent (see Table 6 above). Podiatry recorded the highest proportion of unemployment at 9 per cent (13 students) and these were all accounted for by one institution. A higher than average proportion of unemployment was also recorded for students studying on a nursing return to practice course, with 6 per cent (17 students) of students. Due to the large number of nurses studying for a diploma, the number recorded as unemployed following qualification was 178 students; this represented 4 per cent of all nurses recorded as studying for a diploma.

2.3.3 Recruitment and attrition statistics

2.3.3.1 Recruited

67 Table 3 of the major review reports provides data regarding the recruitment and attrition statistics for all pre-registration programmes and NMC recordable qualifications. Data were provided regarding the number of students who registered, withdrew, discontinued and transferred in and out of programmes. The data supplied by one institution were excluded from the analysis as only the number of cohorts and average cohort withdrawals were provided, making the data incompatible.

68 Without the Standard National Contract being published it was considered more helpful to providers to allow them to define each of the categories in the table using their own existing internal definition, rather than impose a set of definitions that might be redundant on publication of the Contract. As a guide during the briefing sessions and workshops, or on request, a broad definition of withdrawn and discontinuation was offered by QAA: 'withdrawn' - where a student has instigated leaving the programme, 'discontinued' - where the HEI had instigated the student leaving the programme.

69 The number of students recruited varied by institution, discipline, programme and type of programme. The data tables indicated that a total of 19,620 students had been recruited, with just over 60 per cent enrolled on nursing programmes (as indicated through the analysis of the Table 2 achievement data).

Table 7 Recruitment and attrition

---

	Number	%
Recruited	19,620
Withdrawn	1,829	9
Transfer in	1,339	7
Transfer out	1,221	6
Discontinued/interrupted	887	5

Note: All percentages are based upon the recruitment data.

70 The average withdrawal rate across all the relevant programmes in these reviews was 9 per cent (of the initial recruitment) (see Table 7 above), with an additional 7 per cent transferring into the programmes and 6 per cent transferring out. In the latter two cases it is not possible to ascertain whether those students transferring did so within the same discipline.

Table 8 Recruitment and attrition by discipline

Discipline	Recruited	Withdrawn	Transfer in	Transfer out	Discontinuation/interrupted
		%	%	%	%
Audiology	21	10	0	0	0
Clinical psychology	492	0	0	0	0
Diagnostic and therapeutic radiography	1,104	11	5	5	1
Dietetics	128	8	3	2	0
Health visiting	647	10	4	0	1
Midwifery	1,392	14	4	4	5
Nursing	11,558	10	9	9	5
Occupational therapy	1,116	11	2	1	4
Operating department practice	205	15	0	1	1
Orthoptics	195	16	9	5	2
Paramedic science	157	14	7	12	0
Physiotherapy	1,240	5	4	4	1
Podiatry	297	1	5	5	2
Prosthetics and orthotics	90	12	2	9	1
Speech and language therapy	271	6	1	3	0
Other	707	0	0	0	22

Note: Those programmes listed as 'other' were not set against a programme or discipline area within the published review reports.

Table 9 Attrition rates by type of programme

Discipline	Recruited	Withdrawn	Transfer in	Transfer out	Discontinuation/interrupted
		%	%	%	%
Interprofessional	1,119	8	4	2	3
Degree	7,207	10	4	5	3
Professional doctorate	4,92	0	0	0	0
Postgraduate	642	7	9	3	6
Return to practice	485	7	0	0	1
Conversion	336	6	1	1	4
Diploma	9,030	10	10	9	7
Other	309	0	0	0	0

Note: All percentages are based upon the recruitment data.

2.3.3.2 Withdrawal

71 Across all programmes for which information was provided, the average rate of withdrawal was 9 per cent (see Table 8 above). There were five discipline areas (out of the 15, excluding the 'Other' category) that had withdrawal rates in excess of 12 per cent. Orthoptics reported the highest withdrawal rate at 16 per cent, operating department practice a rate of 15 per cent, midwifery 14 per cent, with around one in seven students withdrawing from the programme, and paramedic science had a withdrawal rate of 14 per cent. Clinical psychology had the lowest rate of withdrawal across all the disciplines, at 0.2 per cent, followed by podiatry at 1 per cent.

72 On the basis of level of study, withdrawal rates were highest for diploma programmes (10 per cent), compared to 10 per cent for degrees (see Table 9). The midwifery diploma had the second highest withdrawal rate at 17 per cent (note: the highest rate was for audiology CPD, at 20 per cent, but numerically this is insignificant, as there were fewer than five students on the programme).

2.3.3.3 Transfer in/transfer out

73 There was some variation across both disciplines and levels of programmes in respect of the rates and numbers of students who transferred in or out of the programme. Nursing and orthoptics had the highest transfer in rates (9 per cent). Diplomas and postgraduate programmes had the highest transfer in rates at 10 and 9 per cent respectively. Much of this transfer in was in relation to nursing programmes.

74 The highest transfer out rate was for paramedic science at 12 per cent, which was almost double the average rate. With regard to the transfer in and transfer out of students, it is not known whether transfer was between levels within the same discipline or across disciplines within or outside of healthcare disciplines.

2.3.4 Summary comments

75 The analysis of the data provided in the reports indicates that there is no significant difference between the disciplines reviewed or the programme levels, diploma, degree or postgraduate, in relation to completion and achievement statistics for all awardbearing programmes; employment statistics for pre-registration and post-qualifying programmes (by exception reporting only); and recruitment and attrition statistics for pre-registration and NMC recordable qualifications.

76 The analysis of the data provided in the major review reports in the three tables indicated that there were no statistically significant differences to emerge on the basis of level or discipline for student achievement, employment, recruitment and attrition. In this respect, over two-thirds of students were reportedly registered on nursing programmes and around 10 per cent on midwifery programmes. Around half of all students were enrolled on diploma courses and just over one-third on degree programmes. Two out of the 28 institutions accounted for around one quarter of all students for whom data were provided. This notion of primacy will have inevitably skewed the subsequent analysis of the data, so although patterns emerged it was impossible to elucidate which were the principal controlling factors.

77 In respect of achievement levels nursing degrees and paramedic science programmes, the early pre-registration degree courses allowed for a Pass/Fail outcome, and not a honours classification; therefore, the comparison is not on a like-for-like basis and the pattern of distribution is skewed. Time-series information and information regarding the nature of the degree programme (in respect of possible outcomes) would have been necessary to analyse degree outcomes in relation to all higher education programmes and other healthcare degree programmes, or between programmes or institutions. However, there does not appear to be any clear difference in attainment levels between allied health professions programmes, and health visiting, midwifery and nursing programmes.

78 The employment data revealed emerging patterns of mobility after qualifying or graduating in relation to both discipline and level. Although in some discipline areas those students obtaining a diploma appeared to have the greatest spatial mobility with respect to employment, the converse was the case in other discipline areas, where degree or other programme graduates were most likely to obtain employment outside the local area. In general, greater mobility was seen in students completing diploma and degree programmes in the allied health professions, with the exception of paramedic science, than in health visiting, midwifery and nursing. It is possible that emerging patterns of employment may be related to the local economic conditions, the demand for employees with certain skills levels, pre-qualifying students who had to relocate in order to study returning to their local area, or the personal and family circumstances of the individual students.

79 On the whole, there was a greater variation in the rates of withdrawal from programmes of study on the basis of discipline than of level. In relation to the movement of students between programmes, it is not known whether this movement was in respect of level of study or discipline of study, or even within/outside of the general healthcare programmes discipline areas.

2.4 Action plans

80 The action plan is the formalised response by the partner providers to the points raised in the major review report in relation to strengths, good practice and weaknesses in academic and practitioner standards, the quality of learning opportunities and MESQ. The plan outlines the actions that the providers are committed to undertake, the dates for completion, the constraints preventing delivery, the impact of not delivering the action required, the person responsible for overseeing implementation of the action and the evidence that will demonstrate if quality enhancement has taken place as a result of the action taken. The action plan must be developed and agreed by all partners and be signed off at a senior level by all SHAs/WDCs involved in the review, as well as the Vice-Chancellor/Principal of the university/college. The implementation of the action plan is monitored by the lead SHA and the HEI, and the action plan will form a vital part of the evaluation of progress at approval or monitoring events, undertaken by the SHA and/or the relevant professional/statutory body and the HEI.

81 The actions plans have been welcomed by the providers and stakeholders in providing a vehicle for continued quality enhancement, although it is recognised that there have been one or two logistical issues. Nevertheless, it is recommended that actions plans continue to feature in any future periodic review cycle for healthcare education provision.

82 The potential value of action plans has been highlighted in the prototypes for the OQME process. One of the sites had undergone major review immediately prior to the prototypes and was able to utilise the action plan as an important source of evidence in the self-evaluation against the standards (the first stage of OQME) to demonstrate achievement of the standards and quality enhancement and confirm actions taken in response to the major review.

83 For this report the published action plans were considered under each column heading in the plan, firstly as a whole, looking at common themes and responses emerging from all of the plans, and then on a term-by-term basis, looking for developments and trends emerging through time. The following sections explore the findings for each of the column headings.

84 Of 26 reviews where action plans have been published, nine took place in summer 2004, six in autumn 2004 and 11 in spring 2005. Over this time period, the average number of bullet points to be considered in each report reduced by 67 per cent. This is seen as a positive response to concerns raised by HEIs and SHAs about the number of bullet points to which they needed to respond. It also reflects the discontinuation of stating a strength when the issue was a requirement for approval of the programme and so a requirement for a confidence judgement.

2.4.1 Actions to be taken

85 The bullet points outlining strengths and good practice are largely dealt with by an action to 'maintain' or 'continue'. Such actions include disseminating information and good practice (such as quality enhancement and communications policies, monitoring and review systems etc) with other colleagues, institutions and partners, and regularly reviewing, refining and updating these processes. Another frequent action was to extend good practice, identified in a short-term or pilot event, for a longer timespan or to a wider area of reference, such as the rest of the department, the HEI, across the practice partners, or SHAs/WDCs.

86 The providers also aim to make course information and policies more explicit, clear and accessible for students, staff and practice partners, through the use of IT and/or revised and developed documentation. Some strengths are to be developed or enhanced through revalidation or evaluated through quarterly or annual monitoring processes.

87 Little information was provided on how good practice is to be enhanced or disseminated, but a few reports do provide examples, such as through staff development workshops, inter-HEI meetings, regional networks, conference presentations and publications. Actions to enhance service-user involvement include developing a strategy and policy document, and sharing good practice through countrywide learning schemes.

88 Weaknesses are given more specific, targeted actions. Generally, these are to review the issue in hand, with further or more regular or detailed monitoring, through meetings with staff, committees, and reports; to develop and improve staff and practice educator performance through clarifying roles and responsibilities, documentation or procedures; to increase staffing and/or administrative support, or to increase staff availability to attend briefings and updates.

2.4.2 Target completion dates

89 For the bullet points relating to strengths and good practice, target completion dates are largely either ongoing, or to be reviewed annually through existing monitoring processes or committees. On the other hand, weaknesses are mostly given specific target dates by which the action is to be completed.

90 Looking at the whole data, an average of 60 per cent of strengths are marked as ongoing, across each section of the report, as opposed to an average of 40 per cent with a deadline set. However, considering weaknesses alone, there is an average of 80 per cent of actions with a specific date set, as opposed to around 20 per cent ongoing. Weaknesses achieve the most amount of actions completed, although this is still very small overall (2 per cent of all action points stated), and the majority of these fall within academic and practitioner standards. In good practice, the averages are more similar, with around 45 per cent of actions ongoing and 55 per cent with a specific target date. Notably in good practice, significantly more specific dates have been set against the area of student progression (73 per cent) than in other areas.

91 Over the period of the reports analysed, there has been a notable decrease in the use of 'ongoing/review annually' (a drop of 44 per cent) and a corresponding increase in completion dates set. QAA, in response to its own concerns about the usefulness of the action plans, provided additional guidance through briefings, workshops and the QA Newsletter (a quarterly publication distributed to all HEIs, SHAs, Trusts, reviewers and review coordinators). Only one action point had been completed, by publication of the review report. The term 'ongoing' was used significantly less in addressing weaknesses, only 16 per cent of the time, whereas a future date had been set for 82 per cent of actions. Only 2 per cent of actions had been completed. The trend here is for a small reduction in 'ongoing' (approximately 11 per cent) and a similar small increase in future dates set. Like strengths, good practice showed only one action completed, but a reduction in the use of 'ongoing' over time with an increase in future dates set (60 per cent), and 10 per cent more overall future dates set as opposed to ongoing. The increased setting of completion dates reflects the greater recognition that, without such dates, the action plan becomes redundant as a source of evidence in the other two process elements of the proposed PQAF, and meaningless for the annual monitoring process by the SHA.

2.4.3 Constraints preventing delivering the action required

92 Key constraints identified in all of the reports were resources, staffing and partner organisations. In terms of resources, the main issues are either a change in availability or lack of resources, including funding, IT provision and teaching accommodation. For staffing, the major constraints identified are an inability to provide appropriate staffing levels, a lack of staff enthusiasm or difficulty in attending meetings/briefings/training events, and staff workload and time available for development, balanced with time available for teaching and practice. Constraints linked to partners include a lack of practitioner support due to increased workloads or changing priorities, limitations on accommodation space, IT provision and diverse geographical location of placements. Other constraints listed include the changing political agenda and difficulties in coordinating all stakeholders' availability to meet, discuss and implement change.

93 The most common response under this column heading, however, particularly in earlier reports, was 'no constraint', given by 56 per cent of all responses.

94 Resource issues totalled 44 per cent of constraints identified, with the majority of these in the area of academic and practitioner standards (around 40 per cent of all resource-related constraints). Staff-based issues formed around 31 per cent of all constraints identified, again with the majority of these falling against academic and practitioner standards (62 per cent of staff-based constraints). The areas for action with the least constraints identified were in student progression (less than 10 per cent) and MESQ (6 per cent).

95 Over time, the frequency of a constraint being identified has increased considerably, with no constraints identified 68 per cent of the time in summer 2004, compared to 20 per cent in spring 2005. This might, in part, be as a result of better focused and more actionable bullet points devised by the reviewers.

2.4.4 Impact of not delivering the action required

96 Most action plans focus on a negative impact of not delivering the action required on student support, learning experiences and, in some instances, progression. Negative impacts on staff development, workload and availability for meetings are also noted frequently, as are limited or lost opportunities for enhancement and development activities, or sharing good practice with other departments or placements. Impacts on the programmes focus on whether programmes might lose currency, not reflect market needs, lose credibility or increase the risk of producing students who would not be fit for purpose.

2.4.5 Lead responsibility

97 Lead responsibility for undertaking actions was shared more or less equally between the HEI alone (47 per cent), and a number of stakeholders working together (46 per cent). Few actions were identified where the practice placement providers alone take the lead responsibility (7 per cent). Any identified relate mainly to enabling clinical staff to be available for training and updates, or improving teaching accommodation within practice environments. This might be as a result of the academic focus in many of the bullet points, which by its very nature requires responsibility/leadership from the HEI.

2.4.6 Evidence of quality enhancement

98 Evidence of quality enhancement largely details sources of evidence, rather than the evidence itself, although this is probably because most action points have not been completed at the time of publication and, therefore, evidence of actual achievement is not available.

99 The most common sources of evidence listed are student evaluations, annual monitoring reports, committee minutes and external examiners' reports. Progression, results and diversity statistics are also cited as evidence of achievement, as well as having the monitoring of key issues as a standard agenda item for relevant committee meetings. Positive feedback from employers is also noted, as are annual monitoring by regulatory bodies and reviews by SHAs/WDCs.

100 Physical evidence, such as increased staffing, more equipment, new policies and procedures implemented, or students evidencing improvement of knowledge and skills through assessments, are very few in number, again, most likely due to the short timespan in which the action plan is completed.

101 Some examples of specific evidence of enhancement include practice educators expressing greater clarity and confidence about their role; a reduction in problems raised by students; greater numbers of practice staff attending workshops; continued positive feedback from students and employers on the quality of graduates; results of action research published and acted upon; and successful validation of new programme documentation.

 

Chapter 3 Processes

102 Chapter 3 aims to capture the developments in the review processes and procedures, a challenging requirement given the continually changing context within which major review operates. A summary of the changes is in Appendix 2.

3.1 Pre-review

3.1.1 Scoping and scheduling

103 The original scoping exercise with all the HEIs/SHAs in planning for major review took place in April 2003 and, as expected, did not remain static. In July 2004, four additional professions were included within the scope of major review: audiology, clinical psychology, operating department practice and paramedic science. QAA spent some significant time with providers clarifying the scope. Occasionally, this not only identified new programmes within the provision but also that the providers had developed programmes in new disciplines; this has a direct impact on the composition of the review teams and the number of reviews taking place. In the reviews that took place between January 2003 and spring 2005, the average number of disciplines in a review was three: the minimum one (seven reviews) and the maximum seven disciplines. This latter provider was advised by the QAA that including seven disciplines in one review would be extremely challenging for all involved and meant that a team of 12 reviewers was necessary, with two review coordinators managing it. Feedback after the review from all participants confirmed that they would support this advice and it was agreed with SfH that six disciplines is the maximum number per review. Where there were more than six disciplines in the provision, two reviews would be undertaken.

104 Major review has a common intensity of approach; that is, five days of review with, normally, two reviewers from each discipline. Feedback from the evaluation of the reviews (questionnaires and focus groups) indicate that providers with only one discipline going through major review consider this approach too heavy for the size of the provision. In two instances, the review was reduced to four days; however, feedback from the reviewers confirmed that, as all elements still had to be addressed, time is needed. Additionally, four days offered no opportunities for reflection and writing drafts prior to the judgement meeting. It is suggested that, in any future cycle of review, the intensity of scrutiny is proportionate to the size of the provision and the level of risk attached to it.

105 Providers are notified of the proposed dates for their major review approximately six months prior to the term in which the review will take place, and invited to agree the dates or engage in further discussion with QAA. The system of allocation has been adapted as the major review cycle has progressed, because of the sheer number of reviews taking place in some terms, and the need to take into account the Christmas, Easter and summer vacations and the impact this has on staff availability. The process continues to run smoothly and provides some scope for negotiation between the providers and QAA.

3.1.2 The self-evaluation document

106 In the main, each provider submitted an appropriate SED to QAA which took account of the guidance provided in Annex D of the Handbook and the additional guidance sent to the providers, through the facilitators, about the SED package (the SED plus supporting documentation), developed in light of the experience of the first major review.

107 The reviewers have found it most beneficial where the academic and practitioner section of the SED has been written for each discipline, because that reflects the structure of the judgements that the reviewers have to make, and the structure of the report. Such an approach is recommended in paragraph 10 of the Handbook and is stressed at the facilitator briefings and the SED workshops. Where such disaggregation has not been undertaken, the reviewers have had to spend significantly more time unpicking the evidence in order to make the judgements. Further written guidance has also been provided to HEI/SHAs to stress this point; however, not all providers choose to follow the advice.

108 The reviewers are requested to provide comments on the SED to the review coordinators prior to the preparatory meeting. The review coordinator provides a synopsis of these comments to the MRF and PRF prior to or at the meeting. This document is particularly useful in providing an initial agenda for the review and identifying the queries that the reviewers have. Facilitators have often sent the reviewers a written response from the subject staff; this document is then used throughout the review and has been considered particularly helpful.

109 SED workshops are run for those with responsibility for writing the document. Eight workshops were run between 2003 and the end of the academic year in 2005, with some 209 delegates attending. The workshops have been rated positively in the two years that they have run, with 96 and 94 per cent respectively either rating the workshop as good or very good. The opportunity to spend time discussing the SED with colleagues and sharing experiences is a frequently listed benefit of attending the workshop; 89 per cent of the MRFs, PRFs and subject staff who responded to the evaluation questionnaire after the major review agreed that the workshop helped in preparing the SED.

3.1.3 The preparatory meeting

110 The preparatory meeting has been evaluated positively by all those involved in major review - 100 per cent of respondents either agreed or strongly agreed that the 'preparatory meeting between the review coordinator and the HEI/SHA was useful in preparing for the review'. Ninety six per cent either agreed or strongly agreed that the 'timing of the preparatory meeting enabled the HEI/SHA to make all the necessary arrangements for the review'. It is seen as a meeting which enables discussion about the process, clarification of any queries, confirmation of the programmes involved, clarification of whether they are pre or post-registration and which programmes are under the auspices of the HPC, the NMC or the British Psychological Society, confirmation of the programmes included within the NMC annual monitoring, and the establishment of positive working relationship between the providers and the review coordinator. A standard agenda is used for consistency and a confirmatory letter is sent to the providers and copied to the reviewers. Increasingly, draft programmes of visits to practice placements for both days two and three of the review have been made available by the PRF at the preparatory meeting for discussion. This saves a significant amount of time and ensures that the proposed programme of visits is logistically practicable and covers sufficient breadth of the provision. The role of the PRF has been invaluable in the organisation and preparation of the placement visits.

3.1.4 Preparation of reviewers and facilitators

111 All individuals selected to become reviewers for major review undertake a three-day intensive training, including a simulation of a review led by review coordinators and QAA officers. This feature of involving review coordinators in both the training and the reviews has been very beneficial and has meant that a large amount of experience and good practice developed in the reviews is shared with the trainees. Reviewers from a range of disciplines are trained together, which has been seen to be a very positive experience and reflects review team activity. A total of 22 training events were run between 2003-04 and 2004-05, and all of them have been evaluated positively by the participants. In 2003-04, the events were considered as either good or very good by 98.5 per cent and by 99 per cent in 2004-05. Post-review, 97 per cent of the reviewers and review coordinators confirmed that training had helped to prepare the reviewers for the review. The review coordinators confirmed that the behaviour of the reviewers on the visits reflected the training.

112 A weakness related to the reviewer training has been the time gap between training and undertaking a review for some of the trainees. This has caused some anxiety, although every effort is made to keep reviewers up to date with major review through the QA Newsletter, the reviewers update newsletter, and the annual review trends report.

113 Where there has been a shortfall in the number of reviewers from a specific discipline that could place at risk the composition of a review team, an intensive one-day training event has been run by a QAA officer. Four such events have taken place during the review period, and nine reviewers have been trained initially in this way. All the reviewers have subsequently to attend the full three-day reviewer training before they are able to undertake any further reviews.

114 The MRF and PRF are nominated by the HEI and the SHA respectively and attend a one-day facilitator briefing, ideally together. This has a significant impact on the workload of the PRF who frequently participates in a number of major reviews with different HEIs. This multiple attendance at briefings can be difficult. The briefings have been received positively and have provided an opportunity for facilitators to start thinking about the major review, to share experiences and thoughts with colleagues from different organisations and to clarify queries with the QAA officers. All events have been rated as good or very good by at least 92 per cent of the participants, slightly higher than recorded in the first annual review trends report (2004). The briefings are led by QAA officers and review coordinators, thus enabling a large amount of experience developed through the reviews to be shared with the facilitators and subsequently with colleagues in the HEIs, SHAs and placements. In their post-review evaluation, 90 per cent of the MRFs/PRFs agreed that the briefing proved helpful during the review.

115 A key challenge for the providers and QAA has been the relatively high turnover of PRFs as staffing changes within the SHAs. In some instances this has meant the delivery of special one-off briefing sessions to bring the new PRF up to speed. As the review cycle has progressed, SHAs have been encouraged to send two members of staff to the facilitator briefings to provide some form of contingency.

3.1.5 Team composition

116 Normally, each review team is composed of two reviewers per discipline being reviewed, ideally one from a practice background and the other from an academic background, and each team is managed by a Review Coordinator. In the majority of instances it has been possible to achieve a balance between the reviewers from academic and practice backgrounds, each making an equal input into the team and the review. Indeed, such a split has been seen to be beneficial by the team, the Review Coordinator and the providers. However, in some of the smaller disciplines, it has been more difficult and some teams have not achieved an equal practitioner and academic balance. The gender balance of the teams has, in the main, been achieved and has reflected the balance within the sector and the provision being reviewed.

117 The screening of the nomination forms from those wishing to become a reviewer is undertaken anonymously and has followed the protocol consistently. Of the 426 nominations received to date, only two have been rejected. Throughout the period of reviews, the number of nominations received and reviewers trained successfully has been monitored closely by discipline and against predicted requirements. It has been necessary for QAA to undertake a significant amount of work to encourage nominations from the smaller allied health professions disciplines, a continued trend highlighted in the first annual review trends report (2004). To this end, QAA is particularly grateful to colleagues in the SHAs, professional bodies, HPC and SfH who have spread the word.

118 The team is frequently cited as one of the most positive features of a review by all participants, both in the questionnaires and in the focus groups. The reviewers consider that, while the work is demanding, the experience of working with colleagues from both academic and practice backgrounds and from a range of disciplines is invaluable, as is the sense of team cohesion and support. In recognition of the significant amount of work that the reviewers undertake, an increase in fees has been secured for 2005-06. The opportunity to work with colleagues from different disciplines is also welcomed and is a real strength of major review, particularly in relation to interprofessional working and learning. However, it is noticeable that where cross-discipline cooperation is not evident and the reviewers remain in their professional 'silos', the review is more problematic for all concerned.

119 As noted earlier (paragraph 3) a review may also have an NMC Visitor/reviewer included on a team in both capacities, in order to draw on the evidence gathered by the team and construct the NMC annual monitoring report on a number of programmes identified by the NMC prior to the review. It has been challenging to make clear to the Visitor, the reviewers and the providers how the two roles interlink and that the Visitor is a trained reviewer and a full member of the team. It has required shifts in culture to balance gathering evidence to meet the needs of the regulator and major review. The NMC annual monitoring reports produced by the Visitors have corresponded well to the major review reports and the integrated systems seem to be working well.

120 The reviewers are all professionally qualified and registered with the relevant statutory body. For NMC approved provision, the reviewers need to reflect the relevant Part of the NMC Register, ie nurses, midwives or specialist community public health nurses. For the provision that is being annually monitored, the reviewer/Visitor will come from that Part of the Register, and 'due regard' applies. Fields of practice in nursing, i.e. adult, children's, learning disabilities, and mental health are also recognised in the process of team composition, including the use of specialist advisers where necessary.

121 The reviewers are led by review coordinators, a small group of non-subject specialists who have a wealth of knowledge and experience of quality assurance in higher education and of working with QAA. There are 12 review coordinators; all are involved with the reviewer training, briefing and workshops and the reviews themselves. In addition, one Review Coordinator provides consistent and invaluable advice as editor of all major review reports. Each has undertaken a two-day review coordinator training event and attends an annual continuing professional development update day specifically focused on major review. Review coordinators also receive the QA Newsletter, the reviewer updates and regular update emails from QAA. The review coordinators are a valuable and key part of the reviews and their facilitation, communication and organisation skills are continually rated highly by at least 91 per cent of the reviewers and the providers.

3.2 Review

122 Major review is working well, particularly considering the complexity and breadth of the provision, and is now a tried, tested and refined review methodology. All post-review evaluation indicates a high level of satisfaction. A key strength of this review methodology is that it promotes partnership working between the providers (the HEI, SHA and placement providers) in the production of the SED and action plan, and through the work of the facilitators. This is in no small part because of the commitment of those involved and the responsiveness of the QAA's Health Team to issues, difficulties and evaluation received which, where it is feasible, are acted upon quickly. The approachability and support provided by the Health Team have also had a significant impact and are regularly commended in participant evaluations, at reviewer training, workshops and briefings, the focus groups and the qualitative data from the review feedback questionnaires. 98 per cent of respondents also agreed that 'the QAA supported the review in an effective way'.

123 The structure of major review has meant that practice has an active role, and takes equal responsibility in the quality assurance of healthcare education, an opportunity generally welcomed by practice staff. However, an issue for major review is the lack of service-user involvement directly in the process; equally, the documentation produced by the providers does not always reflect service-user involvement in the provision.

124 Another strength of major review is that it enables the student voice to be heard. The inclusion of a minimum of two student meetings means that the reviewers not only hear from students selected by the providers but ask to see a type of student or students from a particular discipline/s in the second meeting to follow up specific issues and lines of enquiry. Visits to placements ensure that additional students (post-registration as well as pre-registration) are consulted and have the opportunity to comment.

3.2.1 Pattern of reviews

125 The majority of the major reviews have been conducted using the same pattern of review teams visiting the providers for five days. In all but three of the reviews a 2 + 2 + 1 model was used, where two consecutive days are spent on site, followed by a break of usually two weeks before the reviewers return for a further two consecutive days, followed by a further break of one or two weeks before they return for the final day. Days one and four are usually spent at the HEI meeting staff, students, looking at documentation and learning resources. Days two and three are spent visiting a range of practice placements, and day five is spent with the reviewers discussing and deciding on overall judgements, and discussion of an early draft of the review report. Early in the review cycle, one review followed a 2 + 3 visit model and two small reviews followed a 2 + 2 model. In all these instances there was no period of reflection before the final day and the judgements. The review coordinators and reviewers felt that this lack of reflection was unhelpful and the model was time constrained; a view consistent with the first annual review trends report (2004). The remainder of the reviews have followed the 2 + 2 + 1 model.

126 On the whole, the 2 + 2 + 1 model is welcomed by the HEIs, SHAs, Trusts and the reviewers. For the HEI and SHA, although it can provide some logistical challenges in relation to the setting up of the room(s) where the reviewers are located, the pattern of days facilitates reflection and enables preparation before a visit including responding to the reviewers' queries. What is crucial, however, is the continued communication between the Review Coordinator and the facilitators. For some reviewers, these benefits have to be set against the additional travelling and time away from work/home commitments that the model requires. For many reviewers the key to maintaining engagement with the review during the periods away from the providers is continued communication with the review coordinator and other team members through QAA's Academic Reviewer Communication Service (ARCS).

127 A challenge for the providers in preparing the programme of visits to practice placements on days two and three is often the geographical spread of placements. An effective approach adopted has been, where necessary, to locate the team or part of the team at a different hotel closer to the placements to be seen the following day, either day two or more usually day three. This ensures that placements are visited because they can provide appropriate evidence in the verification of the SED, not because they are conveniently placed. Similarly, where providers have a specific 'satellite' provision, often highly specialised and geographically distant, then reviews have been organised so that a small number of the reviewers visit the satellite prior to day one and provide a report of the meetings with staff, students and the resources available, ready for the whole team on day one.

128 A consistent comment through all the evaluation mechanisms is that the time available for major review can be pressured, particularly given the complex structure of healthcare education provision. QAA wishes to acknowledge the level of commitment and workload that reviewers, review coordinators and facilitators have undertaken in preparing for, during and following the reviews.

3.2.2 Major review and practice review facilitators

129 The MRF and PRF roles have worked extremely well in the reviews to date and their significant contribution to the success of major reviews is a consistent trend first highlighted in the 2004 Annual Review Trends Report. The facilitators play an important role in representing and feeding back to the HEI, the SHA and placement providers, and ensuring the transparency of the review process. The review coordinators and reviewers have, in the main, been complimentary about the helpfulness and expertise of the facilitators, noting the positive impact that facilitators can have on the smooth running of a review. Over 92 per cent considered that the skills, knowledge and expertise of the MRF and PRF were good or very good. The pairing and equity of the roles has been particularly helpful in ensuring that campus-based and practice-based elements are considered equally; in promoting partnership working; in providing sufficient flexibility to suit the varying styles of the subject providers, and in ensuring equity of input and engagement in all aspects of the review including the SED and action plan; and enabling feedback from the review coordinators to be heard across all those involved in the provision without prejudice or bias.

130 It has been useful to have, as the MRF, a non-health specialist as they are more easily able to maintain a level of objectivity and not become an advocate for the subject staff. The MRFs have also been key in bringing knowledge and expertise about institutional policy to the review. The PRFs have undertaken a significant amount of work in preparing and engaging practice staff in the review process, as well as organising placement visits, ensuring that those proffered reflect the full range of types, size and geographical location, and are realistically manageable for the different pairs of reviewers in the time available. Often all this is done not just in relation to one HEI but to several of them and in conjunction with other SHAs.

131 A challenge for the providers, the reviewers and QAA has been the turnover of staff in the MRF and, more significantly, in the PRF roles. This has led to a lack of continuity and meant that additional briefing has had to be undertaken often at short notice. PRFs have had an additional challenge of often having to represent more than one SHA in a review, resulting in a significant amount of additional work in order to ensure that the other SHAs inform the process. In a number of instances, local networks have been set up between providers and/or SHAs. A national PRF network has been set up by the PRFs; QAA and SfH are invited to attend as observers and this has been very helpful.

3.2.3 Documentation

132 The Handbook is considered by those involved in reviews as providing clear guidance on major review and the information to be included in the SED (93 per cent of respondents to the post-review evaluation questionnaires). Although, the forms/documentation issued by QAA during the review are rated positively by the majority of respondents to the questionnaire (90 per cent), feedback from the focus groups is not so positive, particularly the practice placement pro forma - a template used by the reviewers to capture the evidence gathered from a visit to practice placement in a consistent manner. Although challenging at times to complete, the pro formas have provided written evidence about each of the visits that can be shared with all members of the team and used to underpin the report, thereby facilitating reflection of practice learning. All major review documentation is reviewed regularly and amendments made where appropriate.

133 The amount of documentation available to reviewers has varied, in common with the experience highlighted in the 2004 annual review trends report. Some providers have given the reviewers a significant amount of additional documentation, while others have been more focused. Increasingly, the SED and the references and supporting documentation have been made available electronically on CD-ROM or through HEI intranets, as recommended in the first Annual Review Trends Report (2004), which the reviewers have found helpful. Review coordinators have tried to keep documentation to a minimum, both in terms of what is provided by the HEI/SHA and what is requested by the reviewers, although, in the former case, it has not always been possible. In the latter case, it has not always been possible to change reviewer preconceptions/expectations about the amount of evidence that can be considered realistically in the time available for the review. The documentation used in major review is considerably less than in previous discipline level reviews.

134 An important document prepared by the reviewers prior to the judgement meeting on the final day of the review is the first draft of the major review report, draft 0. It is this document that the reviewers discuss and test in coming to their judgements. It became apparent early in the cycle of review that, where draft 0 was used as part of the judgement meeting, the subsequent drafting of the final report was much less problematic. It is now used as standard procedure. The discussion of draft 0 at the judgement meeting has demonstrated and encourages further, strong cross-discipline working and support within the team.

3.3 Post-review

3.3.1 Major review reports

135 On the whole, the process for the production of the major review reports is working well. The use of a single Review Coordinator and the QAA Assistant Director (Health) as co-editors for all reports has proved beneficial, particularly in ensuring consistency between reports. The role is challenging due to the complex and ever-changing nature of healthcare education provision, for example, the continually changing and diverse titles after the establishment of SHAs on 1 April 2004. Another example is the development by the HPC of Standards of Proficiency and Standards of Education and Training, and changes in terminology such as NMC's move from competences to standards of proficiency, and the variable terms for learning after registration.

136 Review reports must ensure that disciplines are set out separately in the academic and practitioner standards section to reflect the judgements, a challenge with the current requirement for a separate health visiting section when it is part of the community nursing programmes (due to NMC guidance on curriculum), and the increasing prevalence of interprofessional programmes.

137 The greatest threat to the reports and the credibility of the major review process is the standard of writing of the reviewers, which is why attendees at training have to complete two pieces of overnight work that test their ability to write in the context of quality assurance. Where difficulties are identified, additional help can be given, and where the difficulties are significant and sustained, the individual will not be appointed as a reviewer. On the whole, the standard of writing has been appropriate and many reviewers have demonstrated an excellent awareness of statutory, regulatory and professional body regulations as well as current government policy and initiatives.

3.3.2 Action plans

138 Action plans are a significant part of major review, but are not currently used in any QAA methodology, so there were new challenges in setting guidelines and systems for their production.

139 QAA faced the challenge of developing a template for the action plans that the major review steering group considered user-friendly for providers but could also easily be integrated into the major review report and published on the QAA website and in hard copy using current software and technology. The standard timescale for the production of review/audit reports by QAA is 20 weeks from the end of the review. Fitting the production of the action plan into this schedule is only just possible using very tight timescales; the date for the return of the plan from the HEI/SHA is only two weeks before the publication deadline. Any delays in the production process arising from, say, the incomplete signing-off of the plan, directly affect the ability of QAA to meet its contractual deadlines.

140 The completion of the action plan can be a lengthy and complex procedure, requiring discussion and consent from all stakeholders, often with several stages of drafting. As the reviews progressed, it was clear that the original method of sending the plans in hard copy to the HEI/SHA for completion and signatory was less than helpful. SfH agreed to allow slightly longer timescales to enable facilitator activity, particularly over the holiday periods, and the action plans are now sent in an electronic format and can be returned electronically without the actual signatures. The signatures must be returned but can be provided separately in hard copy.

141 In order to provide the framework for the action plan, the bullet points provided by the reviewers must be actionable. Providing effective guidance and training to the reviewers and the review coordinators was challenging initially and required some revision during the early reviews. An example of this was that in the earlier reviews, the reviewers produced bullet points which were too voluminous to be helpful for the providers, or practicable in the timescales for the production of the plan, and were often difficult to action. Therefore, the reviewers were asked to reduce the number of bullet points to be responded to, limiting them to including key points only. MRFs and PRFs were provided with further guidelines to enable them to complete the plans effectively and return them in a format that QAA could work with. However, this is still an area for ongoing work and, in 2005-06, QAA is running workshops focusing on action plans to enable discussion, feedback and dissemination of good practice by providers.

Chapter 4 Structures

142 During the period of major review so far, there have been rapid and significant changes to the structures and policy framework within which the review methodology operates. Throughout, QAA has been responsive to these changes and developments, for example, ensuring that the reviewers clearly understand the change of emphasis in the key issues, but without placing the robustness of the method at risk. The reviewers are also encouraged to explore how the partners are working together to address these changes so that there is no negative impact on standards or on the student learning experience.

143 Government initiatives such as Agenda for Change, the knowledge and skills framework, and the skills escalator are important contextual issues that the reviewers must be aware of, particularly in relation to understanding the development of the knowledge and skills framework, in order to ensure that the provision effects the current context.

144 The approach to quality assurance in England has also seen change during the major review period to date. At its inception, major review was one of many quality assurance processes in higher education; now it is one of few remaining review methods at discipline level within HEIs. Every effort is made to coordinate with other reviews and audits occurring in the higher education sector, and this has included agreeing review dates and also the scope of major review. There is ongoing discussion with the Healthcare Commission, and all major review reports are sent to the Commission.

Partners and stakeholders

145 Major review was developed with input from, and is owned by the DH, HPC, NMC, SfH and SHAs. All these organisations have seen significant changes that have implications for major review. From 1 October 2004, responsibility for the development of the quality assurance framework for healthcare programmes, of which major review is one element, moved from the DH to SfH under service-level agreements. This saw the quality assurance team responsible within DH also move across to SfH. QAA continues to work closely with SfH and the quality assurance team.

146 The relevant statutory and regulatory bodies have also seen notable changes and developments. The HPC has settled into its remit and structures and, after consultation, has confirmed its quality assurance processes and documentation. These include its approval process, annual monitoring process, the standards of proficiency for each profession, the standards for education and training (threshold standards for safe and effective practice that all registrants must meet) and the standards of conduct performance and ethics. The first approval event took place in spring 2005 by HPC Visitors, and annual monitoring began in September 2005. These processes link into an HEI's internal validation and review events and will provide additional evidence for the reviewers. Another recent development that impacts on major reviews is the agreement to change responsibility and organisation of dietetics placements from a national level to locally-agreed placements. QAA has ensured that reviewers and review coordinators are informed fully of these developments to ensure currency of knowledge.

147 The NMC is going through considerable change, particularly the debate about fitness to practice and newly-qualified practitioners. Of most significance to major review is the change to the Register (on 1 April 2004) from its previous 15 parts to three: nursing, midwifery and specialist community public health nursing. However, this did not remove the NMC's need for separate consideration of health visiting programmes in relation to academic and practitioner standards. It is noticeable that it became more difficult for some providers to disentangle health visiting from its SCPHN provision in order to write a separate section in the SED. In some cases no such separation took place, making exploration of the evidence and subsequent judgements more difficult for the reviewers. There have also been changes in terminology that could affect the accuracy of the reports if not communicated by QAA to the reviewers and review coordinators.

148 Communication with the statutory regulatory bodies has been ongoing through the reviews to ensure that QAA is fully aware of the developments taking place; for example, a QAA officer observes HPC's Education and Training Committee, and also chairs the operational group for QAA, NMC, HPC and SfH. QAA works closely with the NMC operationally to identify the scope of the annual monitoring to be considered through the major review and to identify and confirm the reviewer/Visitor on each team as appropriate.

149 During the major reviews, the 28 SHAs were established. However, each took a different approach to its structure: some maintained a separate WDC identity, others established a Workforce Development Directorate, and others subsumed workforce development fully into the SHA identity. All have maintained their workforce focus but are at different stages of development and have experienced significant changes in human resources and responsibilities. QAA has spent considerable time re-establishing key contacts, confirming responsibilities and identifying PRFs. The SHAs have matured quickly and have, on the whole, engaged fully with major review in their dual roles of commissioner and provider. This is not least because of the commitment and hard work of the PRFs and the development of an effective PRF network.

150 The interrelationships between HEIs and SHAs as commissioners are often complex, but what has come to light through the major reviews is that some SHAs purchase small amounts of provision from an HEI other than their main provider but do not always quality monitor the provision. The reviewers are encouraged/required to look at previous SHA monitoring documentation as part of the review, and these can be absent for these small commissions.

151 The development of Primary Care Trusts (PCTs) and Foundation Trusts has not had a discernible impact on major review. Both have engaged fully with the process, and access to practice placements has been open and welcomed. The changes that are now occurring in PCTs will no doubt have an impact on major review in its final year of delivery.

152 There has been consistent evidence during the reviews that the practice placement facilitators/clinical placement facilitators have a significant and positive influence on healthcare education. The clinical assessor roles take many forms; a raft of different people undertake these roles, some being directly funded by SHAs while others are joint appointments between the Trust and the HEI. However, the role is in a state of flux and their future is uncertain.

153 Until recently, HEIs have seen a significant increase in student numbers commissioned. This increase also saw the development of programmes in disciplines not previously provided by many HEIs. The impact of this for major review is a discrepancy in the scoping information gathered in 2003 and what is now being delivered in reality, causing some difficulties for planning review team composition and the number of reviews that an HEI/SHA require. Once this issue had been identified, a more formal process of clarifying scope was implemented by QAA and providers were reminded that two reviews will be undertaken for provision with more than six disciplines.

QAA

154 The Health Team lead on major review and all other PQAF-related activities. The Team comprises an Assistant Director, a Development Officer, two Project Officers and one Administrator. These are supported by staff from four other administrative teams in QAA: Scheduling and Recruitment, Logistics and Deployment, the Reports Team and, for the analysis of evaluation activities, the Information Unit. The Health Team manages all aspects of the reviews, including drafting and updating the Handbook; undertaking the initial scoping of all the provision; devising the three-year cycle of reviews; setting up and refining internal procedures and protocols to support major review operationally; developing and updating documentation, guidance notes and training materials; issuing and screening nomination forms, facilitating and monitoring reviewer recruitment, including equal opportunities and valuing diversity; training reviewers, briefing subject staff and facilitators, undertaking presentations as requested; outlining team specifications; co-editing all major review reports and monitoring the report and action plan production schedule; conducting post-review evaluation activities; updating reviewers, HEIs/SHAs and clinical staff on developments and providing CPD to the review coordinators; responding to queries; liaising with the DH, SfH, HPC, NMC, professional bodies; chairing and officering relevant working, steering and operational groups; and producing detailed progress reports for the quarterly contract review.

155 In parallel with the major reviews, another part of QAA, contracted by DH now SfH, has been working to complete the last of the subject benchmark statements for healthcare disciplines and to develop further the emerging health professions framework into an overarching framework. The emerging framework outlined the common features, nature and characteristics of healthcare professionals as highlighted in the benchmark statements. However, due to the changes that have taken place since the development of the emerging framework, for example, the increase in interprofessional education, the emergence of new professions and new regulatory bodies, changes in the way in which services are delivered and the increased emphasis on user and carer involvement in healthcare education, the development of an overarching framework would not be relevant. The statement of common purpose builds on and supersedes the emerging framework and identifies the skills, knowledge and approaches that health and social care professionals should have and be able to demonstrate in relation to values in health and social care practice, the practice of health and social care, and knowledge and understanding for health and social care practice. The statement is to be published.

Chapter 5 Conclusions

156 Evidence confirms that the major review process fulfils what it set out to do and the majority of those involved in the major reviews to date are positive about the process, its level of transparency and focus. The process is sufficiently robust, yet flexible in operation, to address the continued and rapidly-changing context in which it operates without putting the integrity of the process at risk. It facilitates partnership working between the providers, gives equal weight to the practice and academic elements of the learning, enables practice placement providers to have an active involvement in the review, and provides opportunities for the student voice to be heard.

157 The outcomes from major reviews are the judgements, the reports and the action plans. The reviewers have made judgements in the correct and appropriate manner, based on robust evidence. In academic and practitioner standards, most judgements have been confidence judgements with only one limited confidence for a single programme. In the quality of learning opportunities, again most judgements have been commendable, a small number of reviews receiving approved judgements but at programme level only. This ability to differentiate judgements by programme, and/or mode, and/or level is seen as a particular strength of major review.

158 Major review reports and actions plans have been produced as planned and provide an important source of information to the public and potential students, as well as being a potentially valuable source of evidence in other elements of the proposed PQAF. Inclusion of the action plans in the major review reports provides a more holistic view of the provision.

159 The plans are becoming more streamlined and meaningful after additional guidance from QAA to the reviewers in response to concerns that the identified strengths were reflecting issues that should be evident as a requirement to deliver the programmes. Providers are more able to respond to the bullet points and outline targeted actions with completion dates that can be monitored. The constraints on delivering the actions are now identified more frequently with increased detail. With little time between the completion of these first action plans and their publication in the major review reports, it is not surprising that there is scant evidence of quality enhancement stated in the plans; rather, providers list the sources of evidence that could be used in future. Responsibility for undertaking the actions continues to lie predominantly with the HEI, although more examples of explicit joint responsibility are starting to be seen.

160 The beneficial role of the clinical practice facilitator (or equivalent role) in enhancing student learning, supporting mentors and assessors and improving standards and quality is a common theme throughout the reports, as is the effective partnerships that exists between most HEIs, SHAs and practice placement providers.

161 The reports are generally positive about academic and practitioner standards, with curriculum and learning outcomes meeting statutory regulatory and professional requirements, taking account of the Academic Infrastructure and integrating theory and practice fully. Assessment methods and criteria are appropriate, with the majority of external examiners supporting the approaches taken. Of particular note is the use of virtual learning environments to support the assessment process. Persistent weaknesses relate to inconsistencies in the use of assessment criteria between modules, the lack of clarity in some assessment processes, and the paucity and/or poor quality of feedback to students on their assessments.

162 Student achievement seems generally to be appropriate, with little statistical difference between allied health profession and health visiting, midwifery and nursing programmes and, in the majority of cases, is endorsed by employers and/or external examiners. However, SHAs would welcome more explicit confirmation in the reports that students are fit for purpose and it is suggested that, in any future cycle of peer review, stronger guidance is provided.

163 In the main, the reviewers are positive about the quality of learning opportunities. Support for students is highlighted frequently as a strength or good practice as are a number of recruitment, retention, widening participation and valuing diversity initiatives and IPL/E developments. Learning resources are reported positively in terms of human resources, information communication technology and library facilities. Areas for more concern were inconsistencies in the return of student work, workload for clinical staff specifically in the context of CPD and mentor updating, and maintaining the live register of mentors in nursing (particularly) and midwifery.

164 Effective arrangements for MESQ are evident from the reports in relation to monitoring, reviewing and enhancing programmes and processes, with appropriate response mechanisms and timeframes particularly in relation to student feedback. Weaknesses tend to be in relation to inconsistency in the implementation of procedures for HEIs at school/department/programme level and for SHAs in contract monitoring, and incomplete recording of student feedback and action taken.

165 The analysis of the data provided in the reports indicates that there is no significant difference between the disciplines reviewed or at the level of the programme (diploma, degree or postgraduate qualification) in relation to completion and achievement statistics for all award bearing programmes; employment statistics for pre-registration and post-qualifying programmes (by exception reporting only); and recruitment and attrition statistics for pre-registration and NMC recordable qualifications.

166 Despite the constraints created by the availability and variability of information provided by the HEIs/SHAs, the following themes emerged:

• by far the majority of students on those programmes reviewed studied on nursing programmes
• some disciplines (and levels) were offered by multiple providers, whereas others were offered by a small number of providers
• there was variation in respect of achievement and attrition between providers. However, the average degree profile for all disciplines displayed congruence with the degree classification profile for all higher education degree students studying at HEIs in England (based on HESA 2003-04 student data). For all programmes of study for which data were provided, the average reported pass rate was in excess of 90 per cent for all disciplines
• with regards to employment following completion of the programme of study, only one in 30 students was known to have been unemployed
• paramedic science has the highest proportion of failures, transfers in and transfers out of programmes, but is one of the newest programmes at degree level and caution is needed as the number of programmes reviewed so far makes this figure unreliable
• return to practice programmes have the lowest withdrawals and transfer out rates. Additionally, they showed the highest rates of local employment post-qualification. This would appear to be meeting the required purpose of the programmes
• overall, the picture for recruitment and retention is positive.

167 The processes that support major review, led by QAA's Health Team, are working effectively, not least because of the continual refinement of these processes in light of the experiences gained with each review and also the feedback from participants. Examples of this include:

• the scheduling process refined to take account of the number of reviews taking place each term, and across the Christmas and Easter periods
• the limitation of the maximum number of disciplines for each review to six
• the revision of the processes for gathering reviewer availability and conflicts of interest to make team composition more efficient, although still problematic at times
• the provision of additional guidance by QAA to HEIs/SHAs on the structure of the SED as a result of some being submitted without the required differentiation of the academic and practitioner standards section per discipline, or with some sections ignored.

168 The elements of the major review processes that have been particularly effective and valued by those participating are:

• the preparatory meeting as an effective mechanism enabling discussion about the process, clarification of scope, identification of practice placements to be visited for day two and more latterly day three, and establishing a positive working relationship between the providers and the Review Coordinator
• the process for addressing the withdrawal of a team member at short notice, including the use of the specialist adviser
• the monitoring of reviewer profiles to ensure an appropriate balance of equal opportunity that values diversity within the pool of reviewers
• the 41 events run by QAA to train reviewers, brief facilitators, and brief subject staff on preparing the SED. Some 770 people have attended these events, the overwhelming majority (at least 92 per cent) evaluating them positively, welcoming the opportunity to work with colleagues from different disciplines or providers, and acknowledging the quality, skills and approachability of those delivering the events
• the reviewers supported by the small group of review coordinators. Both have undertaken a significant amount of work to the benefit of the reviews and to personal development
• the integration of NMC annual monitoring with the major review process, which has functioned effectively
• the MRF and PRF roles which have promoted partnership working, enabled clinical and academic staff to have an equal input into the SED and action plan, and have ensured that feedback from the reviewers, through the Review Coordinator is disseminated to all those involved in the provision
• the significant work undertaken by PRFs in engaging clinical staff in the reviews, as well as deftly organising often-complex and geographically-spread practice placements into a workable two-day programme of visits for the reviewers
• the use of non-health specialists as MRFs in order to maintain a high level of objectivity and to bring increased knowledge of institutional policy and procedures
• the 2 days + 2 days + 1 day model of major review visits in enabling reflection and further preparation between the visits for all concerned
• the use of a draft 0 of the major review report at judgements meetings to discuss and test the evidence again, thereby ensuring rigorous and robustly-evidenced judgements
• the responsiveness of the QAA's Health Team to queries, issues and concerns of participants
• the improved process for the publication of action plans that provides the plans in an electronic format to HEIs/SHAs earlier in the process and for an extended period, to combat the difficulties experienced by the MRF and PRF in securing sign-off of the completed plans by appropriate senior staff.

169 The major review process has faced, and continues to face, a number of challenges, including:

• the common intensity of approach that has meant that HEIs/SHAs with small healthcare provision have received the same level of scrutiny as the larger, more complex provisions; it is suggested that, in any future cycle of major review, the intensity of scrutiny is proportionate to the size of the provision and the level of risk attached
• recruiting reviewers from the smaller disciplines and certain larger disciplines. A significant amount of time has been spent by the Health Team, colleagues in SHAs, professional bodies, HPC, NMC and SfH in generating interest and nominations
• the time gap between training reviewers and their first major review. For some this has been an extended period which, in spite of regular update newsletters, has caused some anxiety
• the challenge for review coordinators in making clear how the reviewer/NMC Visitor's roles interlink and are prioritised
• the continually and rapidly-changing context in which major review operates, exemplified by the move of the DH Quality Assurance Team to SfH, the developments in the quality assurance processes in the statutory regulatory bodies, the introduction and now revision of 28 SHAs with very different structures and development periods, the changing terminology used within both sectors, the changes in emphasis within national policies and new initiatives and the development by providers of new programmes often in new disciplines in response to increased commissioned numbers.

170 This second annual trends report has raised some interesting points and has been able to explore the outcomes, processes and structures relating to major review in some detail. However, with only a third of the reviews undertaken, the assertions made in the report must be viewed as tentative. The 43 reviews, at a minimum, that will take place in 2005-06 will add significantly to our understanding and to the third annual reviews trend report next year.

 

Appendices

Appendix 1

Summaries of the key themes arising from the analysis of the strengths, good practice and weaknesses listed in the published major review reports (spring 2004 - spring 2005), differentiated by discipline area: allied health professions, and nursing, midwifery and health visiting.

Academic and practitioner standards

 

Table 1: Allied Health Professions - Summary of key messages in academic and practitioner standards (strengths, weaknesses and good practice that appear most frequently across the reports analysed)
Strengths	Weaknesses	Good practice
Strong, effective and well-developed links between all stakeholders in curriculum development, and integration of theory and practice. Staff research and expertise enhance teaching quality and inform curriculum development. A wide range of assessment methods achieves intended learning outcomes, which are appropriate to service needs. Employers express high level of satisfaction with students' abilities, knowledge, clinical skills and competencies. Programmes effectively prepare students for employment. Practice-learning is well integrated into curricula, with good support systems in place.	Inconsistencies and lack of clarity in some marking guidelines, assessment criteria and procedures for the redemption of failure. Poor-quality written feedback on assessments, coupled with significant delays in returning feedback to students. Some placement assessments do not form part of the final award. Under-developed opportunities for interprofessional learning within curricula.	Strong collaborative partnerships in curriculum development. Use of portfolio to demonstrate a valuable way to integrate theory and practice, and to help prepare students for continuing professional development. Use of simulation and e-learning environments in teaching and assessment.

 

Table 2: Nursing, Midwifery and Health Visiting - Summary of key messages in academic and practitioner standards (strengths, weaknesses and good practice that appear most frequently across the reports analysed)
Strengths	Weaknesses	Good practice
Strong, effective and well-developed links between all stakeholders in curriculum development and integration of theory and practice. Curricula contain comprehensive coverage of key cognitive and professional skills and competencies. A wide range of assessment methods achieves intended learning outcomes, which are appropriate to service needs. Clear, comprehensive and informative feedback on written work is provided in some programmes. Employers express high level of satisfaction with students' abilities, knowledge, skills, attitude and responsiveness to employers' needs. Clinical assessors play an important role in preparing students for practice. Good examples of practice-based learning.	Not all partners work fully with the university to promote student achievement. Written feedback is not always sufficiently detailed or specific, and there is some delay in returning work to students. Placements are sometimes fragmented, causing a lack of continuity for students and clinical assessors. Inconsistencies and lack of clarity in some marking guidelines, assessment criteria and programme documentation. Some clinical assessors are not appropriately prepared or updated.	Collaborative approach to curriculum development which meets service needs. Interassessor reliability monitoring process. Tripartite meetings between clinical assessor, link lecturer and student provide effective support for student and assessor.

Quality of learning opportunities

Table 3: Allied Health Professions: Summary of Key Messages in quality of learning opportunities (strengths, weaknesses and good practice that appear most frequently across the reports analysed).
Strengths	Weaknesses	Good practice
Theory, research and practice are well integrated into the curriculum and effectively supported by good partnerships between all stakeholders. Learning resources are of a high quality and provide good support to learning and teaching activities. Recruitment processes are strong and result in high retention rates in many programmes. A high standard of learning opportunities is evident in many practice placements. Student support is good, with effective systems in place to identify and support specific needs.	A lack of placements and poor accommodation within placements. A lack of physical resources in placements and restricted access to learning resources while on placement. A lack of formal monitoring, evaluation and response to student feedback. High attrition rates in some programmes.	Strong and effective support for students and staff. E-learning facilities, resource centres and a book-delivery service to placements.

 

Table 4: Nursing, Midwifery and Health Visiting: Summary of key messages in quality of learning opportunities (strengths, weaknesses and good practice that appear most frequently across the reports analysed).
Strengths	Weaknesses	Good practice
Learning resources are of a high quality with many well-equipped skills laboratories as well as e-learning and library facilities. Placements provide excellent learning opportunities, supported by strong links between all stakeholders. Well-structured student support mechanisms, including placement support from clinical assessors and link tutors. Learning and teaching strategies promote a coherent and comprehensive approach to teaching development and dissemination of good practice.	Some clinical assessors find course documentation difficult to use and there is a lack of qualified clinical assessors in some areas. High attrition rates in some programmes. Some clinical assessors are struggling to fulfil their role and there are limitations on the range of experiences available in some nursing placements.	E-learning resources are well-established and provide an effective learning and teaching tool. Retention teams are providing additional support to students in managing the transition to HE and other non-academic issues. A clinical assessor award scheme initiated and supported by the SHA helps to enhance the role and is highly valued by practitioners.

Maintenance and enhancement of standards and quality

Table 5: Summary of key messages in maintenance and enhancement of standards and quality (strengths, weaknesses and good practice that appear most frequently across the reports analysed).
Strengths	Weaknesses	Good practice
Programme monitoring and review processes are generally good, and the implementation and delivery of quality assurance processes is largely effective. Programme teams are responsive to the needs of students and trainees, practitioners and employees. Links between all stakeholders are strong, working together in areas of curriculum development, placement learning and student support. Staff research interests and expertise are used to enhance the teaching and learning experience. Clinical assessors are valued by the students and largely work effectively.	There is some strain on clinical assessors where fewer of them are available to support a large number of students. Feedback to students on assessments could be more detailed and returned earlier. Student evaluation needs more formal structure and monitoring. Interprofessional learning needs to form a structured part of the curriculum where this is not already in place. A lack of consistency in the implementation of procedures at school level.	Actions taken following previous reviews to ensure attention to the maintenance and enhancement of standards and quality. Specialist committee subgroups, including student representatives, to ensure the ongoing quality and enhancement of provision.

Appendix 2

Major review - changes to processes

QAA administrative support teams	Development of process flowcharts for each administrative within QAA. Development and updating of an operational pack for all internal QAA staff, review coordinators and partners. Continued updating of planning tools such as Gantt charts and critical pathway. Updating of scoping exercise due to inclusion of additional disciplines in major review and developments in new provision by HEIs/SHAs. Establishing the maximum number of disciplines per review in light of experience of a large and complex review. Revision of process to allocate review dates in the light of increased number of reviews per term and to take account of holiday periods. Development of team specification forms - to identify specific reviewer requirements for each team. Development and updating of visit status reports to monitor progress of each review. Sending an electronic version of the action plan to facilitate its production within the time available. Development of checklist for internal use regarding reports and action plans.
Review teams	JACS codes adjusted to incorporate new professions. Monitoring of reviewer nominations against discipline and review requirements. Monitoring of equal opportunities in pool of reviewers. Revision of system for checking reviewer availability to ensure a single point of contact for each term. Targeted activity to encourage nominations in disciplines where there is a reviewer shortfall. Refinement of the process to identify NMC Visitor/reviewer. Developing procedures for the withdrawal of reviewers from teams at short notice.
Reviewer training	Adapting training materials to address changes in context and feedback from participants. Revision of programme timings at the end of the first cycle of training events. Development of specific handouts, for example, to explain amendments to the Handbook and contextual development. Development and updating of tutor manuals and tutor notes. Development and refinement of guidance on pre-training and overnight work for trainees. Updates provided to trained reviewers through email as required and the reviewer update publication.
Briefings and workshops	Continued updating of materials and handouts Provision of additional small ad hoc training events to address increased turnover of SHA staff and PRFs Annual contining professional development for review coordinators.
Documentation and template development	Briefing documentation - Executive summary and Frequently asked questions, Fact sheet for reviewers, quarterly QA Newsletter. Development of and revision of pro formas used by the reviewers in light of feedback from reviews. Guidance for review coordinators and/or the reviewers on: oral feedback, bullet points, quantitative data, referencing sources of evidence in reports Guidance for providers on: the SED package, quantitative data tables and action plans, reminder about the structure of the SED. Development of the electronic action plan template. Revision of the major review report template.

Appendix 3

Acronyms

ARCS - Academic Reviewer Communication Service

CPD - Continuing professional development

DH - Department of Health (England)

FHEQ - The framework for higher education qualifications in England, Wales and Northern Ireland

HEIs - Higher education institutions

HESA - Higher Education Statistics Agency

HPC - Health Professions Council

IPL/E - Interprofessional learning/education

IT - Information technology

MESQ - Maintenance and enhancement of standards and quality

MRF - Major Review Facilitator

NMC - Nursing and Midwifery Council

OQME - Ongoing quality monitoring and enhancement

PCTs - Primary Care Trusts

PQAF - Partnership Quality Assurance Framework

PRF - Practice Review Facilitator

QAA - Quality Assurance Agency for Higher Education

SCPHN - Specialist Community Public Health Nurse

SED - Self-evaluation document

SfH - Skills for Health

SHAs - Strategic Health Authorities

WDCs - Workforce Development Confederations

---

ISBN 1 84482 435 7

© Crown copyright 2004