Outcomes from institutional audit

Assessment of students

QAA 097 01/06

Summary

Consideration of the institutional audit reports published by November 2004 shows that, in general, institutions are making progress in developing and using assessment practices. These practices are consistent and fair to students. They also help to assure academic standards.

Features of good practice are cited in the Findings of 10 reports. They identify strengths in the provision of information and guidance about assessment, in the use of assessment boards, in the development of assessment criteria, in the evaluation of assessment outcomes, in the use of assessment to support student learning, and in the review and development of assessment regulations. There are 41 recommendations advising institutions to take action in connection with aspects of assessment practices, 10 of which were considered desirable.

Many institutions have strengthened the reliability and improved the consistency of their assessment arrangements, drawing on QAA's Code of practice for the assurance of academic quality and standards in higher education (the Code of practice) and The framework for higher education qualifications in England, Wales and Northern Ireland (the FHEQ). However, a number of audit reports express concern about the level and justifiability of the variability permitted or tolerated. Several reports discuss the effects of variability on the need to ensure that students are treated fairly. In most of these cases, reports conclude that further progress will be required before this can be demonstrated.

Generally, audit reports comment positively on the clarity of the information provided to students about the ways in which they will be assessed and about the criteria used to judge their performance in assessments. The introduction of programme specifications has helped to facilitate these improvements. Some audit reports describe the attention being given to both generic assessment criteria and discipline-specific assessment criteria; this helps to demonstrate the consistency of institutional academic standards.

Several audit reports discuss the methods used to determine classified honours degrees for undergraduates. Overall, there appears to be a trend towards greater uniformity in these methods. Where reports have considered the circumstances of students whose programmes of study require the use of different and incompatible methods of assessment, the adverse consequences for individuals of such variability has led to critical comment. In some cases, however, institutions have been able to argue in support of the diverse disciplinary traditions and practices giving rise to such difficulties.

Some institutional audit reports comment on how assessment boards and panels have exercised their discretion, and have expressed concerns where it has not been used consistently in determining assessment outcomes. In such cases the chief concern is the potential for individual students to be treated unfairly and for the academic standards of awards to be called into question. Overall, however, audit reports show that assessment boards and panels are properly discharging their responsibilities to ensure that assessment practices are rigorous and reliable.

A number of reports comment on the importance of timely and helpful feedback to students on their assessed work. Some reports describe good practice in such arrangements. Guidelines provided at institutional or subject level appear to have proved useful, though they needed to be accompanied by processes monitoring their implementation.

Some reports noted that institutions have been slow in improving their assessment arrangements. In these cases, institutions had yet to demonstrate that their assessment arrangements were equitable, and made an effective and substantial contribution to the maintenance of academic standards. More generally, however, the institutional audit reports published by November 2004 indicate that most institutions have successfully taken steps to improve aspects of their assessment arrangements.

Preface

An objective of institutional audit is 'to contribute, in conjunction with other mechanisms, to the promotion and enhancement of high quality in teaching and learning'. One of the ways in which this can be accomplished is through identifying features of good practice across the reports and areas where reports have commonly offered recommendations for improvement.

In due course, QAA intends to produce an extended reflection on institutional audit in the Learning from audit series, but since the final institutional audit reports in the present audit cycle will not be published until spring 2006, Learning from institutional audit is unlikely to be published before late 2006. To give institutions and other stakeholders more timely information, QAA has therefore decided to produce a series of short working papers, describing features of good practice and summarising recommendations from the audit reports, to be published under the generic title Outcomes from institutional audit (hereafter, Outcomes...).

A feature of good practice in institutional audit is considered to be a process, a practice, or a way of handling matters which, in the context of the particular institution, is improving, or leading to the improvement of, the management of quality and/or academic standards, and learning and teaching. Outcomes... papers are intended to provide readers with pointers to where features of good practice relating to particular topics can be located in the published audit reports. Each Outcomes... paper therefore identifies the features of good practice in individual reports associated with the particular topic and their location in the Main report. Although all features of good practice are listed, in the interests of brevity not all are discussed in this paper. In the initial listing in paragraph 5, the first reference is to the numbered or bulleted lists of features of good practice at the end of each institutional audit report, the second to the relevant paragraphs in Section 2 of the Main report. Throughout the body of this paper references to features of good practice in the institutional audit reports give the institution's name and the paragraph number from Section 2 of the Main report.

It should be emphasised that the features of good practice mentioned in this paper should be considered in their proper institutional context, and that each is perhaps best viewed as a stimulus to reflection and further development rather than as a model for emulation. A note on the topics identified for the first series of Outcomes... papers, to be published throughout 2005, can be found in Appendix 3 (page 17).

This first series of Outcomes... papers is based on the 70 institutional audit reports published by the end of November 2004. The second series will draw on institutional audit reports published following the 2004-05 audits, and it is likely that there will be some overlap in topics between the first and second series. Papers in each series are perhaps best seen as 'work in progress'. Although QAA retains copyright in the contents of the Outcomes... papers they can be freely downloaded from QAA's website and cited, with acknowledgement.

Assessment of students: introduction and general overview

1 This paper is based on a review of the outcomes of the first 70 institutional audits published by 5 November 2004 (see Appendix 1, page 14). A note on the methodology used to produce this and other papers in the Outcomes... series can be found in Appendix 4 (page 18).

2 Although the structure of institutional audit reports does not require auditors to comment on the principles or practices institutions use to judge the academic performance of their students, almost all the reports published by November 2004 referred to matters relating to assessment and the classification of degrees. This is not unexpected given the relevance of the ways in which institutions assess the work of their students to maintain academic standards. However, it does mean that consideration of assessment arrangements in the reports can be found in different contexts, though the locus for the majority of the most significant comments was the section of the report which describes and evaluates the framework used by the institution for managing quality and standards.

3 Institutions used that framework to gauge the effectiveness of student learning, but there were also important observations on assessment matters in those sections of the reports which considered the ways in which institutions support and guide learning. Academic guidance and support is addressed in another paper in the Outcomes... series, and some of the material it refers to is mentioned in this paper.

4 There is also some common ground between what these reports said about assessment, and what they said about the way institutions used data about the progress students made in their studies and what they achieved as a consequence of them. The use of progression and completion data by institutions in managing quality and academic standards is also the subject of an Outcomes... paper, and again it is likely that it will refer to material used in this paper.

Features of good practice

5 Features of good practice in relation to assessment are cited in the Findings sections of 10 institutional audit reports. They reflect strengths in the provision of information and guidance, in the use of assessment boards to assure academic standards, in the development of assessment criteria, in the evaluation of assessment outcomes, in the use of assessment to support student learning, and in the review and development of assessment regulations. They include:

the report of the 'Project on Indicators of Academic Performance' [University of Cambridge, paragraph 202 i and paragraphs 62 and 165]
the use of diagnostic assessments in responding to difficulties in retaining students [School of Oriental and African Studies, University of London, paragraph 299 i and paragraph 184]
the provision of effective and timely feedback on students' assessed work, particularly the return of marked examination scripts [The School of Pharmacy, University of London, paragraph 159 v and paragraph 99]
the use being made of programme specifications to specify learning outcomes and criteria of assessment [University of Lincoln, paragraph 268 i and paragraphs 74 and 207]
the procedure of returning coursework to students through the adviser system in order to encourage discussions about student progress [University of East Anglia, paragraph 275 and paragraph 119]
the way that the University monitors the security of its academic standards through the [Results Ratification and Awards Classification Panel] [Open University, paragraph 207i and paragraphs 38, 41]
the comprehensive and regularly updated guide to policies and procedures provided for chairs of boards of studies [University of York, paragraph 198 iii and paragraph 28]
the Guide to Assessment Policies and Procedures which is updated systematically and is circulated both on a university-wide basis and to external examiners [University of York, paragraph 198 iii and paragraph 53]
the work undertaken with partner institutions to develop assessment arrangements [University of Brighton, paragraph 255 iii and paragraphs 150 and 163]
the recently published Guidelines for Good Assessment Practice [St Martin's College, Lancaster, paragraph 217 ii and paragraph 30]
the effective use of management information in the context of assessment [St Martin's College, Lancaster, paragraph 217 viii and paragraph 93].

The Academic Infrastructure

6 One of the purposes of institutional audit is to investigate what use is made of the QAA's Code of practice. Section 6 of the Code of practice, which was first issued in May 2000, is concerned with the assessment of students. It sets out a number of precepts concerning the principles and practice of assessment, which are intended to help institutions demonstrate their responsibility for the academic standards of their awards and qualifications. Many institutional audit reports referred specifically to this section of the Code of practice, with several citing evidence that it had led to the development or modification of policy, practice and guidance for assessment. Some reports, however, have indicated that further consideration of the precepts of Section 6 would be beneficial in order that institutional responsibility for academic standards can be discharged more effectively.

7 The FHEQ was published in January 2001. By providing points of reference for setting and assessing academic standards it too has affected the development of assessment policy and practice. In particular, it has informed the development of consistent and clear criteria for the marking and grading of assessments, and the attention given to ensuring that assessment tasks match the qualifications sought.

Themes

8 In the audit reports considered, the major themes emerging from the recommendations relating to assessment of students were:

the development and use of consistent assessment policies
the classification of undergraduate degrees
the operation of assessment boards
the security and reliability of assessments
the use of assessment criteria
the provision of feedback on students' work.

The development and use of consistent assessment policies

9 Section 6 of the Code of practice advises that institutions should design, approve, supervise and review assessment strategies (Precept 1), and that the 'principles, procedures and processes of all assessments should be explicit, valid and reliable' (Precept 2). In response to this advice, institutions had frequently adopted guidelines for assessment practices covering matters such as anonymised examination scripts, second-marking or moderation, the availability of re-assessment when a student has not achieved a pass mark, penalties for late submission of assessed work, and the method of aggregating marks to reach an overall graded result for a programme of study.

10 Several reports indicated that institutions had given careful attention to the development of assessment arrangements that were consistent and fair. These included: Aston University [paragraph 32]; University of Lancaster [paragraph 38]; University of Leeds [paragraph 31]; University of Brighton [paragraph 26]; University of Bradford [paragraph 32]; University of Salford [paragraph 52]; Surrey Institute of Art & Design, University College [paragraph 40]; University of York [paragraph 28]. In some cases, however, this had involved protracted discussions and thus led to delays. In a number of cases reports referred to comments in self-evaluation documents indicating that institutions were alert to potential problems or deficiencies in their assessment arrangements. Several reports, for example, referred to work in progress on reviewing or revising aspects of assessment practice. In such cases, it was suggested that institutions should consider giving priority to the completion of the work.

11 Where institutions had yet to agree and implement principles of assessment, audit reports have recommended that the work needed to reach conclusions and to ensure that practices conformed to them should be given priority.

12 A prominent theme in many reports was the difficulty institutions had encountered in balancing the need for some central control of assessment practices - in order to ensure adherence to relevant elements of the Academic Infrastructure - with the need for some measure of autonomy at a departmental, school or faculty level so that disciplinary needs could be met, including the requirements of professional and statutory bodies.

13 It was apparent that in a number of institutions where there was significant variation in established practices, agreement about certain aspects of assessment policy had not yet been. Many reports drew attention to the implications of variation in aspects of assessment practice for quality and standards, and for consistency and equity in judging students' performance. As indicated in one of these reports, there is a view that conformity to established practices within disciplines across different institutions is more important and appropriate than conformity to guidelines within an institution. It was widely acknowledged, however, that differences in assessment practices within an institution could be problematic.

14 One example of the difficulties is the effect of such differences in assessment practices for students on combined, joint or dual programmes of study. Another example is the effect on academic standards of differences in assessment practices between programmes provided within an institution and comparable programmes provided by a partner. In such cases, and in a number of similar instances, the respective reports concluded that in one or more aspect of assessment practice there was a need for greater consistency.

15 Audit reports for some institutions noted that the agreed guidelines for assessment practices permitted a degree of flexibility in assessment arrangements, justifying comment on the compatibility of the flexibility allowed with the responsibility to ensure that students are treated consistently and fairly. Again, particular attention was drawn to the consequences of flexible interpretations of guidelines for students following combined or joint programmes, which required study in different faculties, schools or departments. For example, a policy for penalties for the late submission of coursework which permits variability in how a late submission is defined could lead to inconsistent treatment of students. Another report noted that discussion of candidates' cases at boards of examiners could happen before or after marks had been identified with individuals. Inequitable treatment can also arise, according to a further report, if there are significant differences in the consequences of a fail mark in a final-year module for students in different faculties.

16 Where guidelines had been agreed, some audit reports stated that it was not always possible to be satisfied that they were given sufficient appropriate publicity. One report recommended the desirability of publishing a code setting out the agreed guidelines. In another the institution was advised to publish information enabling students to understand how their academic performance was assessed and how their degree classification was decided. A further report advised the institution 'in the interests of improving transparency in the information provided to students, to expedite the process of determining those aspects of assessment policy that should be universally applicable and either incorporated in the Academic Regulations (for taught awards), or standardised across schools' assessment policies'.

17 Elsewhere, one audit report recommended the desirability of the institution specifying minimum requirements for the information provided in handbooks to students about regulations and assessment criteria, and another the desirability of ensuring that 'information available to students on course structure and assessment is expressed clearly and can be easily understood'. In its response to the audit report, published as an appendix to the report itself, one institution indicated that in addressing a recommendation of a similar nature, it 'will review the clarity and consistency with which information on assessment processes is provided to all students'.

The classification of undergraduate degrees

18 Variability in methods used to determine the classification of undergraduate honours degrees is mentioned in several reports. Some institutions have made progress towards reducing the extent of variability by permitting only a limited number of ways to determine classifications. Elsewhere, however, the scope for differences is wide and reports have drawn attention to its implications, particularly with reference to equity and consistency across the institution.

19 In one institution 'students in different faculties had different academic hurdles to jump to achieve an honours degree'. This was justified by staff on the grounds that 'it was more important to ensure equity nationally within the discipline rather than within the University'. The report noted, however, that in the institution concerned rules applying to postgraduates had been harmonised across all faculties.

20 Several reports noted that where different methods for classifying undergraduate degrees were permitted, the potential for confusion and unfairness was especially marked for students taking joint or combined degrees in departments using different marking scales and different award classification systems. For example, in one report it is noted that some joint honours students were 'significantly less likely to achieve a First class honours degree' than single honours students. In another report, the institution was advised 'to strengthen arrangements for ensuring parity of treatment for combined honours students whose programmes cross schools with those whose programmes operate within a single school, given the scope for variation in the content of school policies and the format of documentation given to students'.

21 Difficulties in reaching consensus about the methods used to classify undergraduate degrees were reflected in a report that referred to a recommendation of the previous academic quality audit. The earlier audit stated that the institution should establish means for assuring the parity of academic standards and greater consistency of assessment and classification practice. Disparities, it observed, would have particular relevance for the awards attained by combined honours students who almost invariably work within different assessment conventions in each of their principal subjects. However, the report confirmed that the features which had given rise to this recommendation were still observable, and concluded that the institution's assessment conventions would benefit from renewed attention, particularly since consideration of this matter had been overly protracted.

22 Another report noted that the use of different marking scales and award classification systems was a source of confusion to dual degree students working in departments using different marking and classification procedures. There had been some discussion within the institution on the need for appropriate representation on boards of examiners where the classifications of dual degree students were determined, but the report observed that this was not reflected in the approved Recommendations for action.

23 In some reports there are indications that institutions have given, and are continuing to give, consideration to greater uniformity in the methods used for determining degree classifications. In general, reports have endorsed the progress made on the grounds that it has helped avoid inconsistent treatment of students.

24 One report indicated that the institution had succeeded in harmonising a large number of separate classification and marking schemes. A single scheme had been agreed and it was suggested that this development would 'have the effect of increasing consistency in the student experience and making more transparent comparability between programmes of academic requirements and standards'.

The operation of assessment boards

25 The operation of boards responsible for assessing students was mentioned in some reports. In one case, variability in the use of discretion by boards of examiners was noted and, as with an issue identified in a previous audit report, the relevant report concluded that 'inconsistency was still possible in the treatment of students studying different schemes'. In another report, with respect to an institution's work with a collaborating partner, attention was drawn to evidence that 'concerns expressed by external examiners...about the lack of criteria for the use of discretion by boards of examiners in classifying students' work and recommending the award of qualifications, have remained unresolved over successive years'.

26 Processes for assuring the appropriateness of examination papers are relevant in ensuring that academic standards are secured. One report noted evidence that boards of examiners did not always follow the required processes with sufficient rigour. The report recommended that the institution should take steps to 'ensure that its assessment procedures in respect of the internal scrutiny of examination papers are always observed and that there is always prior external scrutiny of examination papers leading to an award'.

27 Institutions use boards or panels of examiners to determine, or to recommend, the awards students receive on completion of their programme of study. There can, though, be questions about the formal authority of such boards or panels, and particularly about their relation to an institution's principal academic committee - usually a Senate or Academic Board which formally has the power to award qualifications. One report observed that boards of examiners, including those used in partner institutions, were awarding degrees without reference to any other board or committee, even though the formal documents governing the institution make no provision for boards of examiners. In this case, the institution was advised 'to review the formal relationship between boards of examiners and the Senate to ensure that the respective responsibilities of the Senate and the boards in the award of degrees are secure, and that the Senate has the ability to maintain an effective institutional-level overview of the work of the boards'.

28 One report commented on the role of the institution's assessment boards in maintaining academic standards. In this case a senior committee reporting directly to the institution's senate has the power to require an examination board 'to revisit its results-list when it appears statistically anomalous, or where there is any other cause for concern'. Here, the report found that 'at all levels' the institution took 'great pains to assure the standards of student performance,' and identified the mechanisms used to ensure the reliability of assessment outcomes as a feature of good practice [Open University, paragraph 41].

The security and reliability of assessments

29 Comments in the institutional audit reports on the security and reliability of assessment practices were largely concerned with the operation of arrangements for second (or double) marking, or moderation of students' work. Some reports indicated that, in accordance with the precepts of Section 6 of the Code of practice, guidelines for this aspect of assessment practice had been agreed and published. Precept 8 of the same section advises institutions to 'ensure that there are robust mechanisms for marking and for the moderation of marks'.

30 In one institutional audit report, however, evidence was noted which suggested that some strengthening of the systems used 'would help ensure that all levels of work, including year one, and all types of work, including theoretical pieces, are included in a systematic approach to internal moderation'. In another report, attention was drawn to doubts about the institution's policy of double-marking for assessments on distance-learning programmes.

31 There are comparable comments in another report where the institution was invited 'to satisfy itself that its current arrangements for double-marking...are entirely consistent with the expectations of Precept 8 of the section of the Code of practice relating to assessment'. Similar comments are also to be found in reports which noted that both students and external examiners had expressed doubts about whether double-marking and/or moderation was always used. In one such case (which could equally stand for a number of others) the report recommended that the institution 'consider the advisability of clarifying its requirements...with respect to the purpose and conduct of double-marking and/or moderation, and satisfying itself that, in all cases, double-marking and/or moderation are being carried out...in line with its requirements'.

32 In another report, auditors found that where moderation was used to provide reliable verification, marks of sampled work could be changed without affecting the marks given to the work from which the sample was drawn. This, it was suggested, could lead to 'inconsistency in the treatment of students who found themselves included or excluded from the sample used for moderation purposes'.

The use of assessment criteria

33 Precept 7 of Section 6 of the Code of practice advises institutions to 'publish, and implement consistently, clear criteria for the marking and grading of assessments'. A consideration of the institutional audit reports published by November 2004 shows that although many institutions are publishing and using clear and consistent assessment criteria, at least 20 per cent of institutions were not able to demonstrate their adherence to this precept.

34 One institutional audit report, for example, indicated that consideration of this precept had resulted in 'several sets of marking and grading criteria'. Institution-wide assessment criteria, however, had not been developed and the audit report suggested that this made it difficult to assure the consistency the precept calls for. The institution was therefore advised to 'define assessment levels to ensure consistent standards across the University'.

35 Programme specifications typically set out intended learning outcomes which students are expected to achieve. Reports which draw attention to assessment criteria and methods have indicated that, for the benefit of students and examiners, it is helpful if they are explicitly aligned with these learning outcomes. The usefulness of programme specifications, and the security of marking procedures, is thereby enhanced. For example, one report advised that each programme specification should provide 'a concise description of the intended learning outcomes, the means by which these outcomes are achieved, and the assessment methods and criteria employed'.

36 It is also helpful if assessment criteria and methods can be used to demonstrate articulation of a programme with the qualification descriptors of the FHEQ. As one report suggested, 'programme specifications and the procedure for approving them would benefit from attention to the ways in which learning outcomes, and the assessment methods used to demonstrate their achievement, can be matched to qualification descriptors'.

37 A consequence of the identification of learning outcomes in programme specifications is that some students may need more than one opportunity to show that they can satisfy them and thereby justify the award of the qualification to which the programme leads. One report noted that in recognition of this there is more opportunity for students to 'retrieve failure through reassessment', though it also noted that since this opportunity is not uniform across the institution there is scope for inconsistent treatment of students.

38 Where institutional audit reports found good practice in the use of assessment criteria they encouraged its dissemination. For example, one report indicated that there was 'very good practice...in the development of both generic and discipline-specific assessment criteria' in provision arising from collaboration with a partner. It was suggested that the approach the effectiveness of this practice demonstrated could usefully be extended to other collaborations and to the provision of the institution's own schools and departments.

39 There are specific contexts in which the identification of assessment criteria has presented institutions with particular challenges. In one report (on an institution which faced difficulties in 'articulating standards in relation to such key attributes as "creativity" and "imagination"'), auditors learnt that the institution encouraged academic staff to adapt perceived good practice to their own contexts. They also learnt, however, that there was 'variable clarity of statements on assessment criteria' in course handbooks, and 'difficulties encountered by a final examination board in determining on what grounds a student's performance was to be judged unsatisfactory'. The institution was advised 'to expedite the formulation and implementation of institution-wide performance descriptors linked to assessment criteria which will provide a secure basis for the determination of examination outcomes'.

Feedback to students on assessed work

40 As indicated in the Introduction to Section 6 of the Code of practice, an important purpose of assessment is to enable students to obtain feedback on their learning and to help them improve their performance. A substantial number of the institutional audit reports reflect this purpose either by identifying feedback arrangements as a feature of good practice, or by recommending that improvements in those arrangements would be advisable or desirable. In one report, the institution was recommended to consider 'establishing and implementing clear requirements for the timely provision to students of formative feedback on coursework to bring the approach in all modules up to the level of good practice apparent in much of its provision'.

41 In some cases where good practice in providing feedback had been implemented, institutional audit reports note that this has featured in the written submission students had provided to support the audit. For example, in one such case students had observed that the character of the feedback they received increased their confidence in the 'transparency of the assessment process'. The report for this institution drew attention to the value to students of a particular feature of good practice in feedback arrangements, namely the provision of feedback on examinations as well as on coursework [School of Pharmacy, University of London, 99].

42 Precept 12 of Section 6 advises that to 'promote learning and facilitate improvement', feedback on assessed work should be provided to students in a timely manner. Some institutional audit reports have, however, noted occasions when 'assessed coursework was not returned until after the examinations in the module or subject had taken place, ensuring that the feedback had little or no value for learning'. Overall, more than one in 10 of the institutional audit reports published by November 2004 recommended to institutions the advisability or desirability of ensuring that feedback is provided to students in a consistent and timely fashion.

43 On the matter of consistency in the provision of feedback, some institutional audit reports noted significant variability in both the timeliness and the quality of feedback to students. One report indicated that even though the institution had produced guidelines and advice on good practice, there remained a need to ensure more consistent practice, and it advised that procedures should be established to assure the institution that the standards it had set were being achieved.

Progression and achievement

44 One institutional audit report was able to identify an instance of good practice in the use of data regarding the performance of undergraduates and the factors that might influence degree classifications. This took the form of an internal report, based on a number of different sets of data, which enabled consideration of the extent to which students' performance in examinations varied between different social and ethnic groups. The internal report also considered the effect of gender, type of school attended and entry qualifications. This internal report was described in the audit report as 'innovative and comprehensive', and as demonstrating the institution's willingness to give careful consideration to the effects of the assessment methods it used [University of Cambridge, paragraph 62].

45 In another report, auditors found that data concerning the effects of new teaching and learning arrangements on students' progression and achievement was being collected and analysed within one of the schools. The conclusions reached informed quality management and, in this particular school, constituted a feature of good practice [University of East Anglia, paragraph 177].

46 Elsewhere other institutional audit reports found that comparative data relating to the outcomes of assessments was either not available or, though available, was not used to inform debate about academic standards. For example, one report noted that although the institution used descriptors to 'express what a school or faculty understands by the different classification bands', there did not appear to be any instances of systematic comparative analysis across the faculties. In this instance the audit report concluded that it would be advisable for the institution to articulate 'how it monitors and evaluates comparability of standards across the breadth of its provision'.

Conclusions

47 It is clear from the institutional audit reports published by November 2004 that institutions have given much careful consideration to the development of their assessment arrangements. There has been sustained, and sometimes protracted, discussion of consistency in assessment practices, security of assessment outcomes, qualification descriptors, generic and subject-specific assessment criteria, and the provision of timely and effective feedback to students. In cases where auditors have indicated that further development is needed, the reason would often appear to be a legitimate desire on the part of institutions to ensure that the character of different disciplines, and different disciplinary practices, are not ignored. Some reports show, however, that these difficulties can be overcome.

48 The large number of recommendations, in comparison with the small number of features of good practice identified by auditors, reflects the contents of the reports. For a substantial number of institutions, further work in the development of assessment arrangements was judged either advisable or desirable. In many of these institutions there was evidence, either in the reports themselves, or in the institutions' comments on the reports published as appendices, that this work was already taking place or was continuing.

Appendices