There are five levels of consensus moderation used at Griffith University:
- Course level planning to ensure the use of appropriate assessment items and assessment regime
- Setting appropriate and consistent standards for marking individual pieces of students' work
- Reviewing the standard of the final course grades to ensure they are appropriate and consisent
- Engaging appropriate and consistent standards between your course and other cognately similar courses (at Griffith and Other Institutions)
- Ensuring appropriate and consistent assessment standards over time
Universities are now required, as part of their accreditation process, to demonstrate how they ensure that adequate standards of assessment and learning outcomes are met. One way that we need to do this is through moderation of assessment. Course convenors are required to engage in moderation of assessment practices in at least one of these levels; every time a course is offered. You will discover that this isn’t as onerous as it sounds.
To identify the consensus moderation practices (processes) you use, or expect to use, in this course; please select one or more of the check boxes at each level in the relevant section of the course profile. The information you provide is then collated to provide a report on overall university practices in moderation of assessment.
This information is not copied across from the previous offering of this course. You will be unable to change the status of the profile from draft to submitted until this section has been completed.
If you do not expect to undertake any consensus moderation at a particular level, tick the last box of “None”.
Introduction to consensus moderation
What is consensus moderation?
Consensus moderation of assessment is the processes used to reach a general agreement about what quality assessment and its outcomes “looks like”; it ensures that the judgments of students' performance are consistent and have the same “meaning” irrespective of time, place or examiner.
What is the purpose of consensus moderation?
The fundamental purpose of consensus moderation in assessment is to ensure appropriate and consistent quality of assessment and its outcomes.
We need to be sure that graduates do in fact have the knowledge and skills that we claim they do; and at the level of proficiency we claim they do.
To ensure the quality of assessment and its outcomes consensus moderation needs to be applied throughout the whole assessment design process, not just at the marking or grading level. W e need to assure that:
- We assess the appropriate learning; in a valid, reliable and fair manner
- Grades (and qualifications) awarded accurately reflect students' levels of learning achievement
- Standards across our courses, programs and those offered by other institutions are comparable
- Our standards remain consistent (there is no "slippage" over time)
Consensus moderation becomes an integral part of assuring academic standards, which in turn guarantees the credibility of qualifications and helps protect the academic reputation of the University.
How do we conduct consensus moderation?
A big step towards gaining agreement and achieving consensus about quality of assessment is by using the principles for good assessment and best practice to guide you. See Assessment Matters https://app.griffith.edu.au/assessment-matters/ to view the principles for good assessment.
Consensus moderation of assessment is most commonly conducted (although not exclusively) via a peer review process. The aim is to reach a shared understanding, or consensus about what constitutes quality.
Peer review can occur before, during and after the assessment.
The person with the most experience usually leads, but should not dominate the peer review discussions. Their expertise helps guide and 'calibrate' those less experienced with the appropriate discipline standards, without exerting authoritative control. The process is a collegial one which recognises the value each person brings to the process. The peer review process may be internal or external; or both.
As a sole convenor or sole examiner, you are not exempt from the processes of consensus moderation. It can be argued that the need for consensus moderation may even be higher for academics who operate in isolation. The judgments of a single academic must still be calibrated and benchmarked to assure the quality, consistency and comparability of standards.
In such situations, you can engage in a peer review process with colleagues or peers from cognate (similar) courses.
For further information about best practice and consensus moderation of assessment; visit the Assessment Matters website at: https://app.griffith.edu.au/assessment-matters/
Levels of consensus moderation explained
Level 1: Course level planning
This section asks you to identify the one or more consensus moderation processes you apply when developing your course assessment plan; to ensure the assessment tasks, together with the assessment pattern (number, timing, weighting and sequence of assessment), are designed to both support and validly certify student learning.
You need to ensure that the assessment plan:
- Is appropriate for students studying the course
- Aligns assessment with the learning objectives for the course
- Also promotes the achievement of the program's specified learning objectives
- Has clarity in the specification of all assessment methods and tasks used : including their type, purpose, timing, sequence and weighting
- Is consistent with assessment plans in other similar courses (either at Griffith, or elsewhere)
- Undertake a self review of the course assessment plan by checking that the assessment is aligned with and validly assesses the course learning objectives and has clear task specifications.
Undertaking self-review of the course assessment plan is regarded as a part of the minimum required practice you engage in when preparing your course profile.
- Engage in a peer review process with a colleague where you discuss the appropriateness of the course assessment plan and/or seek advice about whether it was consistent with their professional expectations.
- Make changes to the course assessment plan initiating peer review and approval of the assessment plan by the Head of School and Dean (Learning and Teaching).
- Review other course assessment plans from cognate (similar) courses and benchmark your course assessment plan; checking for consistency and comparability. You can use internal courses at Griffith or external courses from other Universities to do this.
- Benchmark your course assessment plan through the use of an internationally recognised text that provides examples of assessment items.
Case study examples: Consensus moderation for course assessment planning
Note: A change to a course’s assessment plan represents a major change to the course. Making such a change requires the revised Course Profile and assessment plan to be forwarded using the Course Profile System to both the Head of School and Dean (Learning & Teaching) for review (see Course Approval and Review Process). This process is defined under Course Level Planning as Internal peer-review of the assessment plan.
Level 2: Individual student work
This section asks you to identify the one or more consensus moderation processes you apply to ensure appropriate and consistent standards for marking individual pieces of students' work.
Consensus moderation of marking occurs in two areas.
Moderation of the marking schemes, guides or rubrics occurs to ensure that the criteria and standards set for students’ performance are appropriate.
Moderation of the marking strategy occurs so that these standards are consistently applied when judging the level of a student's performance in the assessment task. That is, all examiners will judge a student's work the same way, for the same reasons, giving rise to the same mark. At this point we say that the examiners are 'calibrated' with each other.
- Engage all examiners in a peer review process of marking samples of students work prior to independent marking.
- Examiners first each trial-mark the same sample of student work, and then compare the marks they have (provisionally) allocated. They then discuss any discrepancies giving their reasons for marks awarded; reach agreement and each examiner then adjusts the way they award marks accordingly. This process continues until a common agreement about appropriate standards has been reached, and the marks awarded are comparable.
- Once a shared understanding, or consensus, about what constitutes quality is reached, each examiner then continues independently to mark their allocated quota of assessment items.
- Co-development of marking guides/rubrics in collaboration with peers that are then provided to students and examiners. As part of this process, you may involve examiners in the development of the marking guide; and/or hold a meeting with the examiners to discuss and reach an agreement and understanding about the meaning of the criteria and standards as applied to the assessment task.
- Benchmark your assessment marking guides/rubrics through the use of an internationally recognised text with exemplar marking guides.
Case study examples: Consensus moderation for marking student work
Level 3: Recommended course grades
This section asks you to identify the one or more consensus moderation processes you apply to ensure the standard of the recommended course grades are appropriate and consistent.
Moderation of grading occurs to ensure that all examiners agree on the reasons why a particular grade is recommended and its appropriateness. At this point we can say that the examiners grading judgments are 'calibrated' with each other.
We need to ensure that the grade recommended and awarded accurately reflects the quality of all of the student's work. Occasionally the total marks converts to a grade that is unexpected i.e. does not seem to reflect the quality of work the student has produced over the entire course. If this occurs, it is imperative to go back and review the actual collection of student work to make adjustments (not simply manipulate the marks).
- Actual students’ performance against assessment criteria (primary data) is used to determine recommended and final course grades.
Note: Manipulation of grades to fit normal grade distribution curves is not a valid process in the quality assurance of assessment and academic standards. Any modification to grades should be based on comparison of student assessment work against the predetermined assessment criteria.
- Engage in a peer review process with examiners/teaching team or colleagues; to look at the full collection of students’ work for a selected sample of students, in combination with the assessment criteria and the recommended grade. The peer review process fundamentally depends on the use of primary data i.e. actual students' work (not simply marks).
- The peer review group meets and looks at the full collection of a student's work (with the assessment criteria for each task) in combination with the recommended grade. They then discuss differences of opinion and reasons why they would or would not award the recommended grade. This continues until a shared understanding, or consensus, about the quality required for each grade and its appropriateness is reached.
- Agreement on grading is not achieved simply by averaging, or taking the mark awarded by the most senior person, or by some simple mathematical manipulation. Except for very small classes, it is impractical to conduct a peer review process for every student's grade. Thus, it is recommended that consensus moderation of grading occurs for a sample of students in each course such as:
- selection of students graded across the grade range
- students graded at all grade borders
- students awarded failing grades
- students awarded High distinctions
The peer review process may be undertaken by internal or external peers or both.
Case study examples: Consensus moderation for grading student work
Level 4: Across cognate courses
This section asks you to identify the one or more consensus moderation processes you apply to ensure that the standards required of students taking your course and similar courses (within Griffith and external Institutions) are comparable.
This means that the grading judgments are consistent, and the level of performance required for a High Distinction in a course is comparable to that required in other similar courses.
Furthermore, part of the consensus moderation process is to ensure that the standards used to reach the consistent judgments are themselves appropriate. Two cognate (similar) courses may have consistent standards, yet if the standards used in both are inappropriate; this is not an assurance of quality.
- Engage in a peer review process with convenors of similar courses; to look at the full collection of students’ work for a selected sample of students in each course.
- The convenors need enough knowledge of each other's discipline (without being experts) to be able to meaningfully discuss the appropriateness and comparability of course assessment plans, tasks and criteria; and the students' achievement standards.
- As with peer review of grading, it is impractical to conduct a peer review process on a full collection of every student's assessments. Thus, it is recommended that peer review at this level occurs for a selected sample of students in each course. The aim is to select students whose collection of assessed work will act as exemplars across the grade range.
- The aim of peer discussion is to reach a shared agreement, or consensus about the quality required for each grade and its appropriateness. Once calibrated, this process assures that the performance level required for a student to be awarded a Credit in Course A is comparable to the performance level required in Course B; and the reasons for reaching that judgment in both courses are similarly appropriate.
- Independent (internal or external) peer review of marks awarded to samples of individual student's work, or final grades. This could be done by convenors of related courses from other institutions; industry liaison groups; or through professional accreditation processes.
Level 5: Course standards over time
This section asks you to identify the one or more consensus moderation processes you apply to ensure that there is no 'slippage' of assessment standards and judgments over time.
When courses are delivered over years, it is not uncommon for small changes made over time to compromise assessment standards and quality. Staff turnover can be a further factor to consider when assuring standards over time.
Assessment judgments need to remain consistent, to ensure that the level of performance required for a High Distinction in a course in any year; is comparable to previous years; and future years.
Furthermore, as with assuring standards across courses, part of the consensus moderation process is to ensure that the standards used to reach the consistent judgments are themselves appropriate.
- Comparison of the marks awarded to examples of students’ work in your current course to the marks awarded to comparable examples of students’ work from previous course offerings.
- A library or repository of course assessment plans, assessment tasks and criteria; and annotated student work is required to effectively apply consensus moderation at this level. The assessment material and annotated student work become the exemplars (that are reused) and act as benchmarks for future course deliveries. Ideally, any annotations will include the reasons behind the decisions that were made.
- The assessment material for peer-review (from repository of prior course deliveries) can include:
- Course (and Program) assessment plans
- Assessment tasks
- Marking criteria and standards (marking scheme)
- Annotated exemplars of actual student work across the grade range
- Annotated exemplars of 'model answers' across the grade range
Case study example: Consensus moderation for standards over time