Framework for evaluating online assessment in business education: Trade-offs in promoting innovation

In the context of the rapid transition to fully online teaching and learning driven by COVID-19 (Ali, 2020), the Australian Business Deans Council commissioned a research project into online assessment in higher education in Australia. The cross-institutional project team consisted of members from the University of Sydney Business School, UTS Business and Chartered Accountants ANZ.

We based our initial investigations on five design considerations for online assessments: the assessments must assure academic integrity, support a positive learning experience for students, assure the integrity of student information, allow for the provision of quality feedback, and be delivered so that all enrolled students have an equal chance to complete the assessment successfully. From the findings of our study we added a sixth design consideration, that of authenticity. See Figure 1.

Figure 1. Framework for quality online assessment

Using input from a comprehensive literature review (Brodzeli, 2022), we engaged key stakeholders throughout the process to assist dissemination of findings (Gannaway et al., 2013). We collected 92 survey responses from staff at Australian institutions to identify and categorise their assessment practices and evaluate these against our framework. Then we held four focus groups with 19 participants to further explore current online assessment practices, barriers to innovation, and refine the evaluation framework.

Survey respondents most commonly reported translating traditional exams, quizzes and written reports for online delivery. A minority had introduced online presentations, discussions, debates, reflections and creative works. Respondents ranked academic integrity, mastering learning outcomes, equity of access and quality feedback as most important to online assessment decisions and ranked working within resource constraints and aligning with institutional assessment culture least important.

Making trade-offs in innovative assessment design

Focus group findings suggested many academics perceived resource constraints as the most important institutional driver of decisions about online assessment design and that this directly impacted capacity for assessment innovation particularly with large undergraduate cohorts. We synthesised the focus group discussion into a series of trade-offs (see report for more detail). For example, academics spoke to us about their concerns with assuring integrity in the online environment without some form of invigilation and identity verification. Most participants who used assessments where students present material synchronously (e.g. exam or live presentation) said there was a trade-off in relation to the student experience. Another trade-off example involves the need for using authentic assessment, which may involve a performance-based design (or demonstration of a skill) with an institutional policy which requires anonymous marking.

A consideration that emerged in our focus groups concerned the stress experienced by academic staff in relation to online assessment and workload. Focus group participants also raised a range of issues about the ways in which requirements of online assessment may be inconsistent with meeting the needs of learners. These concerns were exacerbated by the work environment where job security was not assured. Such concerns cannot be alleviated with advice about assessment design and require separate investigation.

Invigilation and accreditation

We were also particularly interested in invigilation practices operating in the online space as these are key considerations in business education. We noted a common misconception in relation to accreditation requirements. For example, the CPA cite “The Professional Bodies’ expectation is that at least 50 per cent of the overall assessment marks for each subject meeting the professional bodies’ required competency areas, should be invigilated, which means that a student’s identity is confirmed, and they are observed when completing assessment activities…” Many educators interpret this to be a need for an invigilated exam and with the move to online, a proctored online exam was seen as the easiest option.

Participants indicated that decisions about invigilation could be difficult due to a perceived trade-off between academic integrity concerns and concerns over privacy of student data. It was not clear from our study’s findings whether there was any leadership being enacted in this regard and how decision making was occurring. We hope this study will provide options to academics to consider ‘performance’ types of assessment as alternatives to invigilated online exams.

Figure 2. Contextual factors impacting online assessment design

Using the Framework

The results from the survey and focus group discussions led us to extend our initial five design considerations to include authenticity, and our initial two contextual factors to also include institutional policies and accreditation requirements (see Figure 2). We envisage this framework can be used in multiple ways:

  • To evaluate existing assessments, for an individual unit by the coordinator, or for a course or program as part of a general review of assessment design. 
  • To document assessment practice and trade-offs between design considerations. This may open up conversations about the tradeoffs inherent in assessment design, and the pressures that exist in certain contexts.
  • To design new online assessments or redesign existing ones. A proposed change to assessment can use the framework to guide and demonstrate the reasons for or impact of the change.

The findings highlight the importance of our research to identify and share innovative assessment solutions in business education. We have also launched an online portal to share our collected exemplars and framework and provide a forum for academics to actively engage with the findings. We strongly encourage business education practitioners to submit their own innovative assessments through our portal and use the findings from this study to progress institutional conversations of quality online assessment design.

Banner Photo by Sammie Chaffin on Unsplash

About the author

Associate Professor Elaine Huber has been designing curriculum and teaching adults for over 20 years and is currently the Academic Director of the Business Co-Design team at the University of Sydney.

Published by Elaine Huber

Associate Professor Elaine Huber has been designing curriculum and teaching adults for over 20 years and is currently the Academic Director of the Business Co-Design team at the University of Sydney.

One thought on “Framework for evaluating online assessment in business education: Trade-offs in promoting innovation

  1. Tom Worthington – Canberra – An educational technology consultant, Certified Professional member of the Australian Computer Society, and part time university lecturer.
    Tom Worthington says:

    This seems a reasonable summary of the current understanding of assessment. However the article spends a lot of time discussing exams, which in terms of authentic assessment are a waste of time. The idea that professional bodies require exams is nonsense, and they would far rather practical project based assessment. This can be done in a rigorous way, for example, using tools which record the contribution by each student, and timestamp when it was made.

Leave a Reply Cancel reply

Exit mobile version