The University of Sydney Business School is piloting a new system for delivering media-rich feedback to students with the aim of enhancing engagement and personalisation. A custom process has been developed that allows educators to create stratified video feedback quickly and at scale. Augmenting one of the text-based feedback systems currently used in the School with an integrated audiovisual workflow, educators are now able to provide individualised video feedback to students, even across large cohorts. With potential to enhance engagement, understanding and teacher presence, this is an exciting step forward in improving the learning experience for students.
We spoke with educator Robyn Martin, as well as Boyd Britton and Andy Eisenberg from the Business Co-Design team, about the development of a new automated video feedback tool, Vidfwd and what it might bring to learning environments for both students and educators.
Vidfwd: an automated video feedback tool
In broad strokes, could you explain how feedback is currently delivered to students in large cohorts?
RM: There are a variety of systems and techniques currently in use. One option that some educators have adopted is SRES, which stands for Student Relationship Engagement System. This tool, developed by the University’s Educational Innovation team, allows educators to collate student data, such as attendance, grades or live-feedback, and then create customisable and personalised messages to send to select groups of students based on this data. Students receive the communication via email or web portals that can be embedded into Canvas. Some educators use also rubrics on Canvas, our learning management system. Various units have additonally been trialling FeedbackFruits, another LMS-integrated platform for student feedback in large cohorts.
In your view, what are the main challenges of providing feedback across large cohorts?
RM: All the literature highlights that feedback needs to be timely and specific. If the process of providing it is too onerous, or the critique is vague, then it’s unlikely students will yield any value from it.
From an educator’s perspective, calibration is always challenging. When there are multiple educators involved, clear communication is crucial to achieving consistency, otherwise, rubrics may be misinterpreted and resources become wasted simply in trying to get it right (Bloxham, 2009).
What are your thoughts regarding the pedagogical benefits or otherwise of video feedback?
RM: Video can be very effective at building an emotional connection between educators and students – making them feel heard and ameliorating feelings of isolation. So I think feedback in a media format could help to foster stronger relationships between academics and students. If we look at Felten and Lamberts’ work (2020), we can understand how improving this academic-student connection has a potentially profound effect on student success, especially in the undergraduate environment. There is another advantage in developing student-academic connection in the area of academic integrity, as we know students are less likely to ‘cheat’ if they feel they have a relationship with their educator (Bretag et al., 2019).
What was the reason for developing Vidfwd?
BB: There were several reasons. Firstly, we know that video feedback can improve engagement (Jones et al., 2012; Thompson & Lee, 2012). We also know it can enhance understanding (Borup et al., 2014) due to the medium’s visual and temporal properties. It can lend authenticity (Borup et al., 2014) and be more motivating for students (Anson et al., 2016; Harper et al., 2012), who feel recognised and valued (Vincelette & Bostic, 2013). Research has also shown that video can foster greater personal connection between students and their educators (Henderson & Phillips 2015; Anson et al., 2016). For all these reasons, it’s unsurprising that for many students video is their preferred format for feedback (Lamey, 2015; Moore & Filling 2012).
In broad strokes, could you explain what kind of media artefacts the Vidfwd system can produce?
BB: Vidfwd allows educators to provide personalised video feedback at scale. This feedback can be stratified to address a spectrum of outcomes in terms of student performance. In the example below, three tiers have been created: no participation, some participation and full participation.
In this case, each video comprises three key variables: a piece to camera component, opening title with the student’s name, and the student’s results.
Additional tiers and variables can be added and customized to suit the specific needs of a unit or cohort. There is also no limitation on the number of student recipients. An educator could, for example, create stratified video feedback for thousands of students in a matter of minutes. Rendering time is the only real bottleneck, however this part of the process can happen in the background, without the need for input from the user.
How was Vidfwd developed?
AE: The research process was two-fold. Firstly, there was user interaction research done in the form of looking at how users interact with the engagement program (SRES), and then technical research done in the form of coding and scripting. The user interaction research was done by investigating how the engagement system worked and then figuring out a process for getting the relevant data from our learning management system into individually created videos that are then uploaded to a video site.
After a workflow was sketched out and documented, it became a process of writing a custom program to execute the steps that needed to be automated. For that, we researched additional extensions for After Effects (to input the data and create individual videos), building a pipeline straight to Vimeo (to automatically upload videos and return their relevant links) and then converting that data into a format that the engagement platform can read.
What software is integrated into the workflow?
AE: The engagement platform is the start and end point – in this case we are using SRES. It holds the initial data and then accepts the video links at the end.
Adobe After Effects is used to build the initial templates and then a custom build each individual video with its specific student data.
A Node.JS program is then used to upload the videos directly from the workstation to the Vimeo server and also create a list of those videos with each student’s name linked together.
How does the process work?
AE: I can outline the process step-by-step!
Step 1: The educator exports the data required for the videos from the engagement system (SRES) with a student ID number. The SID number allows the system to track and maintain the individual student’s data through the process.
Steps 2 – 5: These steps are automated. The data is imported into After Effects and is parsed with a custom script. From here, the program will read the data and create a different video for each student depending on the data input. Stratified media is batch exported and uploaded directly to the video platform of choice. This step will also create a ‘Return List’ (a document containing the links for each video uploaded to Vimeo with relevant SID number).
Step 6 – Using the return list that has been created, the educator uses the engagement platform’s importer to connect the links back to their original data.
Step 7 – The individual video links now exist in the engagement platform, as a unique column, which can be used in email templates or canvas embeds.
What are the next steps for Vidfwd?
BB: We are currently refining the style options available to educators. The example above was filmed in-studio, but we are exploring remote variations of this workflow as well. Lowering the production threshold may help to give the feedback a more personal flavour, in addition to making the process more accessible to educators.
We are working on integrating a feedback loop, so the system can be used to foster dialogue between students and educators, rather than simply information transmission. There may also be a role for avatars, for example, as an alternative option for the small percentage of students who find video feedback too direct.
Preparations are underway to trial the system with a small unit and to continue to develop and expand the initiative based on feedback from students and educators. Currently the process works with SRES, however it could be modified to integrate with other feedback platforms also.
What are your thoughts as an educator regarding the solution/prototype that’s been developed?
RM: I think it’s a great initiative and definitely a step forward. One challenge to consider going forward will be managing students’ expectations regarding the level of personalisation that the system affords. For example, you wouldn’t want students to be disappointed if they discover a classmate has received the same video and feedback, albeit with a different name in the title. So, I think it’s a question of transparency and setting that up front with students. I also feel the challenge of allocating time either prior to semester or pre-assessment to produce these materials could be challenging at the outset, however, once produced these resources could be utilised across semesters (assuming the assessments/unit structure etc. remains similar).
What kind of applications / situations could you see this kind of system being used for?
RM: I can definitely see applications for this in the formative assessment space. I think the system could also be an effective way of contextualising grades, that is, providing qualitative feedback to students ahead of time, so that they better understand the reasons why they received a particular score. Utilising this resource in this way, could potentially reduce appeals at the end of semester as students receive some indication of grades via the formative video feedback which helps prepare them when their grade is released.
For more information about Vidfwd please leave a comment below!