How to improve your LMS: Formative online checkpoint test to drive engagement

Practical eLearning Tool Pedagogy

These days it seem to be all about iPads. I am not convinced that the shiny stuff makes for good learning. You simply just don't need the best toy to learn the most. 

This is  an example guide of how use simple eLearning tools. It is deliberately not revolutionary; just very effective. 



Problems:

• Student boredom –Subject is Advanced Accounting Theory after all
• Poor textbook purchase rate, reading previously almost non-existent
• Poor sustained student effort – tendency to cram
• Student tendency for “collaborative rote” – commonly known as “cheating”

Goals:

• Increase student engagement, earlier
• Create a “natural” learning experience
• Provide early warning for students-at-risk
• Follow sustainable assessment concepts (Boud, 2000)
  • Spread formative assessment across the semester
  • Flexibly allow for student sickness, interruptions, etc
  • Not to increase teacher marking loads

Strategy & Implementation: formative assessment.

• Students are set a weekly quiz (checkpoint test), online via the LMS
• Each checkpoint test is between 10-14 MCQs
• The average of all online quizzes constitutes 10% of final grade
• Students can do this whenever they want, but the longer they leave it, pressure mounts
• Each checkpoint test unlocks the next topic:
  • Student must do the quiz for week 1 to get to week 2 learning material. 
  • Notes, handouts, powerpoint slides, suggested tutorial solutions are all hidden until student engages online
  • Prevents cramming
  • Encourages studying and reading before progressing
• Questions are grouped by learning objectives to ensure complete subject coverage
• Only one attempt, time limited to 1 hour
  • Enough time to consider each response
  • Most students take between 20 to 30 minutes to complete each checkpoint test
• Both question order and multiple choice answer order is randomised
• Questions are difficult and can be VERY specific about the textbook
• Students receive an “early warning notice” in week 8 if they have not yet attempted 2 checkpoint tests or have not yet logged on
• Only the final mark is provided
  • It’s not sustainable on the academic to endlessly rewrite new questions
  • Exact answer right/wrong not automatically provided, but are provided for viewing at face to face sessions where guidance is given on the spot
  • Encourages students to be self-critical
• It’s simple & elegant, everyone understands what they have to do
• Scatter plots of relative class performance are provided on the LMS
• Tool is equally effective for online courses as it is for blended delivery

Result: Successful implementation (past 3 semesters)

• Students sit the test with textbook in front of them
  • Students effectively forced to read the textbook closely, or at least, search for details in short periods of time
• (an important professional / academic skill)
• Students without texts are effectively penalised
• (Lesson: a professional must be equipped with the right tools)
• Students collude – groups of 4 are common
  • This actually deepens their understanding since they are now exposed to 40 questions rather than just 10 each
• No doubt, there is group discussion
• The student’s natural tendency to “cheat” tricks them into learning more, in a more natural setting
• Validation: Because of the open book / MCQ nature, the task has a very low weighting (only 10%)
  • Enough to make it matter to students…
  • …but not enough to be a decisive factor in a pass / fail situation
  • This assessment tool does not assess competence by itself, rather it rewards interaction 
• Authentication: “cheating” aspect is minimised since it is only worth 10% of the final grade
  • Each student maintains their own incentives to remain engaged since the mark goes against their name.
• To the student, it (deceptively) feels like standard summative assessment
  • Compare to “assess a reflective blog on Moodle”
• Students agonise over their MCQ result despite the low weighting
  • This encourages deeper learning earlier on
  • Low scores are used as early warning indicators
• Online formative MCQs reduces the exam weighting, diversifying the mechanisms a student demonstrates achieved learning
• Anecdotally, exam responses have significantly improved with deeper understanding
• I acknowledge I do not have quantitative evidence to prove the efficacy of these claims; however I am looking for research opportunities for verification.

If you have any queries or would like to find out more about this program please do not hesitate to contact me via email phillip.wong@holmesglen.edu.au

Reference: 

Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education , 151-167.