1.4 A blueprint for using learning data throughout the semester
FACILITATORS
The content below contains a bracketed and highlighted placeholder for you to insert the information relevant to your institution. Don't forget to update it and remove this box before publishing the page.
There is also a reminder to edit the downloadable version of the blueprint; this reminder may be removed from within the Word document in Files.
This “blueprint” identifies foundational instructional questions, provides a structured approach to exploring these questions with Canvas learning data and traditional classroom techniques, and suggests appropriate actions.
This blueprint is informed by Learning Analytics: Translating Data into “Just-in-Time” Interventions (Muljana & Placencia) and Sydney Brown, PhD's Dashboards for Everyday Teaching.
BEFORE THE TERM STARTS
QUESTIONS |
INSTRUCTOR CHECKLIST |
Am I ready to teach? |
☐ Create prior knowledge check assessment(s). ☐ Have at least one graded assignment in the first four weeks, preferably one low-stakes assessment each week. ☐ Use naming conventions for assignments and content to ensure key canvas course elements (content, assignments, quizzes, discussions) are easy to identify in lists of data. ☐ Check that all assignments and quizzes have due dates and are represented in the Gradebook and published, even though they may not be available yet Links to an external site.. ☐ Post syllabus with learning analytics statement. ☐ Create a communication plan: reminders, tips and strategies, and other communication contingent on student performance. ☐ Create a welcome message emphasizing the importance of exploring the course and doing the pre-assessment. ☐ Canvas accessibility checker has been run on page content, the link checker has been run and alerts fixed; all non-Canvas course content (PPTs, PDFs, videos) meet accessibility standards. ☐ Create a template survey for collecting student feedback through the semester. |
BEGINNING OF TERM
QUESTIONS |
INSTRUCTOR ACTIONS |
STUDENT INDICATORS |
ANALYTICS DATA & DASHBOARDS |
ACTIONS TO TAKE IN RESPONSE TO DATA |
---|---|---|---|---|
Are students ready to learn? |
Have students complete a prior knowledge assessment |
Adequate prior knowledge Making use of resources to remedy deficits |
Review New Analytics & Gradebook:
|
Performance on prior knowledge check
|
Have students reviewed important course info? |
Remind students to review the course and syllabus and ask questions |
Students have opened the syllabus and key pages such as published assignments and other content that would indicate having done an overview of the course |
Page views in New Analytics of resources: syllabus, key content, upcoming assignments Participations in New Analytics: e.g. posting a question on a discussion, or sending a message via Canvas |
Review student views of the syllabus and other key course pages, then use Message students who… to:
Review the Canvas Guide about the Message students who... tool for help Links to an external site.. Release a survey to collect feedback from students about their performance |
Are students aware of support services and how they can help? |
[Point students toward key resources for course as well as general learning resources] |
ASSESSMENT
QUESTIONS |
INSTRUCTOR ACTIONS |
STUDENT INDICATORS |
ANALYTICS DATA & DASHBOARDS |
ACTIONS TO TAKE IN RESPONSE TO DATA |
---|---|---|---|---|
How are students performing?
Are students aware of how they are performing? |
Monitor performance on formative and summative quizzes along with assignments
Verify student’s understanding of grade thus far with Mid-Course Self-Check |
Demonstrate conceptual understanding on formative measures Demonstrate adequate performance on summative assessments Demonstrate awareness of course grading scheme, their own grade/performance |
Share resources or adjust class content and activities to remedy misunderstandings Provide supplementary materials customized to student’s current performance Scaffold learning Embed effective study strategies Let students know they are on track for success |
|
Are assessments valid and reliable?
Which concepts are most difficult for students? |
Review exams and analyze the reliability and validity of questions
Identify which questions/concepts are most difficult for students |
Students struggle with specific items (especially top performers overall) |
Review Quiz
|
Throw out questions with poor validity or reliability [Refer students to resources] Release a survey to collect feedback from students to identify circumstances that may be impacting their performance Notices to advisors as appropriate Provide opportunities to get back on track and recover from errors Make a plan to remove or revise questions that do not align to Student Learning Objectives or have poor reliability before the next course offering |
ENGAGEMENT
QUESTIONS |
INSTRUCTOR ACTIONS |
STUDENT INDICATORS |
ANALYTICS DATA & DASHBOARDS |
ACTIONS TO TAKE IN RESPONSE TO DATA |
Are students engaged? |
Review the data and implement traditional strategies to measure engagement to see if students are logging into the course and meaningfully interacting with relevant content Identify activity gaps in the data and use traditional classroom practices to confirm that students are reading, watching, or doing the activities desired before class |
Regular (for the student) patterns of accessing the course and opening pages Assignments turned in on time; timely discussion interactions Pages accessed, videos opened, formative assessments completed |
Page views in New Analytics of resources: syllabus, key content, upcoming assignments Participations in New Analytics: e.g. posting a question on a discussion, or sending a message via Canvas |
Check in with students who might be showing low or fluctuating activity patterns Release a survey to collect feedback from students about their performance [Refer students to support resources] Alerts to advisors or others on support team if the data and student feedback indicate a student is struggling to engage with the course In Gradebook or New Analytics Message students who … to reach those who have:
|
END OF TERM AND BEYOND
QUESTIONS |
INSTRUCTOR ACTIONS |
STUDENT INDICATORS |
ANALYTICS DATA & DASHBOARDS |
ACTIONS TO TAKE IN RESPONSE TO DATA |
---|---|---|---|---|
Did students achieve their goals? How did students perceive their learning experience? |
Review
|
New Analytics or Gradebook |
Make a plan to modify your course based on learning data and student feedback |
|
What were the most and least used resources? |
Analyze overall course usage to see which resources were most heavily and least accessed |
New Analytics Page views |
Make a plan to modify your course based on learning data and student feedback | |
When did students struggle or disengage? |
Analyze performance and usage patterns |
More students than expected had late submissions More students than expected scored lower on quizzes/assignments Students spent less time with or accessed fewer resources |
New Analytics weekly course activity patterns |
Make a plan to modify your course based on learning data and student feedback |
PLANNING & IMPROVEMENT
QUESTIONS |
MAKE A PLAN FOR COURSE ADJUSTMENT |
Given the data that I have, how can I improve my course? |
|