FACILITATORS

The content below contains a bracketed and highlighted placeholder for you to insert the information relevant to your institution. Don't forget to update it and remove this box before publishing the page.

There is also a reminder to edit the downloadable version of the blueprint; this reminder may be removed from within the Word document in Files.

This “blueprint” identifies foundational instructional questions, provides a structured approach to exploring these questions with Canvas learning data and traditional classroom techniques, and suggests appropriate actions.

This blueprint is informed by Learning Analytics: Translating Data into “Just-in-Time” Interventions (Muljana & Placencia) and Sydney Brown, PhD's Dashboards for Everyday Teaching.

Download the blueprint

BEFORE THE TERM STARTS

QUESTIONS

INSTRUCTOR CHECKLIST

Am I ready to teach?

☐ Create prior knowledge check assessment(s).

☐ Have at least one graded assignment in the first four weeks, preferably one low-stakes assessment each week.

☐ Use naming conventions for assignments and content to ensure key canvas course elements (content, assignments, quizzes, discussions) are easy to identify in lists of data.

☐ Check that all assignments and quizzes have due dates and are represented in the Gradebook and published, even though they may not be available yet.

☐ Post syllabus with learning analytics statement.

☐ Create a communication plan: reminders, tips and strategies, and other communication contingent on student performance.

☐ Create a welcome message emphasizing the importance of exploring the course and doing the pre-assessment.

☐ Canvas accessibility checker has been run on page content, the link checker has been run and alerts fixed; all non-Canvas course content (PPTs, PDFs, videos) meet accessibility standards.

☐ Create a template survey for collecting student feedback through the semester.

BEGINNING OF TERM

QUESTIONS

INSTRUCTOR ACTIONS

STUDENT INDICATORS

ANALYTICS DATA & DASHBOARDS

ACTIONS TO TAKE IN RESPONSE TO DATA

Are students ready to learn?

Have students complete a prior knowledge assessment

Adequate prior knowledge

Making use of resources to remedy deficits

Review New Analytics & Gradebook:

Prior-knowledge assessment scores
Formative assessment scores

Performance on prior knowledge check

Identify knowledge gaps and provide resources for closing those gaps
Adjust upcoming instruction and content

Have students reviewed important course info?

Remind students to review the course and syllabus and ask questions

Students have opened the syllabus and key pages such as published assignments and other content that would indicate having done an overview of the course

Page views in New Analytics of resources: syllabus, key content, upcoming assignments

Participations in New Analytics: e.g. posting a question on a discussion, or sending a message via Canvas

Review student views of the syllabus and other key course pages, then use Message students who… to:

Provide a gentle nudge message or announcement to encourage students to review the missed page if they haven't already
Commend students for their preparation
Refer students to resources

Review the Canvas Guide about the Message students who... tool for help.

Release a survey to collect feedback from students about their performance

Are students aware of support services and how they can help?

[Point students toward key resources for course as well as general learning resources]

ASSESSMENT

QUESTIONS

INSTRUCTOR ACTIONS

STUDENT INDICATORS

ANALYTICS DATA & DASHBOARDS

ACTIONS TO TAKE IN RESPONSE TO DATA

How are students performing?

Are students aware of how they are performing?

Monitor performance on formative and summative quizzes along with assignments

Verify student’s understanding of grade thus far with Mid-Course Self-Check

Demonstrate conceptual understanding on formative measures

Demonstrate adequate performance on summative assessments

Demonstrate awareness of course grading scheme, their own grade/performance

Share resources or adjust class content and activities to remedy misunderstandings

Provide supplementary materials customized to student’s current performance

Scaffold learning

Embed effective study strategies

Let students know they are on track for success

Are assessments valid and reliable?

Which concepts are most difficult for students?

Review exams and analyze the reliability and validity of questions

Identify which questions/concepts are most difficult for students

Students struggle with specific items (especially top performers overall)

Review Quiz

Discrimination index
Response frequency
Difficulty index

Throw out questions with poor validity or reliability

[Refer students to resources]

Release a survey to collect feedback from students to identify circumstances that may be impacting their performance

Notices to advisors as appropriate

Provide opportunities to get back on track and recover from errors

Make a plan to remove or revise questions that do not align to Student Learning Objectives or have poor reliability before the next course offering

ENGAGEMENT

QUESTIONS

INSTRUCTOR ACTIONS

STUDENT INDICATORS

ANALYTICS DATA & DASHBOARDS

ACTIONS TO TAKE IN RESPONSE TO DATA

Are students engaged?

Review the data and implement traditional strategies to measure engagement to see if students are logging into the course and meaningfully interacting with relevant content

Identify activity gaps in the data and use traditional classroom practices to confirm that students are reading, watching, or doing the activities desired before class

Regular (for the student) patterns of accessing the course and opening pages

Assignments turned in on time; timely discussion interactions

Pages accessed, videos opened, formative assessments completed

Page views in New Analytics of resources: syllabus, key content, upcoming assignments

Participations in New Analytics: e.g. posting a question on a discussion, or sending a message via Canvas

Check in with students who might be showing low or fluctuating activity patterns

Release a survey to collect feedback from students about their performance

[Refer students to support resources]

Alerts to advisors or others on support team if the data and student feedback indicate a student is struggling to engage with the course

In Gradebook or New Analytics Message students who … to reach those who have:

not yet submitted
not been graded

END OF TERM AND BEYOND

QUESTIONS

INSTRUCTOR ACTIONS

STUDENT INDICATORS

ANALYTICS DATA & DASHBOARDS

ACTIONS TO TAKE IN RESPONSE TO DATA

Did students achieve their goals?

How did students perceive their learning experience?

Review

Final exam, project, or assignment to assess overall student learning
Course evaluations and other feedback from students

New Analytics or Gradebook

Make a plan to modify your course based on learning data and student feedback

What were the most and least used resources?

Analyze overall course usage to see which resources were most heavily and least accessed

New Analytics Page views

Make a plan to modify your course based on learning data and student feedback

When did students struggle or disengage?

Analyze performance and usage patterns

More students than expected had late submissions

More students than expected scored lower on quizzes/assignments

Students spent less time with or accessed fewer resources

New Analytics weekly course activity patterns

Make a plan to modify your course based on learning data and student feedback

PLANNING & IMPROVEMENT

QUESTIONS

MAKE A PLAN FOR COURSE ADJUSTMENT

Given the data that I have, how can I improve my course?

What changes could you make that would impact student success?
Set priorities: if you were to commit to ONE change, what might it be?

You might decide that your priority is to implement one change that would be most impactful, or you might decide to start with low hanging fruit.

What is your timeline for implementing change?
Do you need support to create or implement your plans?
Ongoing, what 3 things might you do to make using learning data part of your teaching practice?