Course Syllabus

Course Description

This introductory course in program evaluation is intended to examine the modern field of formal program evaluation. More specifically we will look at models and methods of evaluating programs, processes, and products in a variety of settings with specific emphasis on education. The course is designed to allow you to acquire a basic understanding of the field of evaluation. Additionally, you will gain practical knowledge of how to begin a formal evaluation by designing a small evaluation.

The course is focused on answering the following twelve questions about program evaluation:

1. What is program evaluation?
2. What is the difference between evaluation and research?
3. Why evaluate?
4. What are some differing theories of evaluation?
5. What is to be evaluated?
6. Who are the evaluation audiences?
7. How is an evaluation designed?
8. What techniques can be used to collect information?
9. How may data be analyzed?
10. How can evaluation data be interpreted?
11. How can the evaluation results be reported?
12. What can be done to encourage evaluation use?

 Learning Outcomes 

Upon completion of this course, students should be able to:

• State the purpose, benefits and importance of program evaluation
• Identify possible contextual or political “barriers” to program evaluation
• Compare and contrast several different approaches to program evaluation
• Understand important aspects of most evaluation plans for a program
  • Write measurable program objectives
  • Identify evaluation audiences
  • Identify sources of and methods for collecting data
  • Develop three data gathering tools: observation form, survey, and an interview and/or focus group guide
  • Select procedures for analyzing and interpreting the data
  • Select procedures for communicating evaluation results
• Identify strategies for enhancing the utilization of the evaluation results

 Course Requirements 

Required Text and Materials

• Fitzpatrick, L., Sanders, J. R.,  Worthen, B. R., & Wingate, L. A. (2023). Program evaluation: Alternative approaches and practical guidelines. Pearson Education. ISBN: 9780137547586
• Additional readings will be available on Canvas

Technical Requirements

You will need the following in order to participate in this course:

• Computer or tablet; 
• Reliable internet connection;
• Access to Canvas using a supported web browser.

 Technical Support

For additional help with technical issues, consult:

• University Information Technology Services (UITS) (human support)
• IU Knowledge Base (IUKB) (guides)
• IUware (download free software)

 Course Assessment & Grades

Course Assessment:

1.    Read all assigned readings.

2.    Evaluation Plan: (Due: see class schedule) – 80% of grade (20% written plan; 15% alignment document with questionnaires/surveys; 40% final project; and 5% presentation)

Written Plan:

Here you and your group members (working in groups is highly recommended—and single projects need prior approval that will only be granted in special circumstances) will be tasked with developing an evaluation plan that could be implemented in an organization. The evaluation plan should be related to a current job or personal or professional interest. A detailed outline/rubric of the project will be handed out during the third class meeting.

All sections of the plan must be submitted for review in assignments consisting of two sections of the plan (see class schedule for due dates). Though a letter grade won’t be assigned for this portion, an assessment of check plus (5 points), check (4 points), or check minus (3 points) will be given to indicate level of progress. My goal for this assignment is to gauge your progress and help you complete a successful final project.

The entire plan (A-J) will be due upon the completion of the course (see class schedule for due date). A rubric that will be distributed with the assignment requirements will also include point distributions for each section.

Alignment document and Questionnaires/Survey:
For this assignment I would like you to submit an alignment document (we will cover this in class) along with all of your data collection instruments for your evaluation. Depending on how you design your evaluation these may include questionnaires, surveys, scripts, etc. This is an opportunity for you to get feedback from your peers and me before your final project.

Similar to your first written plan, a letter grade will not be assigned. For this assignment an assessment of check plus (10 points), check (8 points), or check minus (5 points) will be given to indicate level of progress.

Structured Poster Presentation:

Structured poster presentations will take place the last two days of class. Each group will be given approximately 5-10 minutes to provide the class (and me) with an overview of all aspects of the evaluation. The class will then have time to visit each poster and ask questions about the evaluation. My goal is for you to prove to us that you are a competent evaluator that will use the most up-to-date evaluation techniques and follow standard evaluation guidelines. More information will be provided as the time nears.

3.    Class Attendance: 20% of grade

To get these points you are expected to come to class and participate in the discussion and in-class activities. Course participation is worth 2 points per week, for a total of 30 points, which will be worth 20% of your grade. Things happen, and I don't want you to come to class sick (masking is encouraged as a way of caring for ourselves and one another), so everyone gets 2 free absences. The points for your 2 free absences will be added to your grade at the end of the semester, whether you used them or not. If you miss more than 20% of the semester (approximately 3 weeks) for medical or personal reasons, it may be best to consider withdrawing from all courses until you are able to return and focus on your academic goals.

Attendance is vital, but what does participation mean? This class is heavily interactive and collaborative. The more you engage, the less lecture-y it will feel! To participate meaningfully, you need to keep up with the readings. If it becomes apparent that the majority of the class in not reading the assigned texts, quizzes will be administered and will be graded as an additional component to attendance points.

EXPECTATIONS FOR ASSIGNMENTS

• I expect your writing to be of college-level quality, synthesizing your own ideas with other works you consulted, which at a minimum will include the required course text.
• I expect the papers to use proper and consistent citation (APA, Chicago, or one of the many other standards). Feel free to reach out to me for help on this, and you can also find information on how to cite sources at libraries.indiana.edu/help-citing-sources. .
• I orient to assignments as learning opportunities where we are learning-by-doing and actively grappling with new concepts and content. Accordingly, Generative AI (e.g., Chat GPT, Claude) is not to be used to produce your assignments. AI is embedded in most tools at this point, so what exactly does this mean? I encourage you to use tools that improve accessibility and engagement with the course material (e.g., text-to-speech tools), enable you to learn beyond the required text (e.g., search engines), or proof-read your writing (e.g., spelling and grammar checks). However, I ask you not to use generative AI to construct the ideas and writing for your assignments, not only because AI produces coherent-sounding inaccurate responses, but primarily because AI models are built on unethically sourced intellectual and creative property of actual people who deserve credit. The flip side of this, however, is that you can also expect me to provide you meaningful feedback on your work. I am excited to engage with you on your ideas and help you develop your writing, but I do not want to spend my time providing feedback on Claude's "thinking" and/or composition. See more on this below in Course Policies.
• All assignments should be double spaced, Times New Roman, 12 pt. font with 1 inch margins. 
• Unless otherwise specified, all assignments should be turned in on Canvas, and will be due at 11:59pm on Sunday nights. Please see due dates in Canvas and the semester schedule for details.
• In evaluation timeliness is extremely important. In many circumstances a late evaluation report can result in the termination of funding for your project. Therefore I am fairly strict about late assignments and will only give extensions in extenuating circumstances. Any assignment not turned in on the day it is due will automatically have 20% of total possible points deducted.

If you have questions/concerns about any of these expectations, please talk to me early. I want to ensure all students in my class are supported to meet course expectations. 

 Course Policies

COMMUNICATION POLICY

Please use Canvas messages to send any comments, concerns or questions. Doing this will help me stay organized and assure you get a prompt response. I normally reply to Canvas messages within 24 to 48 hours. I will not reply over the weekend. If you do not hear from me after 48 hours, please feel free to message me again. If your message necessitates lengthy clarification of class readings, assignments or discussions, I will ask that we set up an appointment to talk. 

READINGS

I expect that you do all class readings on the assigned date. On Canvas you will find all required readings for the assigned week. If the citation is only provided I expect you to log on to the library website and find the paper. I will make sure that our library system provides access to all papers assigned.

Remember, part of your class participation grade is being prepared (e.g. reading). Plus, class is simply not as much fun if everyone doesn’t read the required text! So if you don’t want to read for your own learning at least do it for the enjoyment of others in the class.

STUDENTS WITH DISABILITIES

Students with visual, hearing, physical, and/or learning disabilities that may require modification of curriculum, instruction, or assessment should contact me immediately. Modifications and accommodations can be made after the student has presented documentation indicating qualification for services from IU Accessible Educational Services (AES). The AES office is located in Franklin Hall 006, and can be reached at 855-7578, https://studentlife.indiana.edu/care-advocacy/iub-aes/index.html. I understand there are many barriers to formal disability diagnosis and documentation, and hold space for informal accommodations, which can be discussed with me as needed. I cannot guarantee these, but I will do my very best. 

AES accommodations may take time to put into place and are not retroactive. Contact AES at iubaes@iu.edu or (812) 855-7578 as soon as possible if accommodations are needed. The office is located on the third floor, west tower, of the Wells Library, Room W302. Walk-ins are welcome 8AM to 5PM, Monday through Friday. You can also locate a variety of campus resources for students and visitors who need assistance at: https://accessibility.iu.edu/ada/index.html

RELIGIOUS OBSERVANCE

The recognition and celebration of diversity is an important cornerstone for effective educators, and we in Indiana University’s School of Education wish to accommodate the religious practices of students. While our course is online and asynchronous, if religious observance interferes with your ability to meet any course deadlines, I ask you to notify me as soon as possible. The following link provides information on accommodations and procedures to allow students to observe religious holidays: enrollmentbulletin.indiana.edu/pages/relo.php.

ACADEMIC HONESTY

Academic honesty is a fundamental expectation. Dishonesty may result in the failure of this course and may affect your enrollment status in the university. You are expected to abide by the Code of Student Rights, Responsibilities and Conduct. Cheating, plagiarism, sexual harassment, racial/ethnic discrimination and slurs, or any other student misconduct that adversely affects the learning or safety of other students will not be tolerated in this classroom or on this campus. If you become aware of any of these activities, or feel that you have been the victim of sexual harassment, racial/ethnic discrimination, or any other act of malicious intent, please contact the instructor immediately. Additionally, you may contact the Office of Student Conduct or the Dean of Students office. For more information on academic misconduct, go to the Division of Student Affairs website at: studentaffairs.indiana.edu.

Plagiarism is defined for the purposes of this course as submitting work that is not your own. This can include wholesale copying the work of others, presenting someone else’s words or ideas as your own, and improperly citing or failing to cite someone else’s ideas. Plagiarism also includes submitting writing you did for another course, and submitting AI-generated work as your own. Generative AI (e.g., Chat GPT, Claude) is not to be used to construct the ideas and writing for your assignments. This policy does not include AI that is embedded in digital tools that enhance accessibility (e.g., text-to-speech tools), enable broader learning (e.g., search engines), or proof-read your writing (e.g., spelling and grammar checkers). Generative AI being used to produce responses for assignments constitutes plagiarism because these tools are built on unethically sourced intellectual and creative property of actual people who deserve credit. Citing the platform and prompt you used is not the same as citing the people whose work was scraped to build the AI. If I suspect students are using generative AI to produce the content and composition of their assignments, we will move to oral presentations, quizzes, etc. If questionable material is found in your assignments, I reserve the right to require revision, decrease the assignment’s final grade, give you a 0 for the assignment, and/or report the assignment to the department for further disciplinary action. You can find information on how to cite sources at libraries.indiana.edu/help-citing-sources. If you have questions about what counts as plagiarism, contact me before the assignment in question is due.

CHANGES TO THE SYLLABUS

Changes to the syllabus and/or course schedule may take place. Please check Canvas announcements and messages to remain updated on the course. I reserve the right to change or add to course policies as the semester progresses.

TITLE IX AND SEXUAL MISCONDUCT

As your instructors, one of our responsibilities is to help create a safe learning environment on our campus. Title IX and the university’s Sexual Misconduct policy prohibit sexual misconduct. If you have experienced sexual misconduct, or know someone who has, the University can help. You are encouraged to visit stopsexualviolence.iu.edu to learn more. If you are seeking help and would like to speak to someone confidentially, you can make an appointment with a mental health counselor on campus at CAPS or the Center for Human Growth. The 24-hour crisis line for Sexual Assault Counseling Services is (812) 855-5711. Confidential Victim Advocates can provide confidential guidance and can be reached at cva@indiana.edu or by calling (812) 856-2469.

It is also important to know that federal regulations and University policy require instructors to promptly convey any known information about potential sexual misconduct to our Deputy Title IX Coordinator or IU’s Title IX Coordinator. In that event, they will work with a small number of others on campus to ensure that appropriate measures are taken and resources are made available to the student who may have been harmed. Protecting a student’s privacy is of utmost concern, and all involved will only share information with those who need to know to ensure the University can respond and assist.

BIAS-BASED INCIDENT REPORTING

Bias-based incident reports can be made by students, faculty and staff. Any act of discrimination or harassment based on race, ethnicity, religious affiliation, gender, gender identity, sexual orientation or disability can be reported through any of the options: 1) email biasincident@indiana.edu or incident@indiana.edu; 2) call the Dean of Students Office at (812) 855-8188 or 3) use the IU mobile App (m.iu.edu). Reports can be made anonymously.

RESOURCES FOR STUDENTS

Resources for student wellbeing:
College can be hard, and it’s important to look after yourself in all areas of life first in order to make sure you’re able to learn. I encourage students to peruse the resources available from IU for various aspects of wellbeing.

• Free 24/7 mental healthcare through TimelyCare, including unlimited free therapy and self-care resources.
• Free groceries, assistance with emergency funding, and emergency support from Student Care and Resource Center

Digital access and equality at IU:

Digital devices such as laptops and cell phones are becoming increasingly important to success in college, and some students may encounter barriers to accessing these technologies. In this course, you will need digital devices to access course material, complete and submit assignments, take in-class polls, and engage with peers. I recognize that technology access and problems can be a significant source of stress for students. To that end, I encourage students to be aware of the many technology-related resources that Indiana University provides, including:

• Free software (including Microsoft Office, Adobe Creative Suite, statistical software, etc.) for download and for cloud-based use.
• Free, secure, unlimited storage through OneDrive or Google.
• Free 24/7 support with issues related to IU technology (e.g., email, Canvas, wifi, printing, device setup, etc.).
• Free in-person tech support at the Learning Commons in the Wells Library and in IMU room M089 (click here for hours).
• Laptops and tablets that students can borrow from the Learning Commons in the Wells Library (click here for hours). Desktop computers available for use in all campus libraries.
• Discounts on devices from leading technology companies, including Apple, Dell, and Microsoft.
• Free text-to-speech software which will read aloud anything on your screen: TextHelp Read&Write or other accessibility through IUware.

Semester Schedule

1	Aug 25 & 27	Introduction to the course and evaluation	Syllabus; Text Ch. 1	Assigned readings and in-class participation
2	Sept 1 & 3	Ethics & Politics	Text Ch. 3 & 10	Assigned readings and in-class participation
3	Sept 8 & 10	History of Evaluation and Models of evaluation	Text Ch. 2 & 4	Assigned readings and in-class participation
4	Sept 15 & 17	Cultural competence, and when to evaluate Group Work Session 9/17	Text Ch. 9 & 11	Assigned readings and in-class participation Section A & B of written plan due Sept 20th at 11:59pm
5	Sept 22 & 24	Contextual and specific evaluation factors	Text Ch. 12	Assigned readings and in-class participation
6	Sept 29 & Oct 1	Evaluation questions & criteria Guest Speaker: Prof. David Rutkowski	Text Ch. 13	Assigned readings and in-class participation
7	Oct 6 & 8	Planning an evaluation	Text Ch. 14	Assigned readings and in-class participation
8	Oct 13 & 15	Evaluation design Group Work Session 10/15	Text Ch. 15	Assigned readings and in-class participation Section C & D of written plan due Oct 18th at 11:59pm
9	Oct 20 & 22	Data sources and methods Guest Speaker: Moaaz Hamid	Text Ch. 16	Assigned readings and in-class participation
10	Oct 27 & 29	Surveys, questionnaires, interview protocols and focus group guides	Canvas readings	Assigned readings and in-class participation
11	Nov 3 & 5	Data analysis and interpretation Group Work Session 11/5	Text Ch. 16, Canvas readings	Assigned readings and in-class participation; Section E of plan due Sunday, Nov 8th at 11:59pm
12	Nov 10 & 12	Qualitative and quantitative analysis Group Work Session 11/12	Canvas readings	Assigned readings and in-class participation; Alignment document due Sunday, Nov 15th at 11:59pm
13	Nov 17 & 19	Reporting evaluation results	Text Ch. 17	Assigned readings and in-class participation; Section F &G of written plan due Sunday, Nov 22nd at 11:59pm
14	Nov 24 & 26	Thanksgiving break – no class!
15	Dec 1 & 3	In-class group work on final projects	Resources and readings TBA based on class questions and needs.	Assigned readings and in-class participation; poster pdf and sections H & I of plan due Sunday, Dec 6th at 11:59pm.
16	Dec 8 & 10	Structured poster presentations	No readings – posters will be presented in class	Poster presentations to be given in class.
Finals	Dec 15 & 17	No class – final project due Dec 15th at 10:00am.