Course Syllabus

O’Neill School of Public and Environmental Affairs

SPEA V379

Performance Measurement & Program Evaluation

Download:Syllabus_V379_Tong_FALL_2026.pdf

 

Instructor/Contact Information

Instructor: Junying Tong

Contact Information: junytong@iu.edu

 

Office Location: SPEA 334

Class Meeting: Monday and Wednesday 12:45 PM – 2:00 PM, Woodburn Hall (WH) 005

Office Hours: Monday and Wednesday, SPEA 334, 3-4:00 PM. Alternative office hours can be set up via email as needed.

Schedule here: https://calendly.com/junytong-iu/30min

 

Teaching Assistant/Contact Information

Teaching Assistant: Giheon Song

Teaching Assistant Email Address: gihesong@iu.edu

Teaching Assistant Office Hours: Tuesday and Thursday 11:00 AM – 12:00 PM, SPEA A218

 

Email Instruction: Please put the course number (V379) at the beginning of the subject line of all course-related emails. When sending an email to ask questions about course materials, get clarification about assignment instructions, etc., please include both me and the teaching assistant as recipients to ensure a timely response. If your question requires a lengthy/complicated response to address via email, we may instead ask you to come to office hours or arrange an appointment to discuss it.

 

Please don’t hesitate to contact me with any questions, concerns, or feedback about the course. You are welcome to email me at any time, but please expect longer response times outside of regular work hours.

 

Course Description

This course provides an overview of program evaluation as it relates to public affairs, criminal justice, health policy, and environmental science with particular emphasis on measuring program outcomes. The course is designed for students who envision themselves working in management, policy-making, or research roles.

 

This course provides an entry level introduction to various research designs and econometric methods that quantitative social scientists use to estimate causal effects of public programs. Specifically, we will discuss randomized experiments, instrumental variables, difference in differences, regression discontinuity etc.

 

Learning Outcomes

By the end of the course, you should be able to:

• Explain and critically analyze alternative measures of outcomes and performance.
• Formulate evaluation questions in the realm of public programs
• Explain the logic and key assumptions of several research designs.
• Communicate the results of simple data analysis in written documents, conversations, and other media.
• Clean and organize data to create tables and figure and analyze the results using the statistical program R
• Read, interpret, critique, and explain results from published articles and reports concerned with the evaluation of public programs.

 

O’Neill School expectations of civility and professional conduct[1]

The O’Neill School takes matters of honesty and integrity seriously because O’Neill is the training ground for future leaders in government, civic organizations, health organizations, and other institutions charged with providing resources for the public, and for members of society who are vulnerable and who are lacking in power and status. Precisely because O’Neill graduates tend to rise to positions of power and responsibility, it is critical that the lessons of honesty and integrity are learned early.

O’Neill requires that all members of its community – students, faculty, and staff – treat others with an attitude of mutual respect both in the classroom and during all academic and nonacademic activities outside the classroom. A student is expected to show respect through behavior that promotes conditions in which all students can learn without interruption or distraction. These behaviors foster an appropriate atmosphere inside and outside the classroom:

• Students are expected to attend class regularly and to be prepared for class.
• Students must be punctual in their arrival to class and be present and attentive for the duration of the class. Eating, sleeping, doing work for another class, wandering in and out of the classroom, and packing up or leaving class early are not civil or professional behaviors.
• Students must abide by the course policy regarding use of electronic devices in the classroom.
• Students must responsibly participate in class activities and during team meetings.
• Students must address faculty members, other students, and others appropriately and with respect, whether in person, in writing, or in electronic communications.
• Students must show tolerance and respect for others.
• Students must not destroy or deface classroom property nor leave litter in the classroom.

 

Course Texts

1. Angrist, Joshua D., and Jörn-Steffen Pischke. Mastering 'Metrics: The path from cause to effect. Princeton University Press, 2014. (AP)

 

2. Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2016. Impact Evaluation in Practice, Second edition. Washington, DC: Inter-American Development Bank and World Bank. (WB)

Link: https://www.worldbank.org/en/programs/sief-trust-fund/publication/impact-evaluation-in-practice (online available for free)

 

Additional required readings will be posted on Canvas.

 

I strongly recommend that you use the following for help with R:

Healy, Kieran. Data Visualization: A practical introduction. Princeton University Press, 2019.

Link: https://socviz.co (online available for free)

 

Additional free online resources for learning R:

• An Introduction to R on GitHub: https://rstudio.github.io/r-manuals/r-intro/
• Elegant graphics for data analysis using “ggplot2”: https://ggplot2-book.org/
• Introduction to data.table: https://cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.html
• Also, data.table: https://rafalab.dfci.harvard.edu/dsbook/data.html

(*You may also want to check out the author --- Hadley Wickham’s personal website, which contains a wealth of resources for learning R: https://hadley.nz/)

 

Computing & Course Software

We will often spend class time analyzing data. That means you should bring a laptop computer with you when required.

The course will make use of a statistical software package called R and a development environment called R studio. You need to install both of these pieces of software on your computer. Both of these packages are free. Instructions are below:

Install R

1. Open a browser and go to: https://cloud.r-project.org/
2. Click the download link for whatever operating system you use on your computer. (There is a separate link for the macOS if you use an Apple compute, for the Windows OS if you have a Microsoft Windows based computer, and for Linux if you are running Linux).
3. Once the file downloads, double click to open it and follow the installation instructions.

 

Install R Studio

1. Open a browser and go to https://www.rstudio.com/products/rstudio/download/
2. Click the download button for the RStudio desktop. (Choose the version recommended for your computer based on the OS.)
3. Once the file downloads, double click it and follow the instructions.

 

Course Grades and Requirements

Problem Sets (45% of Course Grade)

There will be 4 data analysis assignments throughout the course of the semester. These assignments will ask you to use R to clean, organize, and analyze a specific data set. We will work together on the data assignments in class. This will help you learn how to work in R, which may be new to many students. However, we will not usually complete all of the analysis in class. You'll still have to do some of the work on your own time. For each assignment, you will turn in BOTH a short report and a replication kit consisting of the R code you used to perform the analysis. The assignments will be made available on Monday and will be due on the following Sunday at 11:59 PM. Extensions will only be granted if requested prior to the Friday at 5pm before the Sunday due date, except in extreme circumstances. You may collaborate with up to two other classmates. If you work with others, please list the names of all collaborators on your assignment. However, each student must write and submit their own assignment independently. Your final answers should reflect your own understanding, reasoning, and effort.

 

Midterm Exam (20% of Course Grade)

There will be one midterm exam in this course. It will cover all material from the beginning of class until the date of the exam. All exams must be submitted by the due date and time.

 

Research Design Assignment (25% of Course Grade)

At the end of the course, students will submit a research report based on a previously approved evaluation question. The report should clearly state the main hypothesis, describe and justify the research design, identify and use the appropriate data, present the empirical analysis, and discuss the main findings and conclusions. Additional guidance and detailed requirements for the research report will be provided throughout the semester.

 

Short Assignments (10% of Course Grade)

Over the course of the semester, there will be a number of short quizzes or discussion questions in Canvas or in class about the reading assignments or class materials. These will be due before class or during class. There is no makeup for these assignments and late submission is not accepted. The two lowest grades will be dropped.

 

Note: There are no opportunities for extra credit in this course to ensure fairness.

 

Course Policies

1. Attendance

I do not take attendance. However, students must be aware that I do not record lectures and will not make notes available outside of class. There are no makeups for the short assignments, which will be given in class.

 

2. Make-up Exams

There will be no make-up exams. If you are unable to take the midterm exam due to an emergency or other extenuating circumstances, please contact me as soon as possible to discuss your situation.

 

3. Late Assignments

Extensions can be granted in extreme circumstances if requested at least 2 days prior to the due date. Late assignments will be accepted with a 10% deduction for each day that an assignment is late. This only applies to the four Problem Sets. Late in-class and after-class quizzes (the "short assignments") are not accepted.

 

4. Technology Use

Because we will be using R for our data assignments and will be working on assignments together in class, please bring your laptops to class. You are allowed to use laptops for class-related matters, such as taking notes. However, use of cell phones is not allowed except during designated times or breaks. You may leave your phone turned on in your pocket or backpack so that you can still receive emergency alerts. If you wish to take a photo of a lecture slide for study purposes, please raise your hand first and ask for permission. Other than that, if you really need to use your phone during class, please step outside first. The basic reasoning for this policy is that having phones out (which inevitably leads to some people checking email, using social media, etc. even if they have the best intentions of staying focused on class) creates a distraction not only for those using them but also those around them.

 

5. Late Withdrawal

NOTE: The auto-W deadline is NO LONGER THE LAST DAY OF CLASS (as it was during and post-Covid).  After the auto-W deadline, withdrawal will be significantly limited and you will need permission to withdraw and must meet requirements established by O’Neill.

 

6. Academic Integrity

You are expected to adhere to the Indiana University guidelines regarding academic standards and behavior. I expect that all work you turn in will be your own. If you synthesize or rely on other people’s ideas or words to make your argument, you must cite them. Please take a moment to review your responsibilities here: https://studentcode.iu.edu/.

 

7. Use of AI (such as ChatGPT and other LLMs)

Using AI to assist in completing assignments will be allowed only in the following ways:

• For data assignments using R, you may use ChatGPT or Microsoft 365 Copilot only to assist with writing, debugging, or troubleshooting R code. You are responsible for understanding the code you submit. All code comments and explanations should be written in your own words and should demonstrate your understanding of the analysis.
• If you use an AI tool in any permitted way on an assignment, you MUST include a brief statement describing how you used it (i.e., in which part, for what purpose).
• All other uses of generative AI for completing course assignments are prohibited. This includes using AI to generate, rewrite, or substantially edit answers, interpretations, explanations, or other written content.
• You may use the free version of Grammarly and Microsoft Word’s built-in spelling and grammar tools for basic proofreading and grammatical corrections.
• If you use AI in an unauthorized manner, you will be committing plagiarism* and will be subject to penalties in this class and sanctions by Indiana University. 

 

WARNING: Submitting any questions, assignments, notes, slides, or other instructor-produced course materials into Generative AI without permission is a violation of the faculty member’s exclusive rights and will be reported to the Dean of Students as academic misconduct.

 

*Plagiarism: Plagiarism is defined as presenting someone else’s work, including the work of other students, as one’s own. Any ideas or materials taken from another source for either written or oral use must be fully acknowledged, unless the information is common knowledge. https://studentcode.iu.edu/responsibilities/academic-misconduct.html. (emphasis added)

8. Course Materials—Instructor’s Exclusive Rights.

The faculty member teaching this course holds the exclusive right to distribute, modify, post, and reproduce course materials, including all written materials, study guides, lectures, assignments, exercises, and exams. While you are permitted to take notes on the online materials and lectures posted for this course for your personal use, you are not permitted to re-post in another forum, upload to AI, distribute, reproduce content from this course, or sell the instructor’s course materials (including notes and study guides) without the express written permission of the faculty member.

 

Violations of this policy will be reported to the Dean of Students as academic misconduct (violation of course rules). Sanctions for academic misconduct may include a failing grade on the assignment for which the notes/study guides are being sold, a reduction in your final course grade, or a failing grade in the course, among other possibilities.

 

Additionally, you should know that selling a faculty member’s course using IU email, or via Canvas may also constitute a violation of IU information technology and IU intellectual property policies; additional consequences may result.

 

Schedule of Data Assignments, Midterm, and Research Design Project

 

• Data Assignment 1 due September 28
• Data Assignment 2 due October 12
• Midterm Exam on October 14
• Data Assignment 3 due November 9
• Data Assignment 4 due November 23
• Research Design Project due December 7 at the beginning of class

 

*****************************************************************************

“The above schedule is subject to limited change in the event of extenuating circumstances.”

 

Schedule of Class Topics

Week	Class	Date	Topic	Reading
Week1	1	August 24	Introduction to the course, syllabus overview	None
	2	August 26	History of public programs and evaluation	Read: “What are the major federal safety net programs in the U.S.?” https://poverty.ucdavis.edu/article/war-poverty-and-todays-safety-net-0   Skim “The Social Insurance System in the U.S.: Policies to Protect Workers and Families”  https://www.brookings.edu/wp-content/uploads/2021/06/Social-Insurance-FP_v4.5.pdf
Week2	3	August 31	What is program evaluation?	WB Ch. 1
	4	September 2	Introduction to measurement and variables and identifying program outcomes	WB Ch. 2
Week3		September 7	No class, Labor Day	No readings
	5	September 9	Program theory, identifying evaluation questions	Readings will be posted on Canvas
Week4	6	September 14	Getting Started with R and R Studio I *Bring Laptops*	Recommended: Healy Ch. 1-2; https://rstudio.github.io/r-manuals/r-intro/Simple-manipulations-numbers-and-vectors.html
	7	September 16	Getting Started with R and R Studio II *Bring Laptops*	Recommended: Healy Ch. 1-2; https://rstudio.github.io/r-manuals/r-intro/Simple-manipulations-numbers-and-vectors.html
Week5	8	September 21	Working with data on Income and Poverty	Institute for Research on Poverty at the University of Wisconsin. Poverty FAQs. How Is Poverty Measured? https://www.irp.wisc.edu/resources/how-is-poverty-measured/ Who Is Poor? https://www.irp.wisc.edu/resources/who-is-poor/ What are Poverty Thresholds and Poverty Guidelines? https://www.irp.wisc.edu/resources/what-are-poverty-thresholds-and-poverty-guidelines/ What is the Consumer Price Index? https://www.irp.wisc.edu/resources/what-is-the-consumer-price-index-and-how-is-it-used/
	9	September 23	Statistics Review	Optional reading will be posted to Canvas
Week6	10	September 28	Introduction to Causal Inference and Potential Outcomes	AP, Introduction WB, Ch. 3
	11	September 30	Randomized Experiments	WB, Ch. 4 AP, Ch. 1
Week7	12	October 5	Examples of Randomized Experiments in Public Affairs I	Readings will be posted on Canvas
	13	October 7	Examples of Randomized Experiments in Public Affairs II	Readings will be posted on Canvas
Week8	14	October 12	Midterm review	No readings
	15	October 14	Midterm	No readings
Week9	16	October 19	Regression	AP, Ch. 2
	17	October 21	Regression	AP, Ch. 2
Week10	18	October 26	R Session	Readings will be posted on Canvas
	19	October 28	Difference-in-Differences	WB, Ch. 7 AP, Ch. 5
Week11	20	November 2	Regression, Balance, and Natural Experiments and Examples of Difference-in-Differences in Public Affairs I	Readings will be posted on Canvas
	21	November 4	No class, APPAM travel	Readings will be posted on Canvas
Week12	22	November 9	Regression Discontinuity Design I	WB, Ch. 6 AP, Ch. 4
	23	November 11	Regression Discontinuity Design II	WB, Ch. 6 AP, Ch. 4
Week13	24	November 16	Examples of RDD in Public Affairs I	Readings will be posted on Canvas
	25	November 12	Instrumental Variable I	Readings will be posted on Canvas
Week14		November 23	No class, Thanksgiving Recess	No readings
		November 25	No class, Thanksgiving Recess	No readings
Week15	26	November 30	Instrumental Variable II
	27	December 2	Ethics and Science of Impact Evaluation	WB, Ch. 13
Week16	28	December 7	Review Research Design Assignments	Presentations in class
	29	December 9	Review Research Design Assignments	Presentations in class
Week17		December 15-19	Final Exam Week	There is no final exam in this course.

 

Diversity statement

In my classroom, I value diversity as an essential component of education and training. I am committed to fostering a learning environment that is inclusive, respectful, and equitable for all students, regardless of their backgrounds or experiences.

 

As a class, we recognize that diversity encompasses many dimensions, including but not limited to race, ethnicity, gender, sexuality, age, religion, socio-economic status, ability, and cultural background. We believe that these differences can enrich our learning experience and prepare us to be effective leaders in diverse and multicultural communities.

 

The SPEA V-379 course curriculum is designed to reflect this commitment to diversity, with readings, discussions, and assignments that explore a range of perspectives and experiences. I encourage my students to critically engage with these diverse viewpoints, to challenge their own assumptions and biases, and to actively seek out and learn from voices that differ from their own.

 

As part of our commitment to diversity, I also recognize the importance of creating a safe and respectful learning environment that promotes open communication and dialogue. I encourage all students to share their thoughts and experiences, and as a class, we pledge to listen and respect each other's views, even if they differ from our own.

 

Additional information

Counseling and Psychological Services

For information about services offered to students by CAPS:  http://healthcenter.indiana.edu/counseling/index.shtml

 

TimelyCare

Indiana students have free, 24/7 access to virtual mental health care services with TimelyCare. Students do not need insurance to access TimelyCare services. https://www.iu.edu/mental-health/find-resources/timely-care.html#0

 

Religious Accommodation Requests

The "Request for Accommodation for Religious Observances" form must be submitted by the student to the professor no later than two weeks prior to the anticipated absence.  A separate form must be submitted for each absence and for each course. Faculty do not have to consider accommodations for the purpose of allowing students to travel away from Bloomington for religious observance.  Information about the policy is found here: https://vpfaa.indiana.edu/resources/religious-observances-information.html.

 

Accessible Educational Services (formerly Disability Services for Students) 

Securing accommodations for a student with disabilities is a responsibility shared by the student, the instructor, and the AES Office. For information about support services or accommodations available to students with disabilities, and for the procedures to be followed by students and instructors: https://studentlife.indiana.edu/student-support/iub-aes/index.html

 

Sexual Harassment 

As your instructor, one of my responsibilities is to help create a safe learning environment on our campus. Title IX and our own Sexual Misconduct policy prohibit sexual misconduct.  If you have experienced sexual misconduct, or know someone who has, the University can help.

 

If you are seeking help and would like to talk to someone confidentially, you can make an appointment with:

1. The Sexual Assault Crisis Service (SACS) at 812-855-8900
2. Counseling and Psychological Services (CAPS) at 812-855-5711
3. Confidential Victim Advocates (CVA) at 812-856-2469
4. IU Health Center at 812-855-4011

 

For more information about available resources:  http://stopsexualviolence.iu.edu/help/index.html. It is also important to know that federal regulations and University policy require me to promptly convey any information about potential sexual misconduct known to me to our campus’ Deputy Title IX Coordinator or IU’s Title IX Coordinator.  In that event, they will work with a small number of others on campus to ensure that appropriate measures are taken and resources are made available to the student who may have been harmed.  Protecting a student’s privacy is of utmost concern, and all involved will only share information with those that need to know to ensure the University can respond and assist.  I encourage you to visit http://stopsexualviolence.iu.edu/help/index.html to learn more.

 

IU Culture Centers: https://studentlife.indiana.edu/communities/culture-centers/index.html

 

[1] These expectations are excerpted from the O’Neill School Honor Code which can be found under Student Ethics and Policies: https://oneill.indiana.edu/undergraduate/course-advising/advising/resources.html