Course Syllabus

 Course Details

Instructor: Shellye Suttles, PhD

Teaching Assistant: Dennis Vera

For office hours and contact information, please see the How to Contact Me page.

Class Meeting: Asynchronous online

 

 Course Description

Examination of how the programs of public agencies are proposed, established, operated, and evaluated. Discussion of the role and conduct of research in the program evaluation process. In addition, techniques of effective evaluation and analysis are discussed.

 

 Learning Outcomes 

By the end of the semester, students should be able to:

• Define and describe methods to develop policy metrics.
• Understand the application and mechanics of randomized trials in program evaluation.
• Understand the appropriate application of quasi-experimental quantitative evaluation methods across public programs and policy areas.
• Apply quantitative program evaluation methods in R and RStudio.
• Analyze quantitative program evaluation data in R and RStudio.
• Evaluate the merits and limitations of existing quasi-experimental analysis.

 

 Course Requirements 

Course Format

Asynchronous online course.

 

Grading Scale

 

Term Project

Students will complete a term project. The term project will require a synopsis of one side of the argument on whether a public program is deemed successful. The public program, however, must be approved by the instructor. Students will submit a term project report, including a series of annotated bibliographies and a quasi-experimental analysis on the subject, at the end of the term. Further guidelines will be distributed in class.

 

Midterm Exam

A midterm exam will be administered via Canvas. 

 

Quantitative Exercises

Five assignments with quantitative exercises based on textbook material will be submitted. The exercises will be completed with R and RStudio and submitted as Word documents generated by R Markdown.

 

Canvas Discussions

Participation in asynchronous online Canvas discussions is essential for your understanding of the material and to perform satisfactorily on the quantitative exercises, the midterm exam, and the term project. The quality and quantity of your online Canvas discussion comments are important and indicate to the grader whether you have completed the assigned readings. 

 

 Required Texts

Textbook

Angrist, J. D., & Pischke, J. S. (2014). Mastering 'metrics: The path from cause to effect. Princeton University Press. Princeton, NJ, USA.

• Available in paperback, Kindle, or hardcover
• Supplemental videos: Mastering Econometrics with Joshua Angrist (MIT)
• Supplemental textbook: Causal Inference: The Mixtape by Scott Cunningham

Additional weekly readings for discussions will be assigned and posted on Canvas at least one week in advance. All course readings are required to be successful in our online Canvas discussions unless explicitly marked as optional. The main concepts from the assigned readings will be discussed in the recorded lectures and incorporated into the midterm exam and term project.

 

Useful Resources

• Arnold, J.B. (nd). “Mastering Metrics R Code.” https://jrnold.github.io/masteringmetrics/
• R Studio Cheat Sheets
  • https://www.rstudio.com/resources/cheatsheets/
    • R Studio IDE
    • R Markdown
    • Data Visualization with ggplot2
• Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton University Press.
• Langbein, L. (2014). Public program evaluation: A statistical guide. Routledge.
• Getler et al. (2016). Impact Evaluation in Practice. World Bank, Washington, DC, USA. https://openknowledge.worldbank.org/server/api/core/bitstreams/4659ef23-61ff-5df7-9b4e-89fda12b074d/content
• U.S. federal program evaluation: https://www.evaluation.gov/
  • Centers for Disease Control and Prevention: https://www.cdc.gov/evaluation/guide/introduction/index.htm
  • National Institute of Justice: https://nij.ojp.gov/topics/articles/plan-program-evaluation-start
  • National Science Foundation: https://www.informalscience.org/sites/default/files/TheUserFriendlyGuide.pdf

 

 Course Policies

Makeup Exam

A makeup midterm exam will be allowed in emergency situations with a reduction of 5 points for each day the midterm is postponed.

 

Late Assignments

Late assignments will be accepted as follow:

• Quantitative Exercises, a reduction of 1 point for each day the assignment is late.
• Canvas Discussions, a one-page annotated bibliography for each assigned article missed for discussion. Please contact the instructor for the required annotated bibliography format.

 

Attendance

The asynchronous online course is based on recorded lectures and Canvas discussions. Full performance, particularly discussion rubric creation and grading, requires regular online engagement.

 

Artificial Intelligence (AI)

This course is heavily reliant on existing literature that is generally not accessible with current AI technology (e.g., ChatGPT). In most cases, it is a serious copyright violation to share the copyrighted article with the AI technology to generate a work product. Additionally, using text generated by AI technology without proper citation is a violation of Indiana University’s Student Code of Conduct (see ‘Academic Dishonesty’ below).

Nevertheless, you are permitted to use AI technology to assist with your coding in statistical software as we analyze quantitative data using quasi-experimental methods. Using AI technology to assist in completing assignments will be allowed in the following ways:

• Quantitative Exercises (1-5), can use AI technology as a part of completing the assignment.

If you use AI technology in an unauthorized manner, you will be committing plagiarism and will be subject to penalties in this class and sanctions at Indiana University. 

 

Late Withdrawal

The automatic withdrawal (W) deadline is no longer the last day of class as it was during and post-COVID. The last day to drop the course with an automatic W is Monday, May 18.

After the automatic W deadline, withdrawal will be significantly limited and you will need permission to withdraw and must meet requirements established by the O’Neill School.

 

Internet Etiquette

One thing to always keep in mind when taking any course is that the others with whom you interact throughout the semester - including your instructor - are human beings. The first rule of netiquette is to "remember the human" when you are communicating with me or with your peers. The second rule is to "adhere to the same standards of behavior online that you follow in real life." 

 

 Course Communication Policy

Canvas message is my preferred means of contact. I will reply to your message within 48 business hours. In case of emergency only, you may email me (please see the How to Contact Me page). Please be clear in the subject line and put 'SPEA-P 562’ in all direct e-mails.

 

 Assignments and Grades

Course Assessment

The instructor will assess your grasp of the learning outcomes through Canvas discussions, quantitative exercises, a midterm exam, and a term project that includes an examination of various program evaluation results.

 

Submitting Assignments

Students are expected to submit their work via Canvas as Word documents. Quantitative Exercises should be Word documents generated by R Markdown.

 

How Will I Know How I’m Doing in This Course?

Grades will be posted in Canvas within five (5) business days of the due date. Since Canvas keeps track of all your grades, you should always be able to calculate your current grade in the course. If you need assistance, please contact the instructor.

 

 Assignment Return and Feedback Policy

 As there is no teaching assistant for the course, grades will be posted in Canvas within five (5) business days of the due date. Under extenuating circumstances, grade posting may be delayed.

 

 Schedule of Assignments

 

*****************************************************************************

The above schedule is subject to limited change in the event of extenuating circumstances.

******************************************************************************

O'Neill School Expectations of Civility and Professional Conduct

The O’Neill School takes matters of honesty and integrity seriously because O’Neill is the training ground for future leaders in government, civic organizations, health organizations, and other institutions charged with providing resources for the public, and for members of society who are vulnerable and who are lacking in power and status. Precisely because O’Neill graduates tend to rise to positions of power and responsibility, it is critical that the lessons of honesty and integrity are learned early.

O’Neill requires that all members of its community – students, faculty, and staff – treat others with an attitude of mutual respect both in the classroom and during all academic and nonacademic activities outside the classroom. A student is expected to show respect through behavior that promotes conditions in which all students can learn without interruption or distraction. These behaviors foster an appropriate atmosphere inside and outside the classroom:

• Students are expected to attend class regularly and to be prepared for class.
• Students must be punctual in their arrival to class and be present and attentive for the duration of the class. Eating, sleeping, reading the newspaper, doing work for another class, wandering in and out of the classroom, and packing up or leaving class early are not civil or professional behaviors.
• Students must abide by the course policy regarding use of electronic devices in the classroom.
• Students must responsibly participate in class activities and during team meetings.
• Students must address faculty members, other students, and others appropriately and with respect, whether in person, in writing, or in electronic communications.
• Students must show tolerance and respect for diverse nationalities, religions, races, sexual orientations, and physical abilities.
• Students must not destroy or deface classroom property nor leave litter in the classroom.

** These expectations are excerpted from the O’Neill School Honor Code.

 (Links to an external site.).

 Academic Integrity

Online Course Materials

The faculty member teaching this course holds the exclusive right to distribute, modify, post, and reproduce course materials, including all written materials, study guides, lectures, assignments, exercises, and exams. While you are permitted to take notes on the online materials and lectures posted for this course for your personal use, you are not permitted to re-post in another forum, distribute, or reproduce content from this course without the express written permission of the faculty member. Any violation of this course rule will be reported to the appropriate university offices and officials, including to the Dean of Students as academic misconduct.

Note Selling

Several commercial services have approached students regarding selling class notes/study guides to their classmates. Selling the instructor’s notes/study guides in this course is not permitted. Violations of this policy will be reported to the Dean of Students as academic misconduct (violation of course rules). Sanctions for academic misconduct may include a failing grade on the assignment for which the notes/study guides are being sold, a reduction in your final course grade, or a failing grade in the course, among other possibilities. Additionally, you should know that selling a faculty member’s notes/study guides individually or on behalf of one of these services using IU email, or via Canvas may also constitute a violation of IU information technology and IU intellectual property policies; additional consequences may result.

 

 Technical Support

For additional help with technical issues, consult:

• University Information Technology Services (UITS)  (Links to an external site.) (human support)
• IU Knowledge Base (IUKB)  (Links to an external site.) (guides)
• IUware  (Links to an external site.) (download free software)

 

 Additional Information

1. TimelyCare

   Indiana students have free, 24/7 access to virtual mental health care services with TimelyCare. Students do not need insurance to access TimelyCare services. https://www.iu.edu/mental-health/find-resources/timely-care.html#0

2. Accessible Education Service (AES)
   • Securing accommodations for a student with disabilities is a responsibility shared by the student, the instructor and the DSS Office. For information about support services or accommodations available to students with disabilities, and for the procedures to be followed by students and instructors visit the AES webpage.  (Links to an external site.)
3. Religious Observation
   • In accordance with the Office of the Dean of Faculties, any student who wishes to receive an excused absence from class must submit a Request for Accommodation for Religious Observances  (Links to an external site.) for each day to be absent. This form must be presented to the course instructor by the end of the second week of this semester.  A separate form must be submitted for each day. The instructor will fill in the bottom section of the form and then return the original to the student.  Information about the policy on religious observation can be found on the website for the Office of the Vice Provost for Faculty & Academic Affairs.  (Links to an external site.)
4. Sexual Harassment
   • As your instructor, one of my responsibilities is to help create a safe learning environment on our campus. Title IX and our own Sexual Misconduct policy prohibit sexual misconduct.  If you have experienced sexual misconduct, or know someone who has, the University can help.
     If you are seeking help and would like to talk to someone confidentially, you can make an appointment with:
     • The Sexual Assault Crisis Service (SACS) at 812-855-8900
     • Counseling and Psychological Services (CAPS) at 812-855-5711
     • Confidential Victim Advocates (CVA) at 812-856-2469
     • IU Health Center at 812-855-4011
   • For more information about available resources visit the Stop Sexual Violence webpage

• • •  (Links to an external site.)It is also important to know that federal regulations and University policy require me to promptly convey any information about potential sexual misconduct known to me to our campus’ Deputy Title IX Coordinator or IU’s Title IX Coordinator. In that event, they will work with a small number of others on campus to ensure that appropriate measures are taken and resources are made available to the student who may have been harmed. Protecting a student’s privacy is of utmost concern, and all involved will only share information with those that need to know to ensure the University can respond and assist. I encourage you to visit the  Stop Sexual Violence webpage to learn more.

 

Additional information and resources, including mental health and financial aid, can be found in the O'Neill Student Portal