Research Area: Evaluation

MindPlay Impact Evaluation

The purpose of this study is to examine the impact of MindPlay on student reading achievement in Dayton (Ohio) Public Schools. Implemented as a supplemental resource aligned with Dayton’s existing reading curricula and instructional strategies, MindPlay was made available to all students in grades K-10 beginning with the 2018-19 school year, with a particular focus on grades 2 through 6. To evaluate MindPlay’s impact, CRESP analyzed six years of NWEA MAP Reading test scores for more than 15,000 students in grades K-6 using statistical growth curve modeling comparing Dayton students’ annual growth in reading prior to and during the implementation of MindPlay, while also benchmarking their growth against national averages for the same time period.

Results confirmed that growth rates in reading scores of Dayton students during the implementation of MindPlay were significantly higher than the national average (by +0.2 to +0.6 points per year) with even larger increases in reading growth for students who used MindPlay up to 80 or 150 minutes per week. This suggests that implementation of MindPlay may have significantly reduced the COVID slump in Dayton and, instead, allowed many Dayton students to make gains that moved them closer to national average levels of reading achievement.

Read the research report here: CRESP Technical Report T24-003 – MindPlay Evaluation

Dysolve Randomized Field Trial

The purpose of this research study is to examine the efficacy of a game-based learning platform (Dysolve) that is designed to help reduce language processing deficits in children with reading difficulties, including dyslexia. A randomized controlled trial is designed to evaluate Dysolve’s impact through analysis of student’s test scores in reading and English language arts (ELA) from state accountability tests and commercially developed assessments. Students in grades 3-8 who scored below the 30th percentile on the previous year’s state test are eligible to participate in the study. Recruitment for this study continues into the 2024-25 school year, with a goal of recruiting 480 students in total.

Description of Intervention Condition

Students randomly assigned to receive the Dysolve intervention will be asked to use the Dysolve Program on a computer, tablet, or Chromebook for 15-30 min./day in addition to any other educational activities routinely provided by their school. Dysolve is a computer program designed to address dyslexia and language-related disorders. Dysolve uses Artificial Intelligence to generate individually-tailored sequences of game-based training tasks that target each learner’s unique problems. As it probes the root causes of the problems found, Dysolve designs increasingly targeted activities to correct them for that particular learner. Dysolve is offered in the cloud, allowing members to log on at any time, any place through their PCs or mobile devices. Dysolve is a supplemental intervention; children who are selected to use Dysolve will not miss other instructional time.

Description of Control Condition

Students randomly assigned to the control condition will receive “business-as-usual” (BAU) instruction and intervention already provided by their school. Students in the control condition will be able to use the Dysolve program for free after posttest data are collected for this randomized trial.

Clinical Trial Protocol and Pre-Registration

This randomized clinical trial has been pre-registered with the Registry of Efficacy and Effectiveness Studies (REES).

Read the protocol here: https://sreereg.icpsr.umich.edu/sreereg/subEntry/21840/pdf

Evaluation of NSF ICorps Northeast Hub

The NSF ICorps program is designed to foster entrepreneurship that will lead to the commercialization of technology and innovation supported previously by NSF-funded research. It aims to provide researchers with training, mentoring, and funding to help them translate their research into products or services with real-world impact. The Northeast Hub includes 11 partner institutions: Princeton University, University of Delaware, Rutgers University, Lehigh University, Temple University, New Jersey Institute of Technology, Rowan University, Delaware State University, Drexel University, Yale University, and the University of Connecticut.

The Center for Research in Education and Social Policy (CRESP) at the University of Delaware is the Northeast Hub’s Evaluation and Assessment Lead.

Delaware School Climate and Student Success (SCSS) Study

The Delaware School Climate and Student Success (SCSS) project was funded through the U.S. Department of Education’s School Climate Transformation Grant program. SCSS is a five-year grant awarded to the University of Delaware’s Center for Disabilities Studies (CDS) in 2014. The SCSS project was designed to help LEAs create and maintain a positive school climate by building capacity to provide, improve, and expand services that address provision of: (a) professional development (PD) and data integration tools to LEAs overseeing Delaware Positive Behavior Supports (PBS) in schools throughout the state; and (b) additional technical assistance to three priority districts and nine of their schools with the greatest needs (i.e., focus schools).  CDS contracted with evaluators at the Center for Research in Education and Social Policy (CRESP) at the University of Delaware (UD) to conduct an evaluation of the SCSS project.

CRESP Research team: Henry May, Sue Giancola, Tia Barnes, and Dandan Chen

Resources:

Giancola, S., & Riser, D. (Eds.). (December 2019). Delaware School Climate and Student Success (SCSS) Study: Combined Report (R19-004). Newark, DE: Center for Research in Education and Social Policy.

May, H., & Chen, D.  (September 2019).  Delaware School Climate Study: Analysis of Delaware School Climate Scale-Student (T19-016). Newark, DE: Center for Research in Education and Social Policy.

Barnes, T., Giancola, S., & May, H. (September 2019). Delaware School Climate Project: 2019 Case Study Report (T19-012). Newark, DE: Center for Research in Education and Social Policy.

 

Evaluation of the Special Education Administrative Leadership (SEAL) program

 

SEAL is a leadership program for special education administrators in the State of Delaware. CRESP’s evaluation of SEAL utilizes a participatory, theory-based approach. See the following article from the Association of University Centers on Disabilities for an overview of SEAL and the program roll-out. Articles featuring SEAL have also been published in EdWeek  and UDaily. 

Project Start Date: 10/1/20 (officially 2/7/2020)

Publications:

Riser, D., & Giancola, S. (January, 2020). Delaware Special Education Administrative Leadership (SEAL) Program: Evaluation results from the December 2019 Advisory Group meeting (Publication S20-001). Newark, DE: Center for Research in Education and Social Policy.

earcher: Sue Giancola

HHS grant awards $750,000 for ‘We Care’ program in Delaware`: CRESP to partner as program evaluator

The U.S. Department of Health and Human Services (HHS) awarded a $750,000 grant to implement what could very well be a lifesaving service, We Care, through Sussex County’s Meals on Wheels program.

 The grant, awarded to CHEER Senior Services, La Red Health Center and Educational Health Resources International (EHRI), the nonprofit arm of Nationwide Healthcare Services at the Milford Wellness Village, will give the three groups the opportunity to work together to provide wellness checks with medical follow-up to homebound seniors who receive food from Meals on Wheels. Together, they will receive a third of the grant each year for three years to ensure program stability as they begin, officials said.

A cross-disciplinary team from the University of Delaware– Allison Karpyn (CRESP), Julia O’Hanlon (Biden School) and Elizabeth Orsega-Smith (Behavioral Health and Nutrition)–will partner as program evaluators.

An Efficacy Follow-Up Study of the Long-Term Effects of Reading Recovery Under the i3 Scale-Up

The purpose of the Efficacy Follow-up Study of the Long-Term Effects of Reading Recovery is to test the sustained efficacy of Reading Recovery (RR), an intensive one-on-one reading instruction program for the lowest-achieving first grade students, on state test scores in third and fourth grade. Many first grade students struggle with reading and for some of these students, low literacy achievement in first grade can set them up for continued difficulty in literacy throughout elementary school and beyond. The Reading Recovery program is based on the idea that individualized, short-term, and highly-responsive instruction delivered by an expert teacher can disrupt this trajectory and allow the lowest achieving students to catch up to their peers. This study is a follow-up study to an i3-funded scale-up study of the Reading Recovery program.

Intervention:

Reading Recovery is a fully-developed program that uses intensive one-on-one reading instruction for the lowest-achieving first grade students in a school. These students receive 12- to 20-week cycles of daily, 30-minute, one-on-one lessons from specially trained Reading Recovery expert teachers. Lessons target phonemic awareness, phonics, vocabulary, fluency, and comprehension. Reading Recovery teachers receive specialized training that prepares them to tailor lessons to an individual student’s strengths and needs. Each lesson begins with re-reading familiar books, followed by word and letter work, story composition, assembling a cut-up sentence, and previewing and reading a new book. Reading Recovery also relies on continuous collection of data to gauge student progress.

Study Activities:

In this study, the researchers obtained state test scores for students who participated in the original i3 study as well as students from non-i3 schools to see if the impact of Reading Recovery is sustained through third and fourth grades. In addition to collecting students’ third and fourth grade reading or English language arts state test scores, we also administered an online survey through which RR Teacher Leaders or Teachers documented specific details of the experiences of individual Reading Recovery and comparison group students in terms of continued performance monitoring to detect a recurrence of reading problems and participation in supplemental reading programs and interventions. Data for over 9,000 students in more than 700 schools were collected for this study.

Results:

Three working papers (prior to peer-review) are being presented at the Annual Conference of the American Educational Research Association (AERA) on April 23, 2022. Copies of the presentation slides and working papers are available here.

If you have questions about this study, please send an email to the research team at cresp-rr@udel.edu

Publications:

May, H., Sarfo, A., & Englert, A., (in preparation, 2018). Short-Term Impacts of Reading Recovery in First Grade: A Regression Discontinuity Study from the 2011-12 Cohort of Non-i3 Schools.

Gray, Abigail; Goldsworthy, Heather; May, Henry; and Sirinides, Philip. (April, 2017). Evidence for Early Literacy Intervention: The Impacts of Reading Recovery. CPRE Policy Briefs. Philadelphia: Consortium for Policy Research in Education.

May, H., Sirinides, P., Gray, A., & Goldsworthy, H. (March, 2016). Reading Recovery: An Evaluation of the Four-Year i3 Scale-Up. Philadelphia: Consortium for Policy Research in Education.

May, H., Gray, A., Sirinides, P., Goldsworthy, H., Armijo, M., Sam, C., Gillespie, J., & Tognatta, N. (June, 2015). Year One Results from the Multi-Site Randomized Evaluation of the i3 Scale-Up of Reading Recovery. American Educational Research Journal 52(3), 547 – 581.

May, H., Goldsworthy, H., Armijo, M., Gray, A., Sirinides, P., Blalock, T., Anderson-Clark, H., Schiera, A., Blackman, H., Gillespie, J., & Sam, C. (December, 2014). Evaluation of the i3 Scale-Up of Reading Recovery: Year Two Report, 2012-13. Philadelphia: Consortium for Policy Research in Education.

May, H., Gray, A., Gillespie, J., Sirinides, P., Sam, C., Goldsworthy, H., Armijo, M., & Tognatta, N. (August, 2013). Evaluation of the i3 Scale-Up of Reading Recovery: Year One Report, 2011-12. Philadelphia: Consortium for Policy Research in Education.