Executive Summary

Peter Shulman

Please tell us a little about yourself?

I was the leader of the Teacher Leader Effectiveness Unit (TLEU) for the Delaware Department of Education. In this role, I worked to implement a diversity of human capital initiatives as outlined in both Delaware's award-winning Race to the Top Application and their state education reform plan.

What are three things that are needed for teacher evaluation reform to work really well?

1. Determine the proper level of engagement of stakeholders: Stakeholders include: teachers, teacher’s union, parents, 
We kept everyone at the table. Even though we have had bumps along the road, we have kept everyone at the table and the reforms are being done in full transparency.
students, state education association (SEA) people and local education association (LEA) people. Yet, there is a balance between having everyone at the table and the pace of the work… You can spend a lot of time at the theoretical level, but at some point the work needs to be done.

2. Set defined parameters: Help educators and other team members to develop or procure assessments and know what you are looking for and what constitutes a valid measure.

3. Provide support along the way: Having some level of financial support is not out of bounds. Bring in a third party. Someone who has a level of expertise in assessment development would be a sound decision.

I would add that, at a more practical level, States/LEAs need to (a) be aggressive in taking the first step in areas where they are ready to (e.g. Teacher Practice and tested grades/subjects) and (b) ensure that that evaluators are properly trained and calibrated in a perpetual process.

What worked well for Delaware? What are you most proud of?

  • We realized that we had to recalibrate our work. There has to be honesty in this work. Honesty to do a mid-course correction. We were able to do a mid-course correction.
  • Communication. The element of how teacher evaluation reform that will affect people employments, means we have to do a good job of messaging, because people could be anxious about the evaluation process. For example, all statistical models are imperfect to a certain extent. How do you convey the level of imperfection into a teacher evaluation mode? We have done a good job on this. We could do better going forward.
  • We kept everyone at the table. Even though we have had bumps along the road, we have kept everyone at the table and the reforms are being done in full transparency.

What didn’t work well?

I don’t think we set defined parameters for what constitutes an assessment. For example, a psychometrician might say that for every question on an assessment there needs to be a bank of ten questions. Also, every assessment should have both multiple choice and free responses. In take two, we partnered with an assessment organization that has psychometric background and detailed with the parameters should be and communicated those parameters to educators. Ideally, we would have done that in take one.

What outside help would you have gotten from the outset?

Expertise in areas that we did not have expertise in. Also, just pure capacity at all levels. In Delaware, we have worked with McKinsey, Education Delivery Institute and others.  One thing Delaware did particularly well was integrate these outside agencies into our work and make it clear to stakeholders throughout the state that these consultants represent the DOE’s efforts.

What big questions remained unanswered at this stage?

What will the non-tested grades and subjects assessment look like? Will it be a homegrown assessment or an off the shelf assessment?...After you disaggregate the data, how do you set the bar at the right level?

What is the most promising work in non-tested grades and subjects?

We have trenched the work into different cohorts. Something we can zero in on more quickly can be zeroed in on. Also, not every assessment will look the same, whether they be SLO’s or standardized assessments. The same way we distinguish among educators. We may have to distinguish among assessments. We have a lot of inputs going well. We can’t talk to outputs, as it is too early.

What are the most important milestones you have reached?

  • Legislation. The legislation has put us on the right foundation for this work to happen.
  • Systems. For the better part of two decades we have had the data systems to link students and teachers. We are doing work in our longitudinal data system to make the data more accessible. We have a lot of the levers in place. A lot of other LEAs and SEAs do not have the levers in place, so even if they were given perfect assessments that were statistically valid, they do not have the levers in place to make this happen.

What are the next milestones you are looking for?

●      Deliverables from each of the three cohorts that are working on the assessments. The first one is in December.

●      Our technical advisory group, after it goes through the standard channels. A combination of outside experts: people from NCTQ, business community and others who can look at the work from an output level.



The Federal Government and Teacher Evaluation

E-news sign-up