Evaluation approaches and principles

Over the last three years, we have had the privilege of undertaking the evaluation of the PHF Teacher Development Fund project ​‘Active learning in and through the arts’. Led by Sidegate Primary School and the Active Learning Trust, this project sought to explore the impact of the arts as a teaching mechanism for embedding learning across the curriculum with a specific focus on vocabulary acquisition. Working closely alongside artists from Dance East and the New Wolsey Theatre, a core group of teachers developed their use of the arts as part of their wider teaching toolkit, subsequently supporting other teachers in their schools to embed these approaches into their practice.

Our experience of evaluating a wide range of projects in the education sector allowed us to approach this task with a clear understanding of existing evaluation mechanisms within schools. While the remit of the project had been established, there was a degree of agility within this to reflect the commitment of participants to innovation and evolution over time. This meant establishing an evaluation framework that would allow the project space to grow, while ensuring that there were a clear set of metrics against which to judge the impact of the project. In the age of evidence-led education, it has never been more important to commit to the production and interrogation of evidence sources that could benefit others across the sector and, for the project to retain integrity, this balance was essential.

Designing the architecture of a robust evaluation is a complex task. In most cases, schools lean heavily on quantitative evidence sources to help them identify a clear pathway forwards and, while there was some quantitative evidence in the project, by far the most useful information was collected qualitatively. We used models such as well-trialled structured inquiry interview techniques, fixed choice Likert scale surveys, structured reflection journals and group reflection interviews to glean a rich and diverse body of perspectives about how the project had impacted on pupils, artists, teachers and leaders.

This required close thematic analysis against the core and refined evaluation questions and a strong evaluation team who were willing to challenge and reshape interpretations of competing evidence sources.

Using case studies of the impact on individual pupils and teachers allowed a depth perspective that tracked changes over time. The nature of the project required a long-term commitment from schools and artists, which in turn meant that the evaluation structure had to reflect the need for time to engender cultural and pedagogical shift. And, of course, the project was subject to the interruption of the Covid pandemic, which meant that the second part of the evaluation had to be significantly remodelled.

One of the major findings of the project was the need to crystallise and embed the core aims. Without this, different participants sought to measure the success of the project against their own metrics and, while this is both natural and essential in terms of personal learning, the project aims form the basis of the evaluation questions. For those seeking to undertake robust and meaningful evaluation, it is these questions and the underpinning framework and methodology that delivers the necessary level of integrity in the outcomes.