Pat Cochrane and Vicky Storey, evaluation support consultants for rounds 3–6 of the Teacher Development Fund, reflect on the differing approaches to evaluation across TDF projects and share some of the ways that colleagues have enabled genuine exploration as an integral part of programme design.
Reflective practice and evaluation are built into every Teacher Development Fund (TDF) project and we’ve been fortunate to be able to work with projects supporting their evaluation for the past five years. It’s been an exciting journey.
The term ‘evaluation’ can conjure up a rather dry process of data collection and report writing and the need for an objective outsider to observe and measure change. In our work with TDF projects, we have found that artists, teachers and senior leaders often feel the pressure for evaluation to be about assessment, monitoring and ‘proving’ success and impact on pupil attainment.
But Paul Hamlyn Foundation’s (PHF) Teacher Development Fund (TDF) supports a different kind of approach to evaluation. TDF is unique in that it gives time and permission to dig deep into project design to reach understandings of the approaches, processes and experiences that support teachers in embedding arts-based learning into their practice. In our evaluation support role, we aim to help projects to see reflection and evaluation as a rich, integral part of their project, rather than a dry monitoring or data capture job. By careful and systematic capture, curation and analysis of evidence, each project can develop new understanding about their approaches.
This approach to qualitative, formative, dialogic evaluation is hard, serious work that takes time and commitment from project teams. Capturing the authentic voice and experience of participants to enable teams to draw conclusions about the ingredients of effective CPDL requires consistency and investment of time and energy in analysis.
We want to share three ways in which the projects we have supported have ensured genuine exploration and authentic learning is driving their projects.
Although each project team will have put in much thought when preparing their application for funding, most benefit from investing more time at the beginning of their project to reflect on all the elements of the CPDL design. The excitement of getting a grant from PHF can mean that you want to get going and launch into activity straight away, but we recommend pausing and creating space for teams to question and clarify project aims and plans. What assumptions are you making about how your planned activities will support teacher development? What is it you want to find out about your art form and teaching and learning in schools?
Pausing and digging a little deeper is particularly important if you’ve got a complex project partnership or if partners have not been involved from the outset. Working through a Theory of Change process with a range of stakeholders from your project — ie teachers, senior leaders, arts leads can be incredibly helpful. It is from this starting point that projects can develop relevant and interesting enquiry questions that will ensure the opportunity for genuine learning is set up.
Over five years, we’ve seen a large range of approaches to evaluation in the projects we’ve supported. Some projects lead their own internally — either led by the arts organisation or school leaders; a few have benefited from the expertise of recently retired school leaders; others commission an evaluator or mentor, and others have an evaluation partner in the project from the outset.
We’ve seen almost every combination possible. Our main learning is that although it’s important to have clarity about who is leading the evaluation, the most interesting and valuable insights come from a process of collaboration in which the different participant and partner voices are captured and valued through a regular reflection cycle.
It’s not about delegating the process to one person who writes a report at the end, but rather a formative process with insights informing the project as it develops. And such collaboration depends on mutual trust which develops over time.
These articles from the team at Trestle Theatre and University of Hertfordshire and from YDance explore how this collaboration has influenced their project. The Trestle Theatre and the University of Hertfordshire team explain how working together on a pilot scheme gave them the foundation for the collaborative enquiry in their TDF project. Linzi’s article from YDance shows how a close collaborative relationship between the arts organisation YDance and evaluation partner Robert Owen Centre has been woven into the project design and many projects have had similar experiences.
Projects want to capture the authentic voice of teachers; how they feel when they try a new approach — to explore hopes and fears and what has helped them to change their practice and start integrating the arts into their teaching. The big challenge project teams raise with us is how can we get teachers to reflect when they have so little time and the idea of reflecting on their own learning is often such an unfamiliar experience
There’s no magic solution. Keeping logs, diaries and completing feedback forms has limited success. So projects have been exploring ways of capturing feedback directly after a session which provides useful data for analysis later.
Gemma from Grimm & Co describes how setting aside time for a reflective game after sessions generates valuable and open reflection and Anni Raw describes how requesting teachers to respond to prompt questions on WhatsApp in real time immediately after sessions captures insights that can be collated and analysed.
Nearly all projects have a number of development days when the full team of teachers and artists come together. We encourage projects to set aside some of this time to jointly reflect on and critique their learning so that their professional judgement is valued and becomes a part of the data to inform the evaluation.
And finally, evaluation is not all questionnaires and tick boxes. All TDF projects focus on one or more art forms and have skilled arts practitioners in their teams. Don’t be afraid of using the art form at the heart of your project to capture responses creatively. You can use images to prompt teachers to make observations about their learning, or encourage teachers to model or draw how they feel; use drama or dance to capture reflections on increasing confidence.
Collaborative evaluation approaches were key to the success of the YDance programme, Step It Up. Linzi McLagan, Head of Education for YDance, discusses the importance of their collaboration with the Robert Owen Centre (ROC) at the University of Glasgow and shares some of the approaches used within their programme.
Evaluation consultant, Dr Anni Raw, describes the evaluation approach she is co-developing with project partners on the Parallel Rhythms programme in Huddersfield.
