Reflections and next steps

The classification of the outcomes of PHF-funded work and the ‘impact map’ will inform PHF’s thinking about policy and planning for the future, particularly when the Foundation begins to consider the directions it wishes to take during the period of its next strategic plan. The results prompt thinking about further lines of enquiry and suggest questions for discussion, at both strategic and operational levels. They have identified that we need to act on evidence quality and utility. They also provide a baseline, against which we can examine future trends and changing patterns.

Strategic aims

At the strategic level, the results reveal the ways in which funded work is contributing to the strategic aims that the Foundation set itself for 2006–13. They suggest a number of questions for consideration as we move forward from here, including:

• Are there gaps in the framework? What desirable or intended outcomes are missing or are less numerous than we would like? If there are such gaps in the map of actual outcomes, how does our funding strategy need to change?
• Are some of our intended or actual outcomes more important than others? Which might be future priorities and how do we fund to achieve them?
• Are the outcomes for organisations the ones we want to see? Are there other outcomes that we should seek to encourage? Do different types of organisations have different needs?
• How important is it for the Foundation to try to bring about change by influencing wider practice and policy? What can we learn from where this has happened successfully? What could the Foundation to do to be more effective in this area and enable grantees to have more influence?
• Are there particularly effective approaches, in different contexts, to linking organisational development and/or influence on wider practice and policy to greater benefit for individuals and communities? If so, how can we fund to optimise this?

Working on the evidence

Our findings about the quality and utility of grantees’ evidence underline the importance of finding ways to make evaluation in our sectors more effective. PHF will continue to work with others by participating in the different networks and initiatives that are concerned with improving evidence and impact reporting.

In 2013 our second round of grantee perception research will provide further feedback to PHF about grantees’ experiences of working with us and insight into where we might usefully change the focus of our resources or inputs. It is clear that some grantees, though not all, many need encouragement or support to raise the overall standards of evidence they have about their own work. In addition, PHF will be particularly interested in exploring with grantees ways to follow outcomes over longer periods of time.

The Foundation has recently introduced a ‘relationship agreement’ to make clear how we hope to work with grantees – what we can offer to grantees and what we expect. We have also reviewed and re-issued our reporting guidelines for grantees. As we investigate and consult further, it may be that we conclude that a more tailored approach would be appropriate, with some organisations requiring more or  different types of support from PHF and different types of activity requiring more or less depth of reporting.

The classification of the outcomes of PHF-funded work and the ‘impact map’ will inform PHF’s thinking about policy and planning for the future, particularly when the Foundation begins to consider the directions it wishes to take during the period of its next strategic plan. The results prompt thinking about further lines of enquiry and suggest questions for discussion, at both strategic and operational levels. They have identified that we need to act on evidence quality and utility. They also provide a baseline, against which we can examine future trends and changing patterns.

Our findings about the quality and utility of grantees’ evidence underline the importance of finding ways to make evaluation in our sectors more effective. PHF will continue to work with others by participating in the different networks and initiatives that are concerned with improving evidence and impact reporting.

In 2013 our second round of grantee perception research will provide further feedback to PHF about grantees’ experiences of working with us and insight into where we might usefully change the focus of our resources or inputs. It is clear that some grantees, though not all, many need encouragement or support to raise the overall standards of evidence they have about their own work. In addition, PHF will be particularly interested in exploring with grantees ways to follow outcomes over longer periods of time.

The Foundation has recently introduced a ‘relationship agreement’ to make clear how we hope to work with grantees – what we can offer to grantees and what we expect. We have also reviewed and re-issued our reporting guidelines for grantees. As we investigate and consult further, it may be that we conclude that a more tailored approach would be appropriate, with some organisations requiring more or different types of support from PHF and different types of activity requiring more or less depth of reporting.

Our approach to evaluating the Special Initiatives continues to develop as, collectively across the different initiatives, we learn and understand more about how to carry out evaluation that is helpful in shaping and steering the work in progress and about how
to capture and use evidence of outcomes.

Future use of the framework

We will use the framework to track the outcomes of PHF-funded work, at least for the remainder of the current strategic planning period. Beyond that, our approach to assessing impact will be developed, as necessary, to support the aims of the next strategic plan. This will itself be informed by the impact map and results reported here, which also provide a baseline against which to examine any trends and changes. We hope the framework will not be seen by potential grantees as PHF’s shopping list of outcomes or as a menu from which applicants should select objectives to propose to us. It is not intended to be a set of criteria for trustees’ decisions about whether or not to approve applications for funding. We remain open to all proposals for new activities that meet our aims and funding guidelines, which have not changed. Some new grants may achieve outcomes that fall outside the current framework, which has been built on evidence of outcomes to date. As and when we identify new outcome categories from the evidence of change reported to us, we will amend the framework to include them.

Making further use of the data

In this report we have presented the results at the upper level of the framework, the 14 main outcomes, and for PHF funding as a whole. But it is also possible to produce further types of analysis, to inform our thinking and next steps in other ways, which include:

• Producing separate impact maps for the work funded by each of the three UK programmes (arts, education and learning, social justice), to inform work and planning by programme committees and staff
• Producing a more detailed impact map, at the lower (sub-outcome) level of the framework, to reveal how many Open Grants and Special Initiatives are contributing to each of the 37 more specific types of change
• Bringing the numbers to life by describing and analysing the different approaches taken to achieving each outcome or sub-outcome, linking this to illustrative vignettes or case studies
• Facilitating information exchange and learning between grantees and Special Initiatives within each outcome or sub-outcome group.

This report has also given an overview of our findings about the evidence. We can use the data to guide our next steps in helping grantees to generate more useful data by:

• Examining any differences in evidence quality and utility between different outcomes or sub-outcomes and/or between different types of grantee organisation or sector. This can inform our decisions about where to focus efforts and resources in development work with grantees.