Evaluating community participation policy and practice: Combining Theories of Change and Realist Evaluation methodologies to cope with complexity

Citation

Rolfe S (2016) Evaluating community participation policy and practice: Combining Theories of Change and Realist Evaluation methodologies to cope with complexity. UK Evaluation Society Annual Evaluation Conference 2016, London, 27.04.2016-28.04.2016. http://www.profbriefings.co.uk/ukes2016/ukes2016programme.html

Abstract
This paper presents the methodological findings from an evaluation of community participation policy and practice in Scotland and England, which combined Theories of Change (ToC) and Realist Evaluation (RE) approaches. Community participation is inherently complex for a number of reasons, including emergent outcome targets, the adaptive nature of the organisations and processes, and the openness of the systems involved. In addition, the evaluation process itself has to be both collaborative and adaptive, in order to gain the consent and stimulate the interest of community activists. Theory-based evaluation approaches such as Theories of Change (Connell and Kubisch, 1998) and Realist Evaluation (Pawson and Tilley, 1997) have been presented as a response to complexity, particularly in situations where experimental methods are a non-starter. However, they are not without their critics, since ToC models run the risk of being too general and poorly specified to provide useful evaluation frameworks, whilst RE approaches can become too obsessed with the detail of particular mechanisms, losing the broader picture. This study builds on the suggestion from Blamey and Mackenzie (2007) that the two methodologies could be fruitfully combined. The paper summarises the approach taken to evaluating community participation policy and practice, utilising ToC methodology to provide a theoretical framework and initial policy analysis, before moving on to combine both ToC and RE approaches in mixed method case study work with a range of community organisations. The study suggests that, although combining the two approaches is practically challenging, together they can provide a more robust theoretical framework for evaluating individual projects and for developing generalisable evidence about what works in community participation.