In this blog Dr Ruth Abrams and Professor Jill Maben share their experience of using open science to showcase the conceptualisation of an initial programme theory relating to a realist evaluation of a dementia care coordinator service in Kent and Medway.
The development of an initial programme theory (IPT) is essential to realist reviews and evaluations. It helps to surface taken for granted assumptions about a particular service or intervention, as well as consolidate early ideas and identify potentially useful concepts for later down the line. It can be diagrammatic, and/or consolidated as a series of ‘if/then/because’ statements. Yet rarely does it become an output in its own right, often existing in draft form for the project team’s eyes only.
As a live, working document or string of ideas, an IPT morphs and changes, becoming more refined or refuted as the realist process progresses. It may therefore feel vulnerable, or exposing to show an early IPT, especially when presented independently from the ‘real’ findings and final programme theory. It can also raise questions about when to draw a line under an initial programme theory and how much to subsequently share. It therefore may feel much safer to present a more sanitised or modified version at the point of publication. However, presenting an IPT before the point of final publication may also provide an opportunity for enhanced transparency, rigour and trustworthiness. It may help to chart the emergence of novel components, as well as those that were ‘right’ from the start as a way of exploring and demonstrating the evolution of ideas.
In an attempt to be open researchers, and embody these three pillars of research, as part of our realist evaluation, we have produced a working paper showing the starting point of our project. To do this we have used two platforms. Firstly, using the Open Science Framework (OSF) we created a storage space unique to our evaluation. This platform provides a space to consolidate an entire project. The link to our full project is here, and we are also considering uploading our interview topic guides onto this platform to link them to our final write up.
In addition to setting up this public project space, we uploaded our working paper about our IPT onto SocArXiv, which is a paper server specific to social sciences. The link to our IPT paper is here. This platform moderates submissions but does not peer review them. This may be an important consideration to some, but for us the development of our IPT was a highly collaborative process that had multiple points of input and reflection. Putting this through a peer review system felt like it may have defeated the point of sharing the document in this way as a communication experiment. We have also used this platform to house an interim report, and then linked both of these papers back to our OSF project. When we come to write our final report and paper, we will be in a position to link to these documents that are both time stamped and show engagement through views and downloads, rather than have them as supplementarily material post project or languishing in an email inbox.
It is likely that, when we make another update to our IPT we will upload the next version here too. For example, we are currently mapping a series of if/ then/ because statements related to the health economics component of our evaluation. This again, we hope will provide further transparency, showing how we have moved forward. Sitting with the discomfort of doing this has been an interesting point of reflection but it has equally felt liberating. Having our time-stamped IPT in the public domain means it is there for all to see including the parts we may get ‘wrong’. This however is part of the research process and an endeavour we feel moves away from the black box, into the technicolour rainbow of messy reality.
Please cite this blog as: Abrams, R and Maben, J. (2023) From black box to technicolour rainbow: realist (initial) programme theory development, Realist Health & Social Care SIG, Dec. Available at: https://rabrams0.wixsite.com/realistworkforce-sig/post/from-black-box-to-technicolour-rainbow-realist-initial-programme-theory-development
Comments