*2.4.4 Designing the infrastructure*

The experiences from this project revealed the influence of organisational factors and infrastructure on the uptake of practice changes. Although research on the integration of feedback systems and ROM have identified numerous practical barriers, much of the emphasis has focused on practitioners [9, 15, 31–37]. Indeed, positive attitudes towards feedback have been shown to facilitate the effect on clinical outcomes improvement, while resistance can have the opposite effect [33, 38–40]. Resistance reportedly stems underlying performance anxiety or negativity about the relevance and utility of the practice [9, 15]. However, the learning from this project highlights how positivity and motivation might not be sufficient in isolation.

Despite the generally positive attitudes from the survey and among the management mentees, itself likely a result of the selection process, many TSOs still encountered challenges, many of which appeared to be due to limitations in the infrastructure and frustrations with the technology. This, in turn, affected their capacity to use the system, something which is shown to be a facilitator in implementing EBP [25, 27, 31]. Restrictive and frustrated working practices can lead to negative perceptions forming [25, 27, 36, 41], suggesting attitudes might be mediated by how user-friendly and engaging a system is. For TSOs facing time and resource constraints, the simplicity of a feedback system is perhaps more pivotal. In these circumstances, systems may benefit from a uniform, standardised approach so that training and support can be refined and accessible via fully integrated and self-led instructional packages [32]. In terms of the QIF [8], the critical steps for assessing the needs and resources, capacity, and pre-implementation training would benefit from accessible resources which are intuitive and easy to understand.

### *2.4.5 Refocusing measurement to respond and maximise the value to clients*

Traditionally, measurement in TSOs have been undertaken to satisfy the needs of boards and funders and to a lesser extent, service managers [3, 4]. The pressures on services have meant that pre and post-measurement approaches have dominated, with its purpose serving mainly administrative rather than clinical needs [3, 9]. ROM established a method for improving data quality and representativeness,

### *Evaluating the Efficiency of a Collaborative Learning Network in Supporting Third Sector… DOI: http://dx.doi.org/10.5772/intechopen.84294*

although the emphasis regarding its clinical utility or use in service development has only recently been advanced [7]. This illustrates how the focus and value of measurement have been positioned to satisfy a broader sector-level drive. However, by framing measurement in a way to maximise the value for clients, as observed in this project, there appear to be many cumulative gains for all stakeholders, including practitioners, service managers and boards/funders.

Across each of the common challenges, there seemed to be a critical period, usually within the first four to six sessions, which correlated with eventual outcome. For instance, a large proportion of DNAs tended to occur early in treatment which were a useful indicator of an unplanned ending, and by extension, a reduced chance of reliable improvement [20]. For clients reporting reliable improvement in one TSO, most change seemed to occur during the first four sessions, while those reporting no reliable change or reliable deterioration showed little change across a 10-session period. This emulates the wider literature which identifies the initial stages as being a useful indicator for a client's subsequent engagement and outcome [42–45]. Accordingly, this trend highlights the criticality of early engagement and warrants a further discussion about the implications of keeping clients involved in therapy who report no change or attend infrequently. Evidence has shown that decisions to prolong or conclude therapy despite a lack of positive therapeutic change can be influenced by subjective beliefs, norms and attitudes, sometimes superseding what feedback monitoring and practice guidelines recommend [45].

According to the literature, the clinical benefit of measurement can be mediated by a practitioner's engagement and attitude towards outcomes monitoring [33, 38, 39]. Moreover, timely access to feedback has been shown to be a critical factor in the use of data among practitioners [27, 34, 36, 46]. TSOs which encourage open dialogue and pay greater attention to this information could produce cumulative benefits in each of the quarterly themes identified [10, 30, 47]. An organisational culture of openness and commitment to learning was important and replicates findings reported elsewhere [15, 46]. Additionally, giving practitioners access to service-level data might assist them in overcoming residual ambivalence because its application to service quality development is readily observable.
