**5.3 Key learnings, challenges, and limitations**

By using a structured and previously validated process for developing a decision support tool [33], we were able to work with key stakeholders from several provinces in China during a 1-day workshop to apply the tool to a decision problem. The resulting MCDA spreadsheet model can be used by UEBMI insurance policy-makers in the provinces of China to compare new options of insurance policies.

A few key components in the process are essential for successful implementation. The most important is the engagement of the stakeholders impacted by the decision. Through the engagement of all stakeholders, a variety of factors can be considered and an open exchange is possible through a workshop format to incorporate all perspectives [40]. In addition, stakeholders, who are part of the "inner circle," or the guiding coalition, can further to convey the momentum for change. This becomes of utmost importance once full implementation is realized [37, 40, 41]. In our example, the decision to limit the stakeholders to insurance experts and academic health policy leaders was purposeful to reflect the current decision-makers. However, a broader range of stakeholders may be involved in the testing phase of individuals' provinces or countries.

Another key success factor is to clearly define the decision problem at the outset of the stakeholder collaboration. If there is a general agreement on the objective of the new policy, and on the criteria, which help to measure whether the objectives can be achieved, the MCDA tool offers an opportunity for increasing transparency of the decision process. Increased transparency may also contribute to a higher trust in the policies by the public in China [42].

The participants in the workshop cautioned that it may be necessary to adapt such a model in their respective provinces to address provincial health-care priorities, which may differ from province to province. A practical implication was use of the domains and criteria as a checklist for the design of new insurance policy options.

Finally, for the long-term success of any future policy change, measurement of outcomes is important. The criteria used in the selection process could become key performance indicators around population health and economics as used in other evaluations [2, 5, 43].

Some limitations or pitfalls of using MCDA as a decision support tool need to be considered. First, there is a risk of bias in the MCDA when the criteria overlap and therefore overemphasize certain aspects of the evaluation. In our example, the participants felt this to be the case with the two finance-related criteria which were originally proposed. They, therefore, decided to merge the two criteria into one. Second, the ranking and weighing methodologies can have a high impact on the "resolution" capability of the decision support tool. In this chapter, we have used a modified SMART approach to MCDA which has been recognized broadly as practicable [23, 39]. It should also be emphasized that decision-makers may be tempted to use the summary score as "the decision argument." Instead, decisionmakers should deliberate and compare the entire rating profile to confirm, that the instrument and the summary score have validity in the specific comparison. This will allow a more differentiated argumentation and documentation why a specific decision has been taken and which preventive measures should be taken to overcome the areas of weaknesses of the chosen option, which are those criteria, where it scored low.

There are also a few limitations to our proposed process. In our workshop, only a limited number of stakeholders were represented. Future programs should aim to achieve a good balance between all relevant stakeholders to maintain an active dialog during the workshop and manage the number of participants to avoid limitation of discussion. Preparation through stakeholder research, including mapping and assessment of appointments, is important to strike this balance.

Another limitation was the need to work with translators during the entire process, which was due to the international composition of the project management and moderating team. To minimize the risk for misunderstanding, the workshop was simultaneously translated and all materials, including the model, were available in both English and Chinese language.
