Expert elicitation of multinomial probabilities for decision-analytic modeling: An application to rates of disease progression in undiagnosed and untreated melanoma

Background: Expert elicitation is required to inform decision making when relevant “better quality” data either do not exist or cannot be collected. An example of this is to inform decisions as to whether to screen for melanoma. A key input is the counterfactual, in this case the natural history of melanoma in patients who are undiagnosed and hence untreated. 

Objectives: To elicit expert opinion on the probability of disease progression in patients with melanoma that is undetected and hence untreated. 

Methods: A bespoke webinar-based expert elicitation protocol was administered to 14 participants in the United Kingdom, Australia, and New Zealand, comprising 12 multinomial questions on the probability of progression from one disease stage to another in the absence of treatment. A modified Connor-Mosimann distribution was fitted to individual responses to each question. Individual responses were pooled using a Monte-Carlo simulation approach. Participants were asked to provide feedback on the process. 

Results: A pooled modified Connor-Mosimann distribution was successfully derived from participants’ responses. Feedback from participants was generally positive, with 86% willing to take part in such an exercise again. Nevertheless, only 57% of participants felt that this was a valid approach to determine the risk of disease progression. Qualitative feedback reflected some understanding of the need to rely on expert elicitation in the absence of “hard” data. 

Conclusions: We successfully elicited and pooled the beliefs of experts in melanoma regarding the probability of disease progression in a format suitable for inclusion in a decision-analytic model.