Reliability of identification of behavior change techniques in intervention descriptions

Charles Abraham; Caroline E. Wood; Marie Johnston; Jill Francis; Wendy Hardeman; Michelle Richardson; Susan Michie

doi:10.1007/s12160-015-9727-y

Reliability of identification of behavior change techniques in intervention descriptions

Charles Abraham, Caroline E. Wood, Marie Johnston, Jill Francis, Wendy Hardeman, Michelle Richardson, Susan Michie

Research output: Contribution to journal › Article › peer-review

48 Citations (Scopus)

20 Downloads (Pure)

Abstract

PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.

METHODS: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.

RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.

CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

Original language	English
Pages (from-to)	885-900
Number of pages	16
Journal	Annals of Behavioral Medicine
Volume	49
Issue number	6
DOIs	https://doi.org/10.1007/s12160-015-9727-y
Publication status	Published - Dec 2015

Access to Document

10.1007/s12160-015-9727-y

Abraham-et-al-Accepted-Version-ABM-July-2015Accepted author manuscript, 944 KB

Cite this

@article{7af9c8ae952e4124ad1278b9f3af4710,

title = "Reliability of identification of behavior change techniques in intervention descriptions",

abstract = "PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.METHODS: Forty trained coders applied the {"}Behavior Change Technique Taxonomy version 1{"} (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.",

author = "Charles Abraham and Wood, {Caroline E.} and Marie Johnston and Jill Francis and Wendy Hardeman and Michelle Richardson and Susan Michie",

year = "2015",

month = dec,

doi = "10.1007/s12160-015-9727-y",

language = "English",

volume = "49",

pages = "885--900",

journal = "Annals of Behavioral Medicine",

issn = "0883-6612",

publisher = "Springer",

number = "6",

}

TY - JOUR

T1 - Reliability of identification of behavior change techniques in intervention descriptions

AU - Abraham, Charles

AU - Wood, Caroline E.

AU - Johnston, Marie

AU - Francis, Jill

AU - Hardeman, Wendy

AU - Richardson, Michelle

AU - Michie, Susan

PY - 2015/12

Y1 - 2015/12

N2 - PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.METHODS: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

AB - PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.METHODS: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

U2 - 10.1007/s12160-015-9727-y

DO - 10.1007/s12160-015-9727-y

M3 - Article

C2 - 26290002

VL - 49

SP - 885

EP - 900

JO - Annals of Behavioral Medicine

JF - Annals of Behavioral Medicine

SN - 0883-6612

IS - 6

ER -