Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates

Tengbin Xiong; Sheetal Parekh-Bhurke; Yoon K. Loke; Asmaa Abdelhamid; Alex J. Sutton; Alison J. Eastwood; Richard Holland; Yen-Fu Chen; Tanya Walsh; Anne-Marie Glenny; Fujian Song

doi:10.1016/j.jclinepi.2012.06.022

Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates

Tengbin Xiong, Sheetal Parekh-Bhurke, Yoon K. Loke, Asmaa Abdelhamid, Alex J. Sutton, Alison J. Eastwood, Richard Holland, Yen-Fu Chen, Tanya Walsh, Anne-Marie Glenny, Fujian Song^*

^*Corresponding author for this work

Centre for Reviews and Dissemination

Research output: Contribution to journal › Literature review › peer-review

Abstract

Objectives: Indirect comparison methods have been increasingly used to assess the effectiveness of different interventions comparatively. This study evaluated a Trial Similarity and Evidence Consistency Assessment (TSECA) framework for assessing key assumptions underlying the validity of indirect comparisons.

Study Design and Setting: We applied the TSECA framework to 94 Cochrane Systematic Reviews that provided data to compare two interventions by both direct and indirect comparisons. Using the TSECA framework, two reviewers independently assessed and scored trial similarity and evidence consistency. A detailed case study provided further insight into the usefulness and limitations of the framework proposed.

Results: Trial similarity and evidence consistency scores obtained using the assessment framework were not associated with statistically significant inconsistency between direct and indirect estimates. The case study illustrated that the assessment framework could be used to identify potentially important differences in participants, interventions, and outcome measures between different sets of trials in the indirect comparison.

Conclusion: Although the overall trial similarity and evidence consistency scores are unlikely to be sufficiently accurate for predicting inconsistency between direct and indirect estimates, the assessment framework proposed in this study can be a useful tool for identifying between-trial differences that may threaten the validity of indirect treatment comparisons. (C) 2013 Elsevier Inc. All rights reserved.

Original language	English
Pages (from-to)	184-191
Number of pages	8
Journal	Journal of Clinical Epidemiology
Volume	66
Issue number	2
DOIs	https://doi.org/10.1016/j.jclinepi.2012.06.022
Publication status	Published - Feb 2013

Keywords

Indirect treatment comparison
Mixed treatment comparison
Multiple treatment meta-analysis
BIAS
Network meta-analysis
CONTROLLED TRIALS
Evidence consistency
Trial similarity
MIXED TREATMENT COMPARISONS
VALIDITY
NETWORK METAANALYSIS
INCONSISTENCY
EFFICACY
COMPETING INTERVENTIONS

Access to Document

10.1016/j.jclinepi.2012.06.022

Cite this

Xiong, T., Parekh-Bhurke, S., Loke, Y. K., Abdelhamid, A., Sutton, A. J., Eastwood, A. J., Holland, R., Chen, Y.-F., Walsh, T., Glenny, A.-M., & Song, F. (2013). Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates. Journal of Clinical Epidemiology, 66(2), 184-191. https://doi.org/10.1016/j.jclinepi.2012.06.022

@article{93c90892473547f69567ecbf0efc6a52,

title = "Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates",

abstract = "Objectives: Indirect comparison methods have been increasingly used to assess the effectiveness of different interventions comparatively. This study evaluated a Trial Similarity and Evidence Consistency Assessment (TSECA) framework for assessing key assumptions underlying the validity of indirect comparisons.Study Design and Setting: We applied the TSECA framework to 94 Cochrane Systematic Reviews that provided data to compare two interventions by both direct and indirect comparisons. Using the TSECA framework, two reviewers independently assessed and scored trial similarity and evidence consistency. A detailed case study provided further insight into the usefulness and limitations of the framework proposed.Results: Trial similarity and evidence consistency scores obtained using the assessment framework were not associated with statistically significant inconsistency between direct and indirect estimates. The case study illustrated that the assessment framework could be used to identify potentially important differences in participants, interventions, and outcome measures between different sets of trials in the indirect comparison.Conclusion: Although the overall trial similarity and evidence consistency scores are unlikely to be sufficiently accurate for predicting inconsistency between direct and indirect estimates, the assessment framework proposed in this study can be a useful tool for identifying between-trial differences that may threaten the validity of indirect treatment comparisons. (C) 2013 Elsevier Inc. All rights reserved.",

keywords = "Indirect treatment comparison, Mixed treatment comparison, Multiple treatment meta-analysis, BIAS, Network meta-analysis, CONTROLLED TRIALS, Evidence consistency, Trial similarity, MIXED TREATMENT COMPARISONS, VALIDITY, NETWORK METAANALYSIS, INCONSISTENCY, EFFICACY, COMPETING INTERVENTIONS",

author = "Tengbin Xiong and Sheetal Parekh-Bhurke and Loke, {Yoon K.} and Asmaa Abdelhamid and Sutton, {Alex J.} and Eastwood, {Alison J.} and Richard Holland and Yen-Fu Chen and Tanya Walsh and Anne-Marie Glenny and Fujian Song",

year = "2013",

month = feb,

doi = "10.1016/j.jclinepi.2012.06.022",

language = "English",

volume = "66",

pages = "184--191",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier",

number = "2",

}

Xiong, T, Parekh-Bhurke, S, Loke, YK, Abdelhamid, A, Sutton, AJ, Eastwood, AJ, Holland, R, Chen, Y-F, Walsh, T, Glenny, A-M & Song, F 2013, 'Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates', Journal of Clinical Epidemiology, vol. 66, no. 2, pp. 184-191. https://doi.org/10.1016/j.jclinepi.2012.06.022

Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates. / Xiong, Tengbin; Parekh-Bhurke, Sheetal; Loke, Yoon K. et al.
In: Journal of Clinical Epidemiology, Vol. 66, No. 2, 02.2013, p. 184-191.

Research output: Contribution to journal › Literature review › peer-review

TY - JOUR

T1 - Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates

AU - Xiong, Tengbin

AU - Parekh-Bhurke, Sheetal

AU - Loke, Yoon K.

AU - Abdelhamid, Asmaa

AU - Sutton, Alex J.

AU - Eastwood, Alison J.

AU - Holland, Richard

AU - Chen, Yen-Fu

AU - Walsh, Tanya

AU - Glenny, Anne-Marie

AU - Song, Fujian

PY - 2013/2

Y1 - 2013/2

N2 - Objectives: Indirect comparison methods have been increasingly used to assess the effectiveness of different interventions comparatively. This study evaluated a Trial Similarity and Evidence Consistency Assessment (TSECA) framework for assessing key assumptions underlying the validity of indirect comparisons.Study Design and Setting: We applied the TSECA framework to 94 Cochrane Systematic Reviews that provided data to compare two interventions by both direct and indirect comparisons. Using the TSECA framework, two reviewers independently assessed and scored trial similarity and evidence consistency. A detailed case study provided further insight into the usefulness and limitations of the framework proposed.Results: Trial similarity and evidence consistency scores obtained using the assessment framework were not associated with statistically significant inconsistency between direct and indirect estimates. The case study illustrated that the assessment framework could be used to identify potentially important differences in participants, interventions, and outcome measures between different sets of trials in the indirect comparison.Conclusion: Although the overall trial similarity and evidence consistency scores are unlikely to be sufficiently accurate for predicting inconsistency between direct and indirect estimates, the assessment framework proposed in this study can be a useful tool for identifying between-trial differences that may threaten the validity of indirect treatment comparisons. (C) 2013 Elsevier Inc. All rights reserved.

AB - Objectives: Indirect comparison methods have been increasingly used to assess the effectiveness of different interventions comparatively. This study evaluated a Trial Similarity and Evidence Consistency Assessment (TSECA) framework for assessing key assumptions underlying the validity of indirect comparisons.Study Design and Setting: We applied the TSECA framework to 94 Cochrane Systematic Reviews that provided data to compare two interventions by both direct and indirect comparisons. Using the TSECA framework, two reviewers independently assessed and scored trial similarity and evidence consistency. A detailed case study provided further insight into the usefulness and limitations of the framework proposed.Results: Trial similarity and evidence consistency scores obtained using the assessment framework were not associated with statistically significant inconsistency between direct and indirect estimates. The case study illustrated that the assessment framework could be used to identify potentially important differences in participants, interventions, and outcome measures between different sets of trials in the indirect comparison.Conclusion: Although the overall trial similarity and evidence consistency scores are unlikely to be sufficiently accurate for predicting inconsistency between direct and indirect estimates, the assessment framework proposed in this study can be a useful tool for identifying between-trial differences that may threaten the validity of indirect treatment comparisons. (C) 2013 Elsevier Inc. All rights reserved.

KW - Indirect treatment comparison

KW - Mixed treatment comparison

KW - Multiple treatment meta-analysis

KW - BIAS

KW - Network meta-analysis

KW - CONTROLLED TRIALS

KW - Evidence consistency

KW - Trial similarity

KW - MIXED TREATMENT COMPARISONS

KW - VALIDITY

KW - NETWORK METAANALYSIS

KW - INCONSISTENCY

KW - EFFICACY

KW - COMPETING INTERVENTIONS

UR - http://www.scopus.com/inward/record.url?scp=84871333665&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2012.06.022

DO - 10.1016/j.jclinepi.2012.06.022

M3 - Literature review

SN - 0895-4356

VL - 66

SP - 184

EP - 191

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

IS - 2

ER -

Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates

Abstract

Keywords

Access to Document

Other files and links

Cite this