This article is mentioned in:

Introduction

Globally, lung cancer has been revealed to be the most common cancer, the leading cause of cancer mortality in males, and the second leading cause of cancer mortality in females during the year 2008 (1). Although substantial progress in surgical, chemotherapeutic and radiotherapeutic approaches has been made, the long-term survival rate in patients with lung cancer still remains low (2). A key determinant of the appropriate therapy provided to patients with lung cancer depends on the accurate detection of lymph node metastasis. Therefore, the correct assessment of the lymph node status is crucial for the prognosis of patients with lung cancer.

Non-invasive diagnostic imaging modalities, i.e., computed tomography (CT) and magnetic resonance imaging (MRI), are commonly used for the identification of metastatic lymph nodes in patients with lung cancer. However, conventional CT and MRI rely on morphological criteria to differentiate malignant from benign lymph nodes. Lymph nodes may be enlarged due to other factors, including inflammation, infection, autoimmune disease, and so forth, and even small lymph nodes may be infiltrated by cancer cells (3,4). Therefore, it remains a challenge for radiologists and clinicians, on the basis of morphological characteristics, to distinguish metastatic from non-metastatic lymph nodes. Positron emission tomography (PET)/CT is a generally accepted non-invasive approach for the staging of pulmonary cancer (5,6). However, the use of PET/CT has been known to result in false-negative results for well-differentiated lung adenocarcinoma, and false-positive results for inflammatory lesions (7,8). Furthermore, PET/CT is not widely available on account of its great expense.

Recently, diffusion-weighted imaging (DWI) has successfully been applied to thoracic imaging, and has demonstrated great potential in the detection of lung cancer and metastatic lymph nodes (5,9–15). However, previously published studies have reported the diagnostic performance of DWI for the detection of metastatic lymph nodes with widely varying sensitivities and specificities (60–91 and 70–100%, respectively) (11,14–22). Thus, there has been a burgeoning interest in meta-analysis to identify consistent results for the diagnostic performance of DWI in detecting metastatic lymph nodes of lung cancer.

Wu et al (23) published a meta-analysis that compared the diagnostic capability of DWI with that of 18F-fluorodeoxyglucose PET/CT in the lymph node staging evaluation of non-small cell lung cancer (NSCLC); however, only three DWI studies were included in that study. Another meta-analysis of differential diagnosis between metastatic and non-metastatic lymph nodes using DWI featured a great variety of primary tumors, although this meta-analysis only included four DWI studies on lung cancer (24). The different biological or histological characteristics of each primary tumor may lead to decreased reliability of the pooled results. The aim of the present study was to perform a meta-analysis for more comprehensive and precise assessment of the diagnostic value of DWI, in order to discriminate between metastatic and non-metastatic lymph nodes in patients with lung cancer.

Materials and methods

Literature search strategy

Systematic literature searches of the PubMed, Embase, Web of Science, Cochrane Library, China Biomedicine, China National Knowledge Infrastructure and Wanfang databases were performed for English and Chinese language studies published between January 1994 and June 2015. A search algorithm based on a combination of the following terms was used: ‘Diffusion magnetic resonance imaging’ OR ‘diffusion MR imaging’ OR ‘diffusion weighted imaging [MeSH]’ OR ‘DWI’ AND ‘lung neoplasms [MeSH]’ OR ‘lung cancer’ OR ‘lung lesions’ AND ‘lymphatic metastasis’ OR ‘lymph node metastasis’. The reference lists of identified articles were also manually searched to obtain additional papers.

Inclusion and exclusion criteria

All DWI studies for detecting lymph node metastasis in patients with lung cancer yielded by our document retrieval were assessed for potential eligibility. The studies that fulfilled the following inclusion criteria were included: i) The studies were published in the English or Chinese language in a peer-reviewed journal; ii) the studies must have evaluated the diagnostic performance of DWI for identifying metastatic and non-metastatic lymph nodes in patients with lung cancer; iii) the studies had to present on lymph node-based, not on patient-based, statistical results; iv) sufficient information regarding true-positive (TP), false-positive (FP), true-negative (TN) and false-negative (FN) values could be identified or calculated from data in the original articles; v) pathological examination and/or follow-up imaging findings should have been set as the reference standards; and vi) where data or subsets of data were reported in more than one article, the article with the larger sample size, or the latest article, was adopted.

Studies were excluded if they met at least one of the following deficiencies: i) The studies were conference abstracts, letters, comments, case reports or review articles; ii) the studies were not associated with lung cancer; iii) combinations of multiple magnetic resonance sequences, i.e., contrast enhanced imaging and DWI, were applied in the studies, and the diagnostic performance of DWI could not be assessed alone; iv) sufficient raw data could not be obtained from the articles to construct the four-fold (2×2) tables; or v) sample sizes of <20 patients or 30 lymph nodes were presented in the studies.

Data extraction and quality assessment

Two authors (G.-X.C and M.-H.W) independently searched the literature, screened the retrieved articles, and extracted relevant data, including the study characteristics and results using Microsoft Excel sheets. For each included study, the following characteristics were extracted: The first author's name, year of publication, age and gender of patients, number of patients and lymph nodes, type of lung cancer, study design, patient selection, blind study, B-value, diagnostic threshold of apparent diffusion coefficient (ADC), reference standard, type of scanner, MRI field strength and the relevant data (i.e., TP, FP, TN and FN). The quality of the included studies was also assessed independently by the same two authors according to the ‘quality assessment of diagnostic accuracy studies’ (QUADAS-2) (25). If agreement was not reached between the two authors, a third author (T.Z.) evaluated all discrepant items, and the major viewpoint was adopted for statistical analysis.

Statistical analysis

Heterogeneity was assessed using the Q statistic of the Chi-square value test and inconsistency index (I2), and P<0.1 or I2>50% suggested the presence of heterogeneity (26). If heterogeneity was detected among the studies, the test's performance was summarized using a random-effects model (the DerSimonian and Laird method); otherwise, a fixed-effects model (the Mantel-Haenszel method) was used. The predominant outcome indexes were pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and their corresponding 95% confidence intervals (CIs). The summary receiver operating characteristic (SROC) curve and area under the SROC curve (AUC) were used to demonstrate the diagnostic performance of DWI in the detection of metastatic lymph nodes of lung cancer.

Exploring the sources of heterogeneity is essential to seek the latent factors that affect the pooled accuracy from various studies. In diagnostic accuracy studies, the threshold effect has been regarded as one of the predominant causes of heterogeneity (27). The threshold effect can be recognized visually by observing the typical pattern of a ‘shoulder-arm’ shape in the ROC plane. Furthermore, Spearman's correlation coefficient between the logit of sensitivity and the logit of (1-specificity) was calculated to identify the presence of a threshold effect. Strong positive correlation, with P<0.05, was considered to indicate the existence of a threshold effect (28).

In addition, in order to investigate the influence of heterogeneity between individual studies on pooled diagnostic performance, meta-regression analysis was performed to identify those variables causing heterogeneity. Subgroup analysis was further performed according to these identified variables. The freely available software, Meta-DiSc (version 1.4), was used to perform the heterogeneity test, assessment of threshold effect, pooling of diagnostic performance, and the meta-regression and subgroup analyses (29).

The possible existence of publication bias was visually assessed with the Deeks' funnel plot and asymmetry test, using Stata software (version 12.0; StataCorp LP, College Station, TX, USA). An inverted symmetrical funnel plot with P>0.05 was considered to indicate the absence of publication bias (30).

Results

Included studies

The search strategy initially yielded 115 studies, of which 10 studies (11,14–22) with 11 datasets met the inclusion criteria, including 796 patients (472 male and 324 female; mean age, 66.3 years) with a total of 2,433 lymph nodes. In one study (15), the analyses were performed based on normal-sized lymph nodes and enlarged lymph nodes, respectively. Thus, that study was regarding as comprising two unique datasets, which were included independently in the present meta-analysis. The detailed flowchart of the study selection process is shown in Fig. 1, and the characteristics of the included studies are summarized in Table I.

Figure 1. Flow-chart of the study selection process. DWI, diffusion-weighted imaging.

Table I. Characteristics of the included studies in the meta-analysis.

Table I.

Characteristics of the included studies in the meta-analysis.

Study ID	First author	Year	Age (range/average)	Gender (M/F)	Study sizea	Type of lung cancer	Design	Patient enrollment	Blind	Reference standard	B-value (s/mm2)	DT of ADC (mm2/s)	Field strength	Type of scanner	Refs.
1	Chen	2010	35–76/51	35/21	56/135	NSCLC	UN	Cons	Yes	Pathology and/or follow-up	1,000	UN	1.5 T	Siemens	(14)
2	Nakayama	2010	48–82/68	38/32	70/56	NSCLC	Retro	UN	UN	Pathology	1,000	1.54×10−3	1.5 T	Siemens	(20)
3	Nomori	2008	38–82/70	47/41	88/734	NSCLC	Pros	UN	UN	Pathology	1,000	1.60×10−3	1.5 T	Phillips	(11)
4	Usuda	2013	37–83/68	94/64	158/705	Lung cancer	UN	UN	UN	Pathology	800	1.70×10−3	1.5 T	Siemens	(18)
5	Xu (ELN)	2014	42–78/55	27/15	42/33	NSCLC	Pros	Cons	Yes	Pathology	1,000	1.98×10−3	1.5 T	Phillips	(15)
6	Xu (NLN)	2014	42–78/55	27/15	42/86	NSCLC	Pros	Cons	Yes	Pathology	1,000	2.04×10−3	1.5 T	Phillips	(15)
7	Ohno	2011	61–83/73	136/114	250/270	NSCLC	Pros	Cons	Yes	Pathology	1,000	2.50×10−3	1.5 T	Phillips	(19)
8	Bai	2013	27–72/59	16/10	26/62	NSCLC	UN	UN	UN	Pathology and/or follow-up	600	1.92×10−3	1.5 T	Phillips	(21)
9	He	2013	33–77/58	27/9	36/206	Lung cancer	UN	UN	UN	Pathology	500	3.19×10−3	1.5 T	GE	(16)
10	Zhang	2013	41–72/59	17/8	25/78	NSCLC	UN	UN	Yes	Pathology	800	2.21×10−3	3.0 T	Siemens	(17)
11	Zeng	2012	48–69/58	35/10	45/68	NSCLC	UN	UN	Yes	Pathology	600	2.32×10−3	1.5 T	GE	(22)

a Number of enrolled patients and lymph nodes. NSCLC, non-small cell lung cancer; ELN, enlarged lymph nodes; NLN, normal-sized lymph nodes; DT, diagnostic threshold; ADC, apparent diffusion coefficient; UN, unclear; Retro, retrospective; Pros, prospective; Cons, consecutive; GE, General Electric.

Assessment of study quality and publication bias

The quality assessment of the included studies was moderate according to the QUADAS-2 tool, and the results are shown in Fig. 2. The Deeks' funnel plot asymmetry tests revealed no strong evidence for the presence of publication bias (bias=−0.15, P=0.887).

Figure 2. Methodological quality assessment of the included studies, according to QUADAS-2. (A) Overview of the entire meta-analysis. (B) Quality assessment as determined on an individual study basis. QUADAS, quality assessment of diagnostic accuracy studies.

Homogeneity tests and threshold effect analysis

Homogeneity tests demonstrated significant evidence of heterogeneity for sensitivity (I2=73.7%, P<0.0001), specificity (I2=95.2%, P<0.0001), PLR (I2=86.0%, P<0.0001), NLR (I2=68.6%, P=0.0004), and DOR (I2=66.1%, P=0.0010). The typical pattern of a ‘shoulder-arm’ shape in the ROC plane was not identified. Furthermore, Spearman's correlation coefficient, a further test for the threshold effect, was calculated to be 0.209 (P=0.537), which indicated that no threshold effect existed.

Diagnostic accuracy of DWI

Due to the notable heterogeneity among the included studies, pooled values were determined using a random-effects model analysis. The pooled diagnostic sensitivity was 0.78 (95% CI: 0.74–0.81), and the pooled diagnostic specificity was 0.88 (95% CI: 0.86–0.89). The PLR, NLR, and DOR were 7.11 (95% CI: 4.39–11.52), 0.24 (95% CI: 0.18–0.33), and 31.14 (95% CI: 17.32–55.98), respectively. The forest plots for the included studies are shown in Fig. 3. The SROC curve showed that the Q value was 0.83, while the AUC was 0.90, suggesting good diagnostic performance. The SROC curve is shown in Fig. 4.

Figure 3. Forest plots. Forest plots of the (A) sensitivity, (B) specificity, (C) PLR. (D) NLR and (E) DOR with corresponding 95% CIs for DWI in detection of metastatic lymph nodes of lung cancer from all included studies are shown. QUADAS, quality assessment of diagnostic accuracy studies; PLR, positive likelihood ratio; NLR, negative likelihood ratio; DOR, diagnostic odds ratio; CIs, confidence intervals; DWI, diffusion-weighted imaging.

Figure 4. SROC curve for DWI in detection of metastatic lymph nodes of lung cancer from all included studies. SROC, summary receiver operating characteristic; DWI, diffusion-weighted imaging; AUC, area under the curve.

Meta-regression and subgroup analysis

Meta-regression analysis was subsequently performed to explore other sources (except for threshold effect) of heterogeneity. The results revealed that patient selection, type of lung cancer, number of enrolled lymph nodes, reference standard, B-value and type of scanner were the sources of heterogeneity (P<0.05). Study design, blinding to all items of information concerning the other test results, and field strength did not statistically contribute to the heterogeneity. Table II shows the detailed results of the meta-regression analysis. The subgroup analysis was performed for the above-identified variables whose sample size was not <5 datasets. As the sample sizes of consecutive patient selection (only 4 datasets), the type including small cell lung cancer and reference standard of pathology and follow-up (only 2 datasets) were too small to conduct subgroup analysis, only subgroup analysis of NSCLC and the reference standard of pathology were performed among the three variables. The results of subgroup analysis of NSCLC and B-value equal to 1,000 demonstrated the highest diagnostic performance for pooled sensitivity, specificity and DOR. The results of the subgroup analysis are shown in Table III.

Table II. Results of meta-regression analysis.

Table II.

Results of meta-regression analysis.

Variable	Coefficient	Standard error	P-value	RDOR	95% CI
Design	−0.629	0.5875	0.3630	0.53	0.08–3.46
Patient selection	−1.824	0.5447	0.0154	0.16	0.04–0.61
Blind	−1.455	1.0608	0.2420	0.23	0.01–4.44
Reference standard	−1.752	0.5831	0.0239	0.17	0.04–0.72
Type of lung cancer	1.270	0.4376	0.0198	3.56	1.30–9.76
Study sizea	0.003	0.0008	0.0138	1.00	1.00–1.00
B-value	0.007	0.0016	0.0050	1.01	1.00–1.01
DT of ADC	−4.786	2.0793	0.0828	0.01	0.00–2.68
Type of scanner	0.970	0.3630	0.0442	2.64	1.04–6.71
Field strength	−0.853	0.4270	0.0926	0.43	0.15–1.21

a Number of enrolled lymph nodes. RDOR, relative diagnostic odd ratio; CI, confidence interval; DT, diagnostic threshold; ADC, apparent diffusion coefficient.

Table III. Results of subgroup analysis for DWI in detecting lymph node metastasis.

Table III.

Results of subgroup analysis for DWI in detecting lymph node metastasis.

		Pooled sensitivity		Pooled specificity		DOR
	No. of studies	Value (95% CI)	I2	Value (95% CI)	I2	Value (95% CI)	I2
Totala	11	0.78 (0.74–0.81)	73.0	0.88 (0.86–0.89)	95.2	31.14 (17.32–55.98)	66.1
Type of lung cancer
NSCLC	9	0.80b (0.76–0.85)	50.5c	0.95b (0.93–0.96)	90.7c	37.71b (19.03–74.78)	61.6c
Study sized
≥100	5	0.77 (0.73–0.81)	88.4	0.88 (0.87–0.90)	97.9	37.66b (14.17–100.07)	85.0
<100	6	0.81b (0.74–0.87)	0.0c	0.86 (0.81–0.90)	59.5c	22.96 (12.49–42.23)	0.0c
Reference standard
Pathology	9	0.75 (0.71–0.79)	66.5c	0.88 (0.86–0.89)	96.1	27.46 (14.30–52.71)	68.7
B-value
1,000	6	0.80b (0.75–0.84)	65.3c	0.96b (0.95–0.97)	90.6c	50.73b (19.91–129.26)	68.9
<1,000	5	0.75 (0.70–0.81)	81.7	0.79 (0.76–0.81)	74.3c	18.04 (10.31–31.55)	32.1c
Type of scanner
Phillips	5	0.76 (0.70–0.81)	9.7c	0.96b (0.95–0.98)	91.4c	48.84b (15.90–149.98)	72.5
Siemens and GE	6	0.79b (0.75–0.84)	84.7	0.79 (0.76–0.82)	72.5c	22.54 (11.80–43.06)	54.8c

a Diagnostic accuracy and heterogeneity of all 11 included datasets

b the subgroup of higher diagnostic performance compared with the total

c the subgroup of lower heterogeneity compared with the total

d number of enrolled lymph nodes. CI, confidence interval; NSCLC, non-small cell lung cancer; DOR, diagnostic odds ratio; GE, General Electric.

Discussion

The presence of lymph node metastasis in patients with lung cancer has been demonstrated to be a pivotal prognostic factor for cancer staging and the outcome in these patients (31–33). A number of studies have indicated that DWI is a promising technique to distinguish metastatic from non-metastatic lymph nodes (11,14,15,20). In the present study, the diagnostic performance of DWI in detecting metastatic lymph nodes of lung cancer was investigated. The results revealed that, for lymph node metastasis detection, DWI had relatively low sensitivity (78%) and high specificity (88%). The SROC curve presents a global summary of test performance, and displays the trade-off between sensitivity and specificity (34–36). The AUC was revealed to be 0.90, which indicated a good diagnostic ability. The DOR derived from different combinations of sensitivity and specificity is the ratio of the odds of positivity in the diseased state relative to the odds of positivity in the non-diseased state, and may be used as a single summary measure. The value of a DOR ranges from 0 to infinity, and the greater it is, the better it is able to distinguish test performance (23,37). In the present study, it was identified that the DOR for DWI in detecting metastatic lymph nodes of lung cancer was 31.14 (95% CI: 17.32–55.98), which also indicated high overall diagnostic accuracy.

In clinical practice, it is necessary to know how a diagnostic test result predicts the risk of abnormality. The likelihood ratio (LR) is generally considered to be one of the best measures of diagnostic accuracy, and is even more helpful for decision-making in clinical practice compared with measures of sensitivity, specificity or the AUC (24). An LR >1 is indicative of the fact that the test result is associated with the presence of the disease, whereas, by contrast, an LR <1 indicates that the test result is associated with the absence of disease. The more that the LR deviates from a value of 1, the stronger is the proof for either the presence or absence of disease (38). A higher PLR lends itself to a ruling in favor of a disease, whereas a lower NLR would lend itself to ruling out the possibility of disease. In order to be moderately useful, a diagnostic test should have a higher PLR value (>5) and a lower NLR value (<0.2) (39). In the present study, the pooled PLR and NLR were 7.11 and 0.24, respectively, which demonstrated that DWI had a moderately good diagnostic performance for ruling in cases of lymph node metastasis in patients with lung cancer, but a relatively inferior diagnostic ability for ruling out non-metastatic lymph nodes. Therefore, a lung cancer patient with lymph node metastasis was 7.11 times more likely to have a positive DWI manifestation compared with a patient without lymph node metastasis. Analogously, the probability of having a negative DWI manifestation for lymph node metastasis in patients with lung cancer was 0.24 times (or approximately one-fourth) that of those without lymph node metastasis. In other words, non-metastatic lymph nodes were approximately 4 times more likely to have a negative DWI manifestation compared with metastatic lymph nodes in patients with lung cancer. Owing to the larger number of the included studies in the present meta-analysis, these results were not entirely consistent with previous studies (23,24). However, the results in the present study do circumvent several of the limitations that were acknowledged in previous studies, and provide objective and practical suggestions for DWI in the evaluation of lymph node status in patients with lung cancer.

Significant heterogeneity between the included studies was identified in the present meta-analysis. Exploring the sources of heterogeneity, which is useful in order to understand the potential factors that influence accuracy assessments of the pooled diagnostic performance, is an important aim of meta-analysis (40,41). In the present study, the threshold effect was assessed using the ROC plane and Spearman's correlation coefficient. As indicated by the results of threshold effect analysis, there was no threshold effect, and therefore the heterogeneity may have been caused by other factors, for example, the study characteristics. Thus, meta-regression analysis was performed to further explore the sources of heterogeneity. The results demonstrated that patient selection, the type of lung cancer, the number of enrolled lymph nodes, the reference standard, B-value and type of scanner were strongly associated with diagnostic accuracy. In the subgroup analysis based on the type of lung cancer and the B-value, the results of the subgroup analysis of NSCLC and the B-value equal to 1,000 demonstrated the highest diagnostic performance for pooled sensitivity, specificity and DOR. The results further verified that the type of lung cancer involved and the B-value are able to affect the diagnostic accuracy of DWI, and therefore, the heterogeneity in these studies was increased. In the present study, the two studies did not distinguish the type of lung cancer, i.e. they did not differentiate NSCLC from small-cell lung carcinoma. Previous studies have revealed that the mean ADC value of small-cell carcinoma is significantly different from that of NSCLC (42,43). In addition, the selection of the B-value is particularly important for DWI. At lower B-values, with <600 sec/mm2, signals are prominently influenced by perfusion effects. Thus, their ADC values may not reflect diffusion phenomena alone. To prevent perfusion effects, it is necessary to use higher B-values. However, the higher the B-value, the greater is the chance of distortion and susceptibility artifacts occurring (44,45). The present study revealed that different B-values contributed to adding to the heterogeneity, and studies of the B-value equal to 1,000 had the best diagnostic performance. These findings are helpful for the clinical application of DWI in detecting lymph node metastasis of lung cancer.

Although the present meta-analysis has produced a more robust assessment of the true effect-size, with less random error compared with individual studies, it should be recognized that our study had several limitations. First, although no publication bias was found in our meta-analysis, publication bias may potentially still exist. The present meta-analysis was based exclusively on published studies, and a grey literature search analysis was not performed, which may have led to an overestimation of the true effect. Additionally, the present study only included previously published studies that were written in English or Chinese, which may also have introduced unavoidable inclusion bias. However, this bias would have been likely to have been small, since the majority of studies of high quality were published in English. Secondly, there was a notable heterogeneity among the included studies. Although meta-regression analysis was adopted to explore the sources of heterogeneity, the analysis of heterogeneity may still have been insufficient. The optimal acquisition protocol of DWI has not been defined, and differences attributable to the different manufacturer of the MRI scanner, magnetic field strengths and parameters used in DWI series are capable of affecting the quality of DWI. In the present study, meta-regression analysis and subgroup analysis were utilized to identify variables that may be responsible for the heterogeneity. However, it is impossible to perform meta-regression and subgroup analyses for all variables, due to the excessive number of factors that are involved in the processes of DWI. Therefore, it is necessary to further develop a standard acquisition protocol of DWI as a routine clinical application. Thirdly, among the included studies, there were numerous non-prospective studies, and certain of the studies were designed without a declaration of DWI reviewer-blinding to all items of information concerning the results of other examinations. Furthermore, the reference standards used in two included studies were pathology and/or clinical follow-up, and these studies did not report the exact number of metastatic lymph nodes that had been diagnosed by pathology or follow-up. A reference standard based on clinical follow-up can lead to inaccuracies in the sensitivities or specificities provided by that study, and therefore the credibility of the results is reduced.

In conclusion, our meta-analysis has demonstrated that DWI is a valuable MRI modality, with good diagnostic performance for distinguishing metastatic from non-metastatic lymph nodes in patients with lung cancer. Therefore, DWI has been shown to be a useful supplement to conventional MRI techniques. In the future, larger-scale prospective studies with respect to DWI for the diagnosis of lymph node metastasis will be required to evaluate and confirm its clinical value. Furthermore, optimization of the DWI acquisition protocol, standard image processing and analysis are crucial for the routine clinical application of DWI in detecting lymph node metastasis in patients with lung cancer.

Acknowledgements

The present study was supported by the National Natural Science Foundation of China (grant no. 81541090) and the Joined Foundation of Luzhou Municipal Government and Southwest Medical University [2015LZCYD-S04 (9/15)].

References