Section Editor: Abhijeet Saha

This was a prospective cohort study to determine the interrater reliability (IRR) of lung ultrasonography (LUS) and chest radiography (CXR), and evaluate the accuracy of LUS compared with CXR for detecting pediatric pneumonia compared with chest computed tomography (CT) scan. Children aged 3 months to 18 years with a CXR and LUS performed with or without a clinical diagnosis of pneumonia were included in the study. Four pediatric radiologists blinded to clinical information reported findings for the CXR and LUS images, and two radiologists reviewed CT scans to determine an overall finding. IRR was estimated for 50 LUS and CXR images. The main outcome was the finding from CT ordered clinically or the probability of the CT finding for patients clinically requiring CT. Latent class analysis was used to evaluate the sensitivity and specificity for findings (eg, consolidation) for LUS and CXR compared with CT.

Of the 132 patients in the cohort, 36 (27%) had CT performed for a clinical reason. Pneumonia was clinically documented in 47 patients (36%). The IRR (95% CI) for lung consolidation was 0.55 (0.40, 0.70) for LUS and 0.36 (0.21, 0.51) for CXR. The sensitivity for detecting consolidation, interstitial disease, and pleural effusion was statistically similar for LUS and CXR compared with CT; however, specificity was higher for CXR. The negative predictive value was similar for CXR and LUS. The authors concluded that LUS has a sufficiently high IRR for detection of consolidation; and compared with CT, LUS and CXR have similar sensitivity, but CXR is more specific for findings indicating pneumonia.

Relevance: Chest radiographs are commonly done for managing children with suspected lower respiratory tract conditions including pneumonia, although their value in uncomplicated cases is debatable. Diagnosis of pneumonia in developed countries relies heavily on radiologic ‘confirmation’, whereas the WHO recommends (for developing countries) clinical criteria alone. The crux of the problem is that neither clinical criteria nor radiographic findings can be considered foolproof for diagnosing pneumonia. Even attempts to introduce objectivity to clinical as well as radiologic criteria have not improved the situation. For example, the recent Community Acquired Pneumonia Etiology Study (CAPES) [1] from India showed that only 44% children with WHO/IMNCI-defined pneumonia had WHO Class I X-rays (i.e consolidation and/or pleural effusion). Similarly other smaller studies identified radiological features of pneumonia in less than 40% children with clinical pneumonia [2]. In yet another study, almost half the children with clinical pneumonia [3] had normal chest X-ray (CXR); although two-thirds of those with normal CXR had crackles or rales on auscultation. Even a recent Cochrane review [4] reported that CXR do not affect the clinical outcome in children with clinical pneumonia (although this conclusion was based on limited data). The other major limitation with chest radiography is significant inter-observer variation in interpreting images [5,6].

Lung ultrasonography (LUS) is emerging as a potential alternative to CXR, being radiation-free, relatively affordable and feasible at the point-of-care. The critical issue is whether it is efficacious. Several studies [7-10] compared LUS against CXR, and emerging data suggests a slight superiority of LUS over CXR [11]. LUS is also reportedly superior to chest auscultation for identifying pneumonia [12]. However, an important limitation in such studies is the absence of appropriate reference standard(s) for labeling pneumonia. CXR findings are not the ideal reference standard because they suggest (but not necessarily confirm) pneumonia, but pneumonia can exist with normal CXR. The same limitation holds for clinical diagnosis, and also clinical plus radiographic diagnosis. Obviously, the WHO/IMNCI definition of pneumonia cannot be used as the reference standard for pneumonia as it has moderate sensitivity and poor specificity [13].

Some studies evaluating LUS vs CXR have the limitation that the reference standard for pneumonia diagnosis includes CXR findings as one of the criteria [14-16]. Even a recent meta-analysis [17] of eight studies, reporting excellent sensitivity and specificity of LUS, showed that the reference standards included clinical diagnosis plus CXR in five studies and CXR alone in the other three. While such data can give an overall impression of the utility of LUS, they are inappropriate for comparing LUS against CXR.

Against this backdrop, the recent study by Ambroggio, et al. [18] is different from previous studies in that LUS and CXR have not been directly compared against each other, but against another reference standard (viz CT scan). Briefly, the investigators included children with various respiratory illnesses, and performed CXR as well as LUS in them. A few of the children required CT for clinical reasons and this could be used as the reference test. In the others, a statistical method called latent class modeling was used to estimate the likelihood of CT findings and this was used as the reference standard. The twin objectives were to (i) calculate inter-observer concordance of LUS and CXR reports; and (ii) compare LUS versus CXR for diagnosis of pneumonia.

It should be recognized that this is not the first such study. A recent report from Iran [19] compared LUS and CXR against CT in children with suspected pneumonia; both showed good correlation with CT, although LUS was slightly superior for complicated cases, whereas CXR was superior in uncomplicated cases. Overall concordance was best when combination of LUS and CXR findings was compared against CT. Another study from Taiwan [20] showed that LUS finding of impaired lung perfusion correlated well with the severity of lung necrosis on CT; and the degree of impairment could reliably predict the extent of necrosis.

Critical appraisal: Table I presents the methodological characteristics of the study. The study [18] had several methodological refinements. LUS and CXR images were stripped of identification details and observers were blinded to clinical details. Efforts were made to prevent observers from linking LUS findings with CXR findings of individual patients by assigning codes. Both index tests were performed within 36 hours of each other, which is reasonable although not ideal. Detailed description of LUS procedure has been provided, as well as clear definitions for the terms used to describe findings. Similarly CXR was done obtaining both antero-posterior as well as lateral films and interpreted using objective criteria. Both LUS and CXR interpretation were done taking care to report findings in multiple lung/pleural regions. At least two radiologists reported each LUS and CXR image; and 50 images selected randomly were reviewed by four radiologists. All the radiologists involved in this study were highly trained experts, enhancing reliability of the observations.

Table I Critical Appraisal of the Study

The major limitation in this study [18] is the absence of a radiographic definition for the diagnosis of pneumonia, both for LUS as well as CXR; although to be fair, pneumonia is not a ‘radiological diagnosis.’ In this study, four individual radiographic findings have been evaluated on LUS and CXR (viz consolidation, interstitial disease, effusion and other), without clearly defining a priori which of these (singly or in combination) is to be considered diagnostic of pneumonia. Although the authors correctly emphasized that consolidation and pleural effusion are most suggestive of pneumonia, this does not automatically mean that consolidation and pneumonia are synonymous. In the paper [18], the objective in the abstract section stated that LUS was compared with CXR for diagnosing ‘pneumonia’, whereas the Objective in the text stated comparison to detect ‘disease’. This subtle ambivalence is reflected in title of the paper also, wondering whether LUS could be an alternative to CXR in diagnosing pneumonia (which is the issue of interest); rather than stating a mere comparison between LUS and CXR for specific radiologic findings (which is what the investigators did).

Another important limitation is that the term ‘interstitial disease’ has not been defined at all; therefore it could refer to interstitial pattern of radiographic findings (often seen in viral lower respiratory tract infections), or true interstitial disease of childhood (which can be of infective or non-infective origin). Assuming that the authors intended the former, it is pertinent that this pattern is seen in 30% xrays among children with clinically defined pneumonia [1]. Further the sonographic definition of interstitial disease in this study [18] is one of the criteria for diagnosing pneumonia in some other studies. In fact, a Canadian study reported that the pattern defined as ‘interstitial disease’ in this study [18] was the predominant LUS finding among infants with lower respiratory infection and wheezing. This suggests that the interstitial pattern on LUS or CXR should also contribute to the definition of radiologic pneumonia. This has been completely omitted in this study [18]. This is especially pertinent because the results showed that CXR had higher sensitivity and specificity than LUS for interstitial disease; and also better inter-observer concordance. It is possible that if radiographic pneumonia had been defined including the criteria for interstitial disease, the superiority of LUS highlighted by the authors would be negated.

Yet another limitation is that, not all the 132 LUS and CXR images were examined for inter-observer concordance. The reason(s) for this are not elaborated. It would have also been interesting to examine the LUS and CXR data of the 36 children with clinical pneumonia, to study whether either modality correlated with the clinical diagnosis.

The authors emphasized the superiority of LUS over CXR for detecting consolidation and pleural effusion, as well as better inter-observer concordance for these findings. However, careful analysis of the data shows overlapping confidence intervals (CI) for almost all parameters (specific findings as well as inter-observer concordance), suggesting comparability of the two modalities. In fact the only clear differences (i.e. non-overlapping CI) were superior inter-observer concordance of CXR for interstitial findings, and also greater specificity of CXR for pleural effusion as well as interstitial disease, and higher sensitivity (but lower specificity) of CXR for other findings. These data suggest a slight edge of CXR over LUS.

Extendibility: The CXR protocol in this study [18] included interpretation of both antero-posterior and lateral films; this is rarely done in our setting, making it difficult to extrapolate the findings directly. Last but not the least, the observers in this study were trained pediatric radiologists, whereas in real life, most CXR in emergency rooms are read by clinicians managing patients. Previous studies have documented that inter-observer variation is minimized with high level of training and expertise [21,22]. In our setting, the relative ease and low cost of chest radiography (in urban settings), and interpretation at the point-of-care by professionals not trained to interpret radiologic images, creates the dual problems of overuse and potential for incorrect interpretation. Lung ultrasonography has not become popular in our setting as yet; however the same problems are anticipated with this modality also.

Conclusion: This study showed that LUS is comparable to CXR for detection of certain findings suggestive of pneumonia in children (viz consolidation and pleural effusion), but is inferior for detecting interstitial patterns. The overall utility for diagnosis of pneumonia and potential for replacing CXR as the primary imaging modality cannot be gauged from the data presented in this study.

Funding: None; Competing interest: None stated.

Joseph L Mathew

1. Mathew JL, Singhi S, Ray P, Hagel E, Saghafian-Hedengren S, Bansal A, et al. Etiology of community acquired pneumonia among children in India: prospective, cohort study. J Glob Health. 2015;5:050418.

2. Simbalista R, Andrade DC, Borges IC, Araújo M, Nascimento-Carvalho CM. Differences upon admission and in hospital course of children hospitalized with community-acquired pneumonia with or without radiologically-confirmed pneumonia: a retrospective cohort study. BMC Pediatr. 2015;15:166.

3. Nahir B, Zimmerman DR, Applebaum Y, Kovalski N. New guidelines to decreasex-ray use may increase unnecessary antibiotic use. Harefuah. 2013; 152:713-5.

4. Cao AM, Choy JP, Mohanakrishnan LN, Bain RF, van Driel ML. Chest radiographs for acute lower respiratory tract infections. Cochrane Database Syst Rev. 2013;12:CD009119.

5. Williams GJ, Macaskill P, Kerr M, Fitzgerald DA, Isaacs D, Codarini M, et al. Variability and accuracy in interpretation of consolidation on chest radiography for diagnosing pneumonia in children under 5 years of age. Pediatr Pulmonol. 2013;48:1195-200.

6. Bada C, Carreazo NY, Chalco JP, Huicho L. Inter-observer agreement in interpreting chest X-rays on children with acute lower respiratory tract infections and concurrent wheezing. Sao Paulo Med J. 2007;125:150-4.

7. Urbankowska E, Krenke K, Drobczyñski £, Korczyñski P, Urbankowski T, Krawiec M, et al. Lung ultrasound in the diagnosis and monitoring of community acquired pneumonia in children. Respir Med. 2015;109:1207-12.

8. Ho MC, Ker CR, Hsu JH, Wu JR, Dai ZK, Chen IC. Usefulness of lung ultrasound in the diagnosis of community-acquired pneumonia in children. Pediatr Neonatol. 2015;56:40-5.

9. Esposito S, Papa SS, Borzani I, Pinzani R, Giannitto C, Consonni D, et al. Performance of lung ultrasonography in children with community-acquired pneumonia. Ital J Pediatr. 2014;40:37.

10. Shah VP, Tunik MG, Tsung JW. Prospective evaluation of point-of-care ultrasonography for the diagnosis of pneumonia in children and young adults. JAMA Pediatr. 2013;167:119-25.

11. Guerra M, Crichiutti G, Pecile P, Romanello C, Busolini E, Valent F, et al. Ultrasound detection of pneumonia in febrile children with respiratory distress: a prospective study. Eur J Pediatr. 2016;175:163-70.

12. Lovrenski J, Petroviæ S, Balj-Barbir S, Jokiæ R, Vilotijeviæ-Dautoviæ G. Stethoscope vs. ultrasound probe - which is more reliable in children with suspected pneumonia? Acta Med Acad. 2016;45:39-50.

13. Chavez MA, Naithani N, Gilman RH, Tielsch JM, Khatry S, Ellington LE, et al. Agreement between the world health organization algorithm and lung consolidation identified using point-of-care ultrasound for the diagnosis of childhood pneumonia by general practitioners. Lung. 2015;193:531-8.

14. Iorio G, Capasso M, De Luca G, Prisco S, Mancusi C, Laganà B, et al. Lung ultrasound in the diagnosis of pneumonia in children: proposal for a new diagnostic algorithm. Peer J. 2015;3:e1374.

15. Reali F, Sferrazza Papa GF, Carlucci P, Fracasso P, Di Marco F, et al. Can lung ultrasound replace chest radiography for the diagnosis of pneumonia in hospitalized children? Respiration. 2014;88:112-5.

16. Caiulo VA, Gargani L, Caiulo S, Fisicaro A, Moramarco F, Latini G, et al. Lung ultrasound characteristics of community-acquired pneumonia in hospitalized children. Pediatr Pulmonol. 2013;48:280-7.

17. Pereda MA, Chavez MA, Hooper-Miele CC, Gilman RH, Steinhoff MC, Ellington LE, et al. Lung ultrasound for the diagnosis of pneumonia in children: a meta-analysis. Pediatrics. 2015;135:714-22.

18. Ambroggio L, Sucharew H, Rattan MS, O’Hara SM, Babcock DS, Clohessy C, et al. Lung ultrasonography: A viable alternative to chest radiography in children with suspected pneumonia? J Pediatr. 2016; 176:93-98.

19. Hajalioghli P, Nemati M, Dinparast Saleh L, Fouladi DF. Can chest computed tomography be replaced by lung ultrasonography with or without plain chest radiography in pediatric pneumonia? J Thorac Imaging. 2016;31:247-52.

20. Lai SH, Wong KS, Liao SL. Value of lung ultrasonography in the diagnosis and outcome prediction of pediatric community-acquired pneumonia with necrotizing change. PLoS One. 2015;10:e0130082.

21. Xavier-Souza G, Vilas-Boas AL, Fontoura MS, Araújo-Neto CA, Andrade SC, Cardoso MR, et al. The inter-observer variation of chest radiograph reading in acute lower respiratory tract infection among children. Pediatr Pulmonol. 2013;48:464-9.

22. Johnson J, Kline JA. Intraobserver and interobserver agreement of the interpretation of pediatric chest radiographs. Emerg Radiol. 2010;17:285-90.

Funding: None; Competing interest: None stated.

1. Pereda MA, Chavez MA, Hooper-Miele CC, Gilman RH, Steinhoff MC, Ellington LE, et al. Lung ultrasound for the diagnosis of pneumonia in children: a meta-analysis. Pediatrics 2015;135:714-22.

2. Jones BP, Tay ET, Elikashvili I, 　Sanders JE,　Paul AZ,　Nelson BP, et al. Feasibility and safety of substituting lung ultrasonography for chest radiography when diagnosing pneumonia in children: a randomized controlled trial. Chest. 2016;150:131-8.

3. Shah VP, Tunik MG, Tsung JW. Prospective evaluation of point-of-care ultrasonography for the diagnosis of pneumonia in children and young adults. JAMA Pediatr. 2013;167:119-25.