The prevalence of Autism spectrum disorders (ASD) is high in Indian children [1], but there is a scarcity of Indian mental health professionals and paraprofessionals skilled in their management [2]. As early detection and initiation of intervention before 4 years has a better prognosis [3], delayed diagnosis has far-reaching implications well in adulthood. There is a strong need to identify a tool best suited for diagnosing ASD in young children. Feasibility issues related to applying international practice parameters in the Indian population [4,5] prompted the development of the indigenous tools – Indian Scale for Assessment of Autism (ISAA) [6] and INCLEN Diagnostic Tool for ASD (INDT-ASD) [7]. Validation studies in diagnosis naive children are inadequate.

In 2016, the National Trust for Welfare of Children with Autism (India) recommended that after formal training, INDT-ASD should be used for diagnosing Autism, and ISAA should be restricted to certify disability in children ³6 years [8-10]. The consensus statement of the Childhood Disability Group (Indian Academy of Pediatrics) listed ISAA in the assessment tools for ASD in children, but did not recommend any age criteria [11]. Pediatricians and paraprofessionals routinely use ISAA to diagnose Autism in children under 5 years of age. Prevalence studies have been published based on evaluation by ISAA, despite its uncertain accuracy [12-15]. These issues have serious implications in the management of pre-schoolers with ASD. Hence, we studied the diagnostic accuracy of ISAA in 2-5-year-old children and also assessed its level of agreement with Childhood Autism Rating Scale 2nd edition (CARS-2) and expert comprehensive assessment.

Each eligible child underwent evaluation for 1.5-2 hours after obtaining written informed consent. This included review of home videos, parental interview, observation of the child and administration of the following: (i) Developmental Profile, 3rd edition (DP 3) for evaluating developmental status by the General Developmental Score (GDS), wherein GDS <70 is a ‘delay’ [16]; (ii) Vineland Adaptive Behavior Scale, 2nd edition (VABS-II) for assessing adaptive function by the Adaptive Behavior Composite (ABC), wherein ABC <70 is ‘low’ [17]; (iii) Diagnostic criteria for ASD of ‘Diagnostic and Statistical Manual of Mental Disorders’, 5th edition (DSM 5), in which all criteria pertaining to persistent difficulty in social communication and social interaction, at least two out of four criteria pertaining to restrictive and repetitive patterns of behaviour, and all obligatory criteria need to be satisfied [18]; (iv) CARS 2 which rates severity of symptoms of ASD based on total scores, ‘minimal-to-no-symptoms’ if 15-29.5, ‘mild-to-moderate’ if 30-36.5 and ‘severe’ if >37 (reference tool 1) [19]; and (v) Brainstem evoked response auditory (BERA). The psychometric instruments were adminis-tered by a clinician psychologist. The final diagnosis (ASD/ No ASD) was established by a professor with 14 years of experience in developmental pediatrics based on a comprehensive assessment (reference tool 2). This comprised of a synthesis of the following; personal elicitation of history, review of home videos, observation, examination, confirmation of DSM 5 criteria and review of assessment scores.

The Hindi version of ISAA (test tool) was then administered by a blinded social worker and trained pediatrician over 45-60 minutes. ISAA comprises of 40 items covering 6 domains; social relationship and reciprocity, emotional responsiveness, speech-language and communication, behavior patterns, sensory aspects and cognitive component [6]. Each item is assigned a Likert score ranging from 1 (rarely) to 5 (always) based on parental history and interviewer observations. The total score denotes the degree of autism and corresponding disability; £70 denotes no Autism (<40% disability), 71 to 106 mild (50-60%), 107 to 153 moderate (70-90%) and >153 severe (100%). Parental counseling was done by the expert, and the children were managed as per hospital protocol.

The outcome measures were: children diagnosed with ASD based on expert assessment, CARS-2 and ISAA. Statistical analysis was performed by the STATA software package, version 10.1, 2011. Standard parameters of diagnostic accuracy and Receiver Operating Charac-teristic (ROC) curves (Open 80 software) were determined.

Results

The flow of participants from recruitment till diagnosis is depicted in Fig. 1. The mean age of the entire group, group A and group B was 37.8 , 28.6 and 47.1 months respectively. The sex ratio was 7:3 in each. The distribution of socio-economic strata was similar, predominantly upper and middle socio-economic strata. An age-wise comparison of the clinical and psychometric profiles of the study population is presented in Table I. Diagnostic accuracy of ISAA (test tool) with respect to expert and CARS-2 evaluation are compared in Table II.

Fig. 1 Flow of recruitment, selection and evaluation of study participants.

 

Table I: Clinical and Psychometric Profiles of Study Population

Profile	Descriptor	2-5 y	A: 2-3 y	B: 3-5 y	P value
		(n=500)	(n=250)	(n=250)	A vs B
Presenting	Developmental delay	235 (47.0)	106 (42.4)	129 (51.6)	0.04
complaints, n (%)	Age inappropriate. cognition	293 (58.6)	142 (56.8)	151 (60.4)	0.41
	Speech issues	356 (71.2)	165 (66.0)	191 (76.4)	0.01
	Regression	95 (19.0)	79 (31.6)	16 (6.4)	<0.05
	Age inappropriate play	184 (36.8)	73 (29.2)	111 (44.4)	0.05
	Age inapp. behavior / SI	276 (55.2)	98 (39.2)	178 (71.2)	<0.05
Co-morbid	Epilepsy	143 (28.6)	71 (28.4)	72 (28.8)	0.92
conditions, n (%)	Sleep related issues	337 (67.4)	179 (71.6)	158 (63.2)	0.04
	Feeding issues	334 (66.8)	172 (68.8)	162 (64.8)	0.34
Significant	Caesarean section	140 (28)	67 (26.8)	73 (29.2)	0.55
histories, n (%)	Preterm gestation	33 (6.6)	18 (7.2)	15 (6.0)	0.59
	Low birth weight	70 (14)	39 (15.6)	31 (12.4)	0.30
	Neonatal hospitalization	179 (35.8)	100 (40.0)	79 (31.6)	0.05
	Positive family history	61 (12.2)	32 (12.8)	29 (11.6)	0.68
Nutritional	Underweight	126 (25.2)	74 (29.6)	52 (20.8)	0.02
status, n (%)	Severe underweight	48 (9.6)	30 (12.0)	18 (7.2)	0.06
	Wasting	102 (20.6)	77 (30.8)	26 (10.4)	<0.05
	Severe wasting	10 (2)	6 (2.4)	4 (1.6)	0.5
	Stunting	123 (24.6)	64 (25.6)	59 (23.6)	0.6
	Severe stunting	69 (13.8)	38 (15.2)	31 (12.4)	0.36
Head	Microcephaly	128 (25.6)	72 (28.8)	56 (22.4)	0.1
circumference, n (%)	Macrocephaly	14 (2.8)	7 (2.8)	7 (2.8)	1.0
Dysmorphism, n (%)	Major physical anomalies	188 (37.6)	81 (32.4)	107 (42.8)	0.02
Development	Delay#, n (%)	481 (96.2)	245 (98)	246 (98.4)	0.73
Adaptive function	Mean ABC (95% CI)	58 (56.9-59)	61.5 (59.9-63)	54.5 (53.2-55.9)	<0.05
Comp. Ass.	Low$, n (%)	442 (88.4)	207 (82.8)	235 (94.0)	<0.05
	ASD, n (%)	440 (88)	205 (82)	235 (95)	<0.05
CARS-2:	No –minimal	22 (4.4)	15 (6)	7 (2.8)	0.08
symptoms of	Total score	21.7 (20.1-23.3)	19.7 (18.6-20.8)	25.9(23.6-28.2)	0.2
ASD, n (%) with	Mild to moderate	85 (17)	55 (22)	30 (12)	<0.05*
mean (95% CI) scores	Total score	31.6 (30.5-32.6)	31.5(30.5-32.5)	31.8(29.6-34)	<0.05*
	Severe	393 (78.6)	180 (72)	213 (85.2)	<0.05*
	Total score	43.5(43.1-43.9)	43.2(42.7-43.7)	43.8(43.2-44.4)	0.15
ISAA:	No Autism	19 (3.8)	14 (5.6)	5 (2)	0.04*
degree of	Total score	62.5(59.4-65.6)	62.9(59.4-66.4)	61.6(54.2-69.0)	0.45
Autism, n (%)	Mild Autism	127 (25.4)	66 (26.4)	61 (24.4)	0.6
with mean (95% CI)	Total score	94.4(92.4-96.4)	92.7(89.6-95.8)	96.3(93.9-98.7)	0.01*
total scores	Moderate Autism	349 (69.8)	168 (67.2)	181 (72.4)	0.2
	Total score	125.2	125.1	125.2
		(124-126.4)	(123.4-126.8)	(123.6-126.8)	0.63
	Severe Autism	5 (1.0)	2 (0.8)	3 (1.2)	0.6
	Total score	154.6	145.5	160.7
		(140.7-168.5)	(117.1-173.9)	(146.7-174.7)	0.48
ABC: Adaptive Behavior Composite, ASD: Autism Spectrum Disorder, CARS-2: Childhood Autism Rating Scale, 2nd edition, ISAA: Indian Scale for assessment of Autism, SES: Socio-economic status, SI: social interaction, Comp.Ass: Comprehensive assessment; #Development has a Delay’ when General Developmental Score <70. $Adaptive Function is ‘Low’ when ABC <70.

Table II: Parameters of Diagnostic accuracy of ISAA in Children Aged 2-5 years

Parameter	2-3 years  (n=250)		3-5 years  (n=250)		2-5 years (n=500)
	Comp. Ass.	CARS-2	Comp. Ass.	CARS-2	Comp. Ass.	CARS-2
Sensitivity	100	97.6	99.6	98.7	99.8	98.1
Specificity	28.9	53.3	33.3	28.6	30	45.5
Posivitive predictive value	86.5	97.2	95.9	97.9	91.3	97.5
Negative predictive value	100	57.1	83.3	40	94.7	52.6
Positive likelihood ratio	1.4	2.1	1.5	1.38	1.4	1.8
Negative likelihood ratio	0	0.1	0.01	0.04	0	0.04
Diagnostic accuracy	87.2	95.0	95.6	96.8	91.4	95.8
CARS-2: Childhood Autism Rating Scale (2nd edition); Comp. Ass: Comprehensive assement.

Table III: Sensitivity and Specificity of ISAA in Children Aged 2-5 Years at Different  Levels and Cut-off Scores

Cut-off on ROC curve	Cut-off	2-3 years		3-5 years		2-5 years
	score	Sn (%)	Sp (%)	Sn (%)	Sp (%)	Sn (%)	Sp (%)
³Level 1	71	100	28.9	99.5	33.3	99.7	30
³Level 2	107	79.5	82.2	77.9	100	78.6	86.7
³Level 3	153	1.5	100	0.85	100	1.1	100
ISAA: Indian Scale for Assessment of Autism, ROC curve: receiver operating characteristic curve, Sn: Sensitivity, Sp: specificity

ROC curves to determine overall accuracy of ISAA for discriminating between ASD and Non-ASD across the ISAA cut-off scores are displayed in Fig. 2 and Web Fig. 1. The optimal threshold point (closest to 0,1) was observed at a total ISAA score of 107. The discriminatory power was good in group A with area under curve (AUC) 0.83 (95% CI 0.78, 0.90), excellent in group B with AUC 0.92 (95% CI 0.89, 0.96) and good (collectively) with AUC 0.85 (95% CI 0.8, 0.9). The age-wise sensitivity and specificity at different cut-off scores at different levels are presented in Table III. The comparative ROC of the discriminatory power of CARS-2 and ISAA with the comprehensive assessment was good: AUC 0.88 (95% CI 0.84, 0.93) for CARS-2 and AUC 0.85 (95% CI 0.80, 0.90) for ISAA.

Discussion

In our study of the diagnostic accuracy of ISAA (test tool) in 2-5-year-old children against a comprehensive expert assessment (reference tool 1) and evaluation by CARS-2 (reference tool 2), sensitivity was acceptable (>80%), but specificity was very low. The ROC also revealed that the optimal sensitivity and specificity in differentiating between ASD and non-ASD children differed from the existing cut-offs resulting in inaccurate assessment of the extent of Autism.

The high sensitivity (>90%) of ISAA found in the present and earlier studies is explainable by similarity of content (autistic symptoms) of the reference tools used (CARS-2 and DSM V) [6,18,19]. Comparison of previous studies with our results revealed a significant difference in specificity at different ages of the participants–92 in 3-22 year olds (mean age 9.4 years) (6), 97 in 2-9 year olds (mean age 4.5 years) (4), 33.3 in 3-5 year olds (mean age 3.9 years), and 28.9 in 2-3 year olds (mean age 2.3 years) in the present study. Specificity becomes sub-optimal (<80%) when the study population is restricted to children under 5 years. Also, a low possible likelihood ratio means that this may lead to significantly large numbers of children incorrectly identified as ASD.

We also observed that the discriminatory power of ISAA and CARS-2 were comparable and acceptable. This can be explained by the overlapping content of both instruments and acquisition of similar levels of competence of the interviewers after training. Although their performance was not on par with the, expert, it was satisfactory (³80%). The possibility that para professionals involved in the care of children with ASD can become adept in administering a validated tool (with acceptable psychometric properties) for evaluation of ASD after appropriate training is definitely worth studying in-depth, especially given the lack of mental health professionals in our country.