INDIVIDUAL OR GROUP-BASED APPROACH TO THE ASSESSMENT OF PRESCHOOL CHILDREN : A COMPARISON USING THE INTERGROWTH-21 ST NEURODEVELOPMENT ASSESSMENT ( INTER-NDA )

It has been estimated that 250 million preschool children in low-and middle-income countries are at risk of not achieving their developmental potential [1]. The first step in addressing this issue is the accurate identification of children at risk of developmental disturbances. Most assessments of early child development (ECD) are complex and targeted at the individual, including the Bayley Scales of Infant Development [2], the Griffiths Scales of Child Development [3] and the Denver Development Screening Test [4]. There is limited information on ‘small groupbased’ approaches to ECD measurement as a strategy to increase coverage and it is not known whether assessing children in small groups is feasible and comparable to individual assessment.


INTRODUCTION
It has been estimated that 250 million preschool children in low-and middle-income countries are at risk of not achieving their developmental potential [1].The first step in addressing this issue is the accurate identification of children at risk of developmental disturbances.Most assessments of early child development (ECD) are complex and targeted at the individual, including the Bayley Scales of Infant Development [2], the Griffiths Scales of Child Development [3] and the Denver Development Screening Test [4].There is limited information on 'small groupbased' approaches to ECD measurement as a strategy to increase coverage and it is not known whether assessing children in small groups is feasible and comparable to individual assessment.
Concerns about the reliability of developmental scores obtained through group-based ECD assessments arise because of inter-child interference, disruption and mimicry.Scoring multiple children simultaneously in an unbiased format can be challenging.It may be possible, however, to implement ECD assessments in small groups in settings such as preschools where the children are already familiar with the assessor, the environment and other participants.
The INTERGROWTH-21 st Neurodevelopment Assessment (INTER-NDA) is a 53-item ECD assessment developed by The International Fetal and Newborn Growth Consortium for the 21 st Century (INTERGROWTH-21 st ) Project [5,6].The INTER-NDA measures cognition, language (expressive and receptive), motor skills (fine and gross), behavior (positive, negative and global), attention and socio-emotional reactivity in 22 to 26 months old children.It consists of directly administered, concurrently observed and caregiver-reported items, scored on a 5-point scale.It was developed for, and implemented in, the INTERGROWTH-21 st Project on an individual child basis.
During the piloting of the INTER-NDA in preschools in Mexico City, it was suggested that a small group strategy might be more appropriate for large-scale dissemination through the educational system and for achieving large coverage as a first level-screening program.We therefore designed this study, in the context of the INTERGROWTH-21 st Project global dissemination program, with the aim of comparing INTER-NDA scores obtained from the individual and group approaches after controlling for age, sex, nutritional status and location of the school.and CBI).They are located in Mexico City catering for children of lower-middle socio-economic status who participated in the study.All the teachers were staff of the Casa Telmex-Centro Histórico, qualified in preschool education through a degree from the institution conducting this study, the Programama de Educación Inicial, Fundacion Carlos Slim, a private educational service committed to promoting the preschool education of Mexican children.

Recruitment
Children were recruited into the study if they: (i) were aged 22 to 26 months; (ii) were healthy at the time of the assessment; (iii) had been enrolled in the preschool for a minimum of 6 months (to ensure they were familiar to their peers and teachers); (iv) were not previously exposed to neurodevelopmental testing, and (v) were native in Mexican Spanish.

Methods
The INTER-NDA's scoring system is an objective report of the child's performance rather than subjective judgment by the assessor.The assessment was designed for use in international settings with emphasis on avoiding the use of culture-specific items.It has good inter-rater and testretest reliability (k=0.70,95%CI 0.47-0.88;k=0.79, 95%CI 0.48-0.96)[5].The administration and scoring of the INTER-NDA follow a standardized protocol (www.intergrowth21.org.uk).The INTER-NDA takes 15-20 min to complete in its original individual format.
The individual assessment protocol of the INTER-NDA was modified (by MF) to ensure that groups of three children could be assessed simultaneously.This involved INTER-NDA items being presented one at a time to each child, ensuring that each child was presented with different items during each presentation cycle.The sequence of presentation of cognitive, language and fine motor items (items 1-21 of the INTER-NDA) for each of the three children in a group was designed so that (i) all items were eventually presented to all the children and (ii) once an item in a cycle was presented to any one child, it would not be presented to the other children until at least two cycles comprising different items were completed.The rationale was to reduce the ability of the second child to remember how to solve a particular task after witnessing the first child's performance.Care was taken to ensure that items, which follow on from each other (for example, the puzzle board and reverse puzzle board, and the naming and matching of colored cubes) were not affected.Children were encouraged to complete gross motor items together, before the conclusion of the assessment.Caregiver reported and behavior related items were completed by the teachers, after the cognitive, language and motor items were administered.
The INTER-NDA was translated into Mexican Spanish following a standardized translation and back-translation protocol (by SS, GGV and FHC) [7].Eight preschool teachers (two from each participating preschool) were trained and standardized in the INTER-NDA during a 3-day training session (by MF and SS) that included both individual and group assessments.Teachers were supported by the technical staff responsible for developmental evaluation of children of the Sistema Nacional para el Desarrollo Integral de la Familia (DIF) of Mexico.
The weight, length and head circumference of the children were measured according to anthropometric protocols of the INTERGROWTH-21 st Project [8].Weight was measured using a portable electronic scale (Seca) with taring capability and calibrated to 0.1 kg.Length was measured in cm to the last completed mm using the Harpenden Portable Stadiometer.The equipment was calibrated before measuring each child.A 0.7-cm-wide, flat paper tape (range, 0-200 cm, calibrated to 1 mm) was used to measure head circumference.Two sets of measurements for weight, length and head circumference were taken and recorded by two trained preschool teachers.The preschool teachers were standardized and trained by the INTERGROWTH-21 st Project's lead anthropometrist (LCI).
Forty-two children were selected from the four preschools and divided into six groups.Each group consisted of two girls and a boy, or two boys and a girl.All assessments were carried out in a preschool classroom.In the first part of the assessment, the teacher administered the INTER-NDA according to the group assessment protocol described above to a group of three children seated at a table.A second teacher was seated non-intrusively in the room, and recorded the performance of each child on a score sheet.
In the second part of the comparison, the second teacher assessed children individually, without other children in the room.At the end of the final assessment, the child's weight, length and head circumference were recorded.
The individual and group assessments were conducted a day apart.For the first three groups of children, the group assessment was conducted on day 1 and the individual assessment on day 2; this order was reversed for the remaining three groups of children.All assessments were conducted in Mexican Spanish.The results were not used for any clinical or academic evaluation of children and were analyzed anonymously without any link or report to the school record of the children.
The authorities with legal responsibility for; and the head of the institution of; the Centros de Cuidado y Atención Infantil of the Sistema Nacional para el Desarrollo Integral de la Familia, Mexico City, Mexico, reviewed the project and provided written approval for its conduct.The investigators then had a detailed discussion with the teaching staff and parents/guardians.The teachers themselves obtained written informed consent from the parents/guardians of all participating children.

Statistical analysis
Descriptive statistics for each INTER-NDA domain, i.e. cognition, gross motor, fine motor, expressive language, receptive language, positive behavior, negative behavior, global behavior, attention and socio-emotional reactivity were calculated for each child for the group and individual assessments.To determine whether there was consistency between scores obtained from group and individual assessments, continuous correlations were calculated for each INTER-NDA domain.To determine whether significant differences in INTER-NDA domain scores existed between individual and group assessments, we carried out paired t tests.Due to the similarity in scores for some INTER-NDA domains, we did not use intra-class correlation coefficients between group and individual scores, as they are known to function best in situations of high heterogeneity in the data [9].
To determine agreement between INTER-NDA domain scores obtained from group and individual assessments a Bland-Altman analysis was carried out using the recommended strategy [9,10], i.e. (i) independent sample t-tests to assess whether there was a difference between group and individual INTER-NDA domain scores within subjects; (ii) biases and limits of agreement statistics; (iii) Bland-Altman plots to identify whether INTER-NDA domain scores differed systematically between the group and individual assessments, and (iv) linear regression analyses of the relationship between the difference score and mean of the group and individual assessments.
Descriptive statistics were calculated for weight, length, head circumference and age at the time of assessment.A covariate analysis was carried out to determine whether weight, length, head circumference, child age and sex, and location of the preschool were associated with mean INTER-NDA scores for group and individual assessments separately.T tests, ANOVAs and continuous correlations were used to examine the effect of: (i) sex; (ii) location of preschool, and (iii) age and anthropometry respectively on INTER-NDA scores.
A hierarchical linear regression analysis was conducted to examine associations between type of assessment (group or individual) and INTER-NDA domain scores using potential confounders as covariates.For each INTER-NDA domain, the analysis consisted of four models.In model 1, unadjusted associations between type of assessment and INTER-NDA domain scores are presented.In model 2, the association is further adjusted for the age and sex of the children.In model 3, the association is adjusted for the factors in model 2 and the nutritional status of the children (weight, length and head circumference).In the final model, the association is adjusted for the factors in model 3 together with the location of the preschool.

RESULTS
Of the 42 children (mean age 2.08 y, SD 0.29) assessed using individual assessments in the study, 26 were girls and 16 were boys.Six of these children were not available for the group assessment; therefore, the study groups consisted of 36 children organized into 12 triads.Ten triads consisted of 2 girls and a boy, and 2 triads consisted of 2 boys and a girl.All children were native in Mexican Spanish and spoke no other languages.
The sequence of the INTER-NDA group assessment for a group of three children is presented in Supporting Information 1.The forms for each child within a group are presented in Supporting Information 2-4.The INTER-NDA form for individual assessment is presented in Supporting Information 5.
The mean INTER-NDA domain scores for the sample for group and individual assessments are presented in table 1, along with the results of the correlational analysis and paired t tests.The results of the correlational analysis found that the mean INTER-NDA domain scores from group and individual assessments were significantly highly correlated for 9 of the 10 domains (r=0.35 to 1.00, p=0.03 to<0.001), with actual scores being identical in five domains.
The only domain where the group and individual assessment scores were not significantly correlated was receptive language (r=0.25,p=0.14).Paired t tests did not find any significant differences in INTER-NDA domain scores between group and individual assessments on any domains.The difference between individual and group scores on the fine motor domain approached significance (individual mean 1.46, SD 0.65; group mean 1.24, SD 0.27; p=0.05); however, for all other domains the scores were very similar.The results of the linear regression showed a significant linear association between group and individual scores on these domains.Taken together, these results suggest that the cognition, gross and fine motor scores for group and individual assessments are positively correlated but not numerically in agreement.Scores on all other domains are numerically in agreement.This is further illustrated in the Bland-Altman plots (fig.1).
For all plots, the central horizontal line represents the mean difference, and the two other horizontal lines represent the upper and lower limits of agreement, respectively.The sample size was 36 for all plots.Examples of the variability observed in the Bland Altman plots are shown for (A) cognition, (B) fine motor, (C) expressive language, (D) receptive language, and (E) socio-emotional reactivity domains.No plots are depicted for behavior (positive, negative and global) and attention because the Bland-Altman analysis indicated no bias at all (0%) in these domains.For gross motor, the scores were identical for 31 of the 36 children; hence, the plot is not presented either.
The results of the exploratory covariate analysis found significant association between INTER-NDA scores and location of preschool, age and anthropometric measures; hence, these were included as covariates in the regression analysis.Although no association with sex was found, sex was included as a covariate in the analysis based on evidence from the literature that neurodevelopment scores can differ between boys and girls [12][13][14].
Table 3 reports the results of the multiple linear regression analysis examining the association of the mode of the assessment (group or individual) on mean INTER-NDA domain scores after adjusting for age, sex, weight, length, head circumference and preschool location (further information on the covariate analysis is presented in S6).For all domains, there were no significant differences between scores obtained from group and individual assessments.The only near significant difference between group and individual scores was for the gross motor domain, which reached p=0.05 level after adjusting for the confounding variables.

DISCUSSION
Through several complementary statistical strategies, we have demonstrated consistent similarities and lack of differences between group and individual INTER-NDA domain scores.Scores for group and individual assessments were significantly correlated for all domains, except receptive language.There were no significant differences between scores obtained from group and individual assessments following inter-group comparisons.The results of the Bland-Altman analysis showed agreement between group and individual scores for the language, behavior, attention and socio-emotional reactivity domains, as well as consistency (but not agreement) between group and individual scores for the cognitive and motor domains.None of the differences between the two assessment strategies examined in the Bland Altman analysis were statistically significant.Adjusting for the age, sex, nutritional status of the children and preschool location did not change these results.Hence, we conclude there is strong preliminary evidence that the INTER-NDA may be used to assess ECD outcomes in groups of three children within a preschool setting, giving test scores comparable to those obtained from individual assessment.
Although no statistically significant differences between group and individual scores were detected, the scores for cognitive, language and motor domains were higher for the group assessment than those for individual children.This finding may reflect the better performance of children when assessed in a group or an imitation effect.
Although there are no previous reports of group ECD assessments, the group-based approach has been applied, to good effect, in the implementation of ECD interventions such as play therapy, physical education and cognitive behavior therapy (CBT) among children with Asperger syndrome, language disorders and anxiety [15][16][17].These studies have shown significant improvement in development skill scores among children who received group-based therapies [15][16][17].The present study is, to our knowledge, the first to apply a group strategy to ECD assessment.Moreover, group and individual assessments were conducted by pre-school teachers with a secondary education degree, specifically trained and standardized in how to administer and score the INTER-NDA.
Importantly, the INTER-NDA could easily be adapted for group assessment because of a number of features: (i) it is easy and quick to administer; (ii) its items are free from cultural and gender bias; (iii) it is based on objective reporting (rather than subjective judgment) of the child's performance, and (iv) each item is reported on a 5-point scale, which includes options for 'no attempt' and 'unable to assess' [5].Nevertheless, our group assessment protocol has been applied only to groups of three children; in its current form, it may not be applicable to larger groups.
Our study is limited in that the teacher who carried out the individual assessment was present in the room during the group assessment and was, therefore, not blind to the child's performance during the group assessment.Six of the children participating in the individual assessments did not return for the group assessment resulting in an imbalance of male-female ratios in the triads, meaning that an analysis by sex is beyond the scope of this study.
Children were assessed on the INTER-NDA in two different formats within a 2-day period.It is possible that repeating the same items within a short span of time could result in familiarity and learning biases during subsequent assessments.In an effort to minimize these effects, we introduced variations in the order of testing between the group and individual assessments; however, the effects of these biases cannot be wholly eliminated.
From an implementation perspective, the preschool teachers neither reported difficulty with the administration of items to three children during the group assessment, nor did they report interference between children.They reported that children tended to encourage each other during the group assessment, and that this expedited the child's assessment on expressive language and gross motor items.The overall impression of the observers was that children tended to be more reserved during individual, compared to group, assessment.The individual and group assessments took approximately 15 and 40 min, respectively.An optimistic interpretation of this feedback might suggest that assessments in the preschool environments, with peers, are closer to the real-life context of children.We believe that it is important, in the group assessment, for children to be assessed by a person, and in the company of peers, with whom they are already familiar.The preschool may, therefore, provide an ideal setting for group-based ECD assessment by teachers.
Our study was conducted in Mexico among lowermiddle class children.In the situation commonly encountered globally, where resources for ECD assessment and intervention are limited, it may be possible to harness the expertise of preschool teachers to enable preschools to function as population-level sites for multi-domain ECD assessment (i.e., beyond that of routine observation of gross motor skills).This may provide a possible solution to one of the ratelimiting steps in population-based ECD assessment, i.e. its dependence on specialized personnel.
In conclusion, we have shown that INTER-NDA domain scores obtained following group and individual assessment of 22 to 26 months old preschool children are comparable.Furthermore, we have demonstrated that is feasible for preschool teachers to administer both group and individual INTER-NDA assessments.These results, if replicated, could strengthen the INTER-NDA's ability to be used as a rapid, scalable ECD assessment tool for field research projects and first level screening strategy at population level.

Supplementary Files S1 INTER-NDA sequence for group assessment
This order of items is designed specifically for the assessment of 3 children simultaneously by a single assessor and as ingle scorer.This proforma is not suitable for group assessments involving more than 3 children.
In the first sequence i.e. sequence no.1-21, the order of the items is different for different children -hence the assessor must pay more attention to the administration of these items and also practice this part of the assessment more until the assessor is fluent in the administration of the items in this sequence.The assessor MUST assess all children in this sequence and never change this sequence.The scorer must also pay attention to this sequence.Sequence numbers 22-53 are not administered but either maternal report or concurrently observed items, and so these can be scored simultaneously for all children.

Table 2 : Bland-Altman analysis for the comparison between individual and group assessments scores INTER- NDA domain One sample t test Bland-Altman Statistics Linear regression (difference scores vs. mean of group and individual scores) Mean difference SD difference
*p<0.05, **p<0.001.n/a = analysis not applicable as values are identical.

NDA assessment form -for child 1 in a group Child 1 -GROUP ASSESSMENT
What is the child's native (first) language?What is the language in which the assessment is being conducted?Does the child speak/understand any languages other than his/her native (first) language?How

Caregiver Reported Child Behaviour Questionnaire Instructions
to caregiver: Please fill in this form to reflect your view of your child's behaviour, even if others do not agree

Able to make a cup of tea with the toy tea set when requested by examiner (Examiner says "Can you make a cup of tea?")
What is the child's native (first) language?______________________________________________________________________ What is the language in which the assessment is being conducted?________________________________________________ Does the child speak/understand any languages other than his/her native (first) language?________________________________

How often were the following behaviours in the child during the assessment?
Combines word and gesture when asked (Do not demonstrate, trials=3, use different example if mother says child does not know the one you are asking)