Development and preliminary psychometric evaluation of a Narrative Comprehension Assessment (NCA) for Indian children
Figures
Abstract
Narrative comprehension requires children to understand explicitly presented information and integrate events to derive implicit meanings. Despite its developmental and educational importance, English-language narrative assessments developed within the Indian cultural context remain limited. This study developed and conducted a preliminary psychometric evaluation of the Narrative Comprehension Assessment (NCA), a picture-based assessment for Indian children aged 4–8 years. The NCA was developed across three phases. An initial set of 27 comprehension questions was refined through expert review to 20 items comprising 9 literal and 11 inferential questions. The final assessment used two wordless picture narratives and four sequential components: Picture Walk, Retelling, Prompted Comprehension, and Guided Story Narration. Preliminary evaluation was conducted with 80 typically developing children, equally distributed across four age groups from 4–5–7–8 years. Content validity was high (S-CVI/Ave = 0.93), and internal consistency was high for both narratives (α = 0.97 and 0.96). Significant age-group differences were observed across major NCA components for both narratives (p < .001), with large effect sizes. Literal comprehension exceeded inferential comprehension across all age groups (p < 0.001). Guided Story Narration yielded higher overall scores than Prompted Comprehension for both narratives (p < 0.001). Three-week test–retest reliability was good (ICC = 0.89 and 0.91). The findings provide preliminary evidence for the NCA’s content relevance, internal consistency, developmental differentiation, and temporal stability. Larger and more diverse samples are required to further establish broader psychometric, cultural, and clinical validity and to support future normative development.
Citation: Kaur R, Dsouza DJ, Davis D, Dsouza MS (2026) Development and preliminary psychometric evaluation of a Narrative Comprehension Assessment (NCA) for Indian children. PLoS One 21(11): e0363483. https://doi.org/10.1371/journal.pone.0363483
Editor: Michael Flor, Educational Testing Service: ETS, UNITED STATES OF AMERICA
Received: May 29, 2026; Accepted: November 8, 2026; Published: November 20, 2026
Copyright: © 2026 Kaur et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The individual-level participant data for this study are not publicly available due to the ethical and confidentiality considerations associated with the involvement of children. Appropriately anonymized data may be made available to qualified researchers upon reasonable request, subject to appropriate ethical review and applicable data sharing requirements. Data for this study are available upon request from the Father Muller Institutional Ethics Committee via email (fmiethicscommitte@fathermuller.in) for researchers who meet the criteria for access to confidential data.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Narrative comprehension (NC) is a higher-order language ability that requires children to construct coherent representations of events by integrating information about characters, actions, causal and temporal relationships, intentions, and outcomes [1–3]. Operating at the discourse level, it extends beyond understanding individual words or sentences and becomes increasingly sophisticated across childhood as children develop greater capacity to connect events and derive meanings that are not explicitly presented [4,5].
NC can involve both information that is explicitly presented and meanings that must be inferred. This distinction is commonly reflected in literal and inferential comprehension. Literal comprehension (LC) involves understanding explicitly presented information, such as characters, actions, settings, and events, whereas inferential comprehension (IC) requires integrating information across the narrative to derive unstated meanings, including causal relationships, emotions, intentions, predictions, and consequences [3–6]. Although inferential abilities emerge early, they continue to develop in complexity across childhood. Thus, LC and IC represent related but distinguishable aspects of narrative understanding and provide complementary information about how children construct meaning from stories. Given these multiple levels of understanding, their assessment requires approaches that can capture different aspects of how children process and respond to narratives.
Narrative assessment procedures
Narrative abilities are assessed through varied formats, including examiner-presented story retelling, picture-based story generation, wordless picture narratives, and direct comprehension questions. These approaches place different demands on memory, expressive language, narrative organization, and comprehension and therefore assess different aspects of narrative ability.
There are various established measures for assessing NC which reflect this diversity. The Renfrew Bus Story uses examiner-presented story retelling [7]. The Test of Narrative Language–Second Edition (TNL-2) [8] assesses both NC and oral narrative production. Narrative production may be examined through macrostructure, including overall story organization and elements such as setting, goals, attempts, outcomes, and resolution, and microstructure, including vocabulary, syntax, grammatical complexity, and cohesion [9]. The Monitoring Indicators of Scholarly Language (MISL) and Narrative Assessment Protocol/Narrative Assessment Protocol-2 (NAP/NAP-2) provide structured approaches to macrostructural and/or microstructural analysis of narrative production. The development of NAP-2 also reflects an important practical consideration in narrative assessment—the need to obtain clinically useful narrative information while reducing the time and burden associated with transcription and detailed language-sample analysis [9–11].
Picture-based approaches have also been used to examine narrative comprehension. Paris and Paris [12] developed a wordless picture-book approach in which children constructed narratives from sequential illustrations and responded to prompted questions assessing their understanding of story events, causal relationships, and character intentions. Building on this approach, the Squirrel Story Narrative Comprehension Assessment provided a more focused examination of oral narrative comprehension by distinguishing between literal and inferential understanding. Dawes et al. [13] reported developmental increases in both components between 4 and 6 years and differentiation between typically developing children and children with developmental language disorder. Together, these approaches also highlight that narrative assessment may involve both the production of a narrative and the comprehension of its content, making it important to distinguish the demands and information provided by each.
Narrative production and narrative comprehension
Narrative production and comprehension represent related but distinct aspects of narrative ability. Narrative production may be examined through macrostructure, including overall story organization and elements such as setting, goals, attempts, outcomes, and resolution, and microstructure, including vocabulary, syntax, grammatical complexity, and cohesion [9]. Comprehension tasks provide complementary information by assessing understanding of explicitly presented content along with ability to integrate events and derive inferential meaning [13,14]. Together, these approaches can provide information about both how a child constructs a narrative and how well the narrative itself is understood. This complementary approach is illustrated by the Squirrel Story, which combines narrative retelling with cued literal and inferential comprehension measures [13]. Beyond determining which aspects of narrative ability are assessed, however, it is also important to consider whether the assessment materials and procedures are appropriate for the linguistic and cultural context in which they are used.
Cross-cultural narrative assessment
This consideration becomes particularly important when narrative assessments are used with children from different linguistic and cultural backgrounds. Recognizing this need, some narrative assessments have been specifically developed to support assessment across multiple languages and populations. The Multilingual Assessment Instrument for Narratives (MAIN), for example, uses carefully constructed picture-based stories to assess narrative production and comprehension in multilingual children, with attention to linguistic and cultural comparability [15]. Nevertheless, an assessment designed for multilingual use cannot automatically be assumed to be culturally appropriate for every population, as children’s everyday experiences, sociocultural knowledge, and language exposure vary across contexts.
This is particularly relevant to NC because understanding a story depends not only on the information presented within it, but also on the child’s existing knowledge and familiarity with its context. Children draw on prior knowledge of situations, activities, social interactions, and cultural contexts to interpret narrative events and relationships. Cultural familiarity has been shown to facilitate comprehension [16], particularly inferential comprehension even when language proficiency is controlled [17]. Unfamiliar narrative contexts may increase background-knowledge demands required to interpret events, character intentions, and implicit meanings, such that poorer performance may reflect contextual unfamiliarity rather than narrative-comprehension ability alone. Supporting this, Alptekin [17] found better inferential comprehension when an American narrative was adapted to a culturally familiar context despite comparable linguistic demands. Such contextual influences may introduce construct-irrelevant variance [18] and potentially lead to underestimation of a child’s narrative-comprehension ability.
Narrative assessment in the Indian multilingual context
These considerations are particularly relevant in India, where many children acquire English alongside one or more regional or home languages within multilingual educational environments. English also serves as a medium of instruction in a substantial segment of Indian schooling, particularly in private and nationally affiliated schools, making children’s ability to comprehend discourse in English educationally relevant [19]. Importantly, English-medium education in India does not necessarily represent a monolingual English environment; classroom research demonstrates extensive use of multiple languages alongside English [20].
The need to account for linguistic and cultural diversity has been recognized in narrative assessment, with MAIN adapted into several Indian languages through modifications to language, vocabulary, storytelling conventions, and selected contextual elements [21,22]. However, these adaptations primarily address assessment in Indian languages, while English-language familiarity does not necessarily ensure familiarity with the cultural content of assessments developed elsewhere. Children may understand the linguistic demands of an English task yet be unfamiliar with its settings, activities, or social routines. Thus, there remains a need to consider both language and cultural context when assessing English narrative comprehension in Indian children.
Moreover, MAIN provides a broader framework encompassing narrative production and comprehension rather than specifically differentiating LC and IC. Measures explicitly distinguishing these components, such as the Squirrel Story Narrative Comprehension Assessment, have largely been reported in non-Indian populations [13]. At the time of the literature review, we did not identify a published English-language narrative assessment developed within an Indian cultural context that directly and separately evaluates LC and IC in young Indian children.
Although several established measures assess children’s narrative abilities, narrative assessment encompasses multiple constructs, and performance on narrative production or retelling does not necessarily provide a direct measure of how well a child has understood a story. A focused assessment of narrative comprehension is therefore needed to determine not only whether children understand explicitly presented information, but also whether they can integrate events, establish causal and interpersonal relationships, and derive meanings that are not directly stated. Distinguishing literal from inferential comprehension may consequently provide a more specific representation of the levels at which narrative understanding occurs.
The validity of narrative assessment also depends on whether performance predominantly reflects the intended comprehension abilities rather than additional demands arising from unfamiliar contexts. This consideration is particularly relevant in India, where previous assessment-development and adaptation studies have recognized the need to modify stimuli, language, and contextual features to improve their familiarity and cultural appropriateness for Indian children [21–23]. Narrative materials developed in other sociocultural contexts may contain unfamiliar settings, objects, everyday routines, social interactions, or culturally shaped expectations about events and behaviours. Thus, an English-language assessment may be linguistically accessible to an Indian child while some of its contextual content remains unfamiliar. When understanding such content is necessary to interpret an event or generate an inference, performance may partly reflect the child’s contextual knowledge rather than narrative-comprehension ability alone.
Alongside cultural appropriateness, developmental sensitivity is important when selecting and interpreting narrative-comprehension tasks. The 4–8-year age range represents an important period for examining narrative comprehension, spanning the transition from preschool to the early school years, during which narrative understanding shows developmental change [12,13]. Children’s ability to integrate narrative information and derive literal and inferential meaning becomes increasingly sophisticated during these years and continues to develop through later childhood and adolescence rather than reaching a fixed developmental endpoint [4,5]. Thus, examining children aged 4–8 years provides an opportunity to capture an important period of change in these abilities without implying that narrative-comprehension development is complete by this age.
Taken together, these cultural and developmental considerations support the need for an English-language narrative comprehension assessment developed for the Indian context that combines developmentally appropriate and culturally contextualized narrative material with direct examination of LC and IC. In such an assessment, cultural contextualization should extend beyond the language of administration to the scenarios, settings, objects, activities, social interactions, and visual representations through which narrative meaning is conveyed. This may help limit avoidable contextual demands and support interpretation of performance as reflecting narrative comprehension rather than unfamiliarity with the narrative context.
Present study
Against this background, the present study aimed to develop and preliminarily evaluate the Narrative Comprehension Assessment (NCA) for Indian children learning English. The developed NCA was designed to address linguistic, cultural, and developmental considerations to examine literal and inferential narrative comprehension.
The specific objectives were:
- To develop and establish the content validity of culturally and developmentally appropriate narrative materials and comprehension items for the NCA.
- To examine the reliability and preliminary psychometric properties of the NCA in typically developing Indian children aged 4–8 years.
- To examine age-related differences in literal and inferential narrative comprehension across children aged 4–8 years.
Materials and methods
The study followed a staged instrument-development design comprising three sequential phases: Phase I, development of the NCA; Phase II, expert content validation and refinement; and Phase III, pilot application and preliminary psychometric evaluation. The assessment produced in each phase progressed to evaluation in the subsequent phase.
Phase I: Development of the NCA
Phase I involved development of the NCA as an English-language, picture-supported assessment of narrative comprehension for Indian children aged 4–8 years, with particular emphasis on literal comprehension (LC) and inferential comprehension (IC).
Generation of narrative illustrations
The illustrations used in the Mouse Mystery and Remote Car narratives were generated by the author using Midjourney, an AI-based image-generation platform. The illustrations were created approximately two years prior to manuscript submission using original text prompts developed specifically for the Narrative Comprehension Assessment (NCA). No third-party images were uploaded or incorporated into the generation process.
Conceptualization and development
Narrative-assessment approaches and children’s narrative materials were reviewed to inform the format and content of the NCA. These included the Squirrel Story Narrative Comprehension Assessment, picture-based narrative-comprehension work by Paris and Paris [12], wordless materials such as Robot-Bot-Bot and One Frog Too Many, and Indian children’s picture books including Ammachi’s Glasses and Gajapati Kulapati. These materials informed specific design features rather than serving as direct templates for adaptation. Earlier picture-based approaches supported the use of sequential visual information for constructing narrative meaning, while subsequent developments informed the distinction between narrative production and direct comprehension and the separate examination of LC and IC [13].
Questions were generated from specific narrative events and relationships represented across the picture sequence so that each item could be linked to a defined source of explicit or inferential information within the story. This is described in Table 1.
Alternative formats—including examiner-told stories, story generation, picture-supported generation, picture-sequence retelling, and prompted comprehension—were considered. A sequential wordless picture format was selected to provide a common visual narrative while reducing dependence on encoding and recalling a complete examiner-presented verbal story. Sequential scenes allowed temporal and causal relationships, characters’ goals and actions, problems, consequences, and outcomes to be represented visually, providing opportunities to examine both explicitly represented information and meanings requiring integration across events.
Two wordless narratives were developed: Mouse Mystery (22 illustrated pages) and Racing Car (19 illustrated pages). Their length was selected to provide sufficient sequential events to represent story structure, causal relationships, and opportunities for inference while remaining manageable for the target age group. The narratives were constructed using predefined criteria including recognizable event sequences, identifiable characters and actions, temporal and causal relationships, opportunities for literal and inferential interpretation, developmental appropriateness, and familiarity of settings, characters, objects, activities, social interactions, and visual representations within the Indian context. These cultural and contextual features were subsequently evaluated by the expert panel in Phase II.
The NCA targeted children aged 4–8 years, a developmental period during which narrative understanding, including LC and IC, shows substantial change [12,13]. The narratives were therefore designed to provide accessible event sequences while permitting increasingly integrated and inferential interpretations across the target age range.
Assessment structure and item development
The NCA comprised four sequential components differing in the aspect of narrative performance examined and the level of examiner support provided (Table 2).
The components progressed from spontaneous interaction and independent retelling to direct questioning and supported narrative interpretation. Prompted Comprehension and Guided Story Narration used the same response-scoring framework, with the latter providing additional examiner support.
Macrostructural organization was assessed specifically through Retelling, using the six story-grammar elements of Setting, Characters, Goal/Initiating Event, Problem/Episodes, Solution, and Resolution/Ending. These elements represented organization of the overall story structure and were scored independently of the LC and IC questions. Macrostructure was therefore not treated as a separate comprehension domain within the NCA; the principal comprehension constructs were LC and IC.
Development of comprehension questions
An initial pool of 27 narrative-comprehension questions was developed to provide broad coverage of explicit and inferential narrative demands and permit subsequent expert refinement. LC items targeted explicitly represented information, including characters, settings, actions, events, and outcomes, whereas IC items required integration across narrative events to derive emotions and intentions, causal relationships, predictions, dialogue, and broader narrative meaning. Expected responses and scoring criteria were developed concurrently to differentiate absent or inappropriate responses, partial understanding, and more complete, integrated interpretations.
Phase I therefore produced two wordless picture narratives, a four-component administration and scoring framework, and an initial pool of 27 narrative-comprehension questions with corresponding response and scoring criteria. All illustrations used in the two narratives were created specifically for the NCA and were not reproduced, copied, or adapted from third-party copyrighted sources. This complete initial version progressed to expert content evaluation and refinement in Phase II.
Phase II: Expert content validation and refinement
Phase II involved expert evaluation of the initial NCA to examine the relevance, clarity, developmental suitability, cultural/contextual appropriateness, and scoring adequacy of its content before pilot administration. This phase provided content-related validity evidence rather than evidence of broader construct or criterion-related validity.
Expert panel and evaluation
The expert panel comprised five professionals with relevant experience in child language and development: three speech-language pathologists with Master’s degrees in Speech-Language Pathology and at least 8 years of experience in child-language assessment and intervention, and two child psychologists with MPhil qualifications and at least 8 years of experience with preschool- and school-aged children. Experts independently reviewed the complete NCA developed in Phase I. The areas evaluated are summarized in Table 3.
Expert rating and content validation
Experts rated the relevance and appropriateness of NCA content using a five-point scale: Very Poor, Poor, Fair, Good, and Excellent. Ratings of Good or Excellent were considered acceptable. For each item, the item-level Content Validity Index (I-CVI) was calculated as:
Items with an I-CVI < .80 were considered for deletion, while qualitative expert feedback was used to guide modifications to retained items. Modifications primarily involved refinement of question wording and expected-response/scoring criteria to improve clarity and correspondence with the intended narrative information; no changes were made to the overall four-component assessment structure. Following expert evaluation and refinement, the resulting version of the NCA underwent a final structured review before progressing to Phase III pilot administration. Thus, NCA complete tool consisted of two wordless narratives, namely Mouse Mystery (22 illustrated pages) and Racing Car (19 illustrated pages) along with scoring sheets comprising of 9 Literal and 11 inferential comprehension for each story. These stories are attached in the supplementary materials, S1 File and S2 File.
Phase III: Pilot application and preliminary psychometric evaluation
Phase III involved administration of the refined NCA to typically developing children to examine its applicability across the target age range and obtain preliminary evidence regarding score performance and psychometric properties. This phase was considered a pilot evaluation and not a standardization or normative study.
Participants and study setting
The study was conducted in regions of Kerala and Karnataka, India, between July and December 2024. Participants were recruited through local schools and early childhood educational settings and assessed individually in a quiet environment within the school or clinical setting.
The final sample comprised 80 typically developing children aged 4–8 years, equally distributed across four age groups: 4–5, 5–6, 6–7, and 7–8 years (n = 20 per group). This age range was selected to capture the developmental period from preschool through the early school years, during which narrative comprehension, including LC and IC, undergoes substantial development [12,13].
Eligible children had Malayalam or Kannada as their primary/home language and had received English-medium education from the preschool level. English was selected as the language of NCA administration because, despite having Malayalam or Kannada as their primary language, the children had sustained exposure to English from the preschool years and used it routinely as the principal language for academic instruction and school-based learning. Parent reports further indicated regular use and exposure to English in the home environment. Thus, English represented a familiar and functionally used language for the participating children rather than a language encountered only for the purpose of assessment. Children were included when parent and school reports indicated above average academic performance, age-appropriate development and no history of language delay, academic difficulty, cognitive or social impairment, or developmental disability. Children with reported neurological disorders, diagnosed learning disabilities, significant behavioural difficulties interfering with assessment, or previous participation in similar assessment studies were excluded.
Because recruitment was restricted to a single geographic region and socioeconomic status was not systematically measured, the sample was considered a preliminary regional sample rather than representative of Indian children nationally.
NCA administration and scoring
The refined NCA developed through Phases I and II was administered individually by a trained examiner using both wordless picture narratives. Administration followed the same four-component sequence for each child, as summarized in Table 4.
The principal comprehension outcomes were LC and IC scores, while Picture Walk, Retelling, and Guided Story Narration provided complementary information about spontaneous narrative engagement, recall and organization, and comprehension under adult support, respectively.
Administration and scoring burden
The complete NCA required approximately 30–40 minutes per child. Retelling required recording and verbatim transcription for analysis of the six story-grammar elements, whereas Prompted Comprehension and Guided Story Narration were scored directly using predefined response criteria and did not require transcription of extended narrative samples. Thus, although transcription was required for Retelling, the principal LC and IC scores could be obtained directly from the structured scoring framework.
Test–retest reliability
Test–retest reliability was examined in all 80 participants. The NCA was readministered after a three-week interval by the same examiner using the same narratives, administration procedures, and scoring criteria as the initial assessment. Stability of scores across the two administrations was examined using the intraclass correlation coefficient (ICC).
Scope of the preliminary evaluation
No established narrative assessment was administered concurrently as a comparator because this preliminary phase focused on the initial application and score performance of the newly developed NCA following expert refinement. Consequently, the present study does not provide evidence of convergent or criterion-related validity; these properties require subsequent evaluation against appropriate established narrative measures.
The Phase III sample was also not intended to establish normative reference values. Rather, it provided preliminary data on NCA performance across the four age groups and enabled examination of developmental differences and internal consistency. Future validation will require larger multisite samples representing broader linguistic, socioeconomic, educational, and geographic diversity across India.
Ethical considerations
The study was reviewed and approved by the Father Muller Institutional Ethics Committee (Reference number: FMIEC/CCM/023/2024). All procedures performed in this study were in accordance with the ethical standards of the institutional ethics committee.
Statistical analysis
Descriptive statistics were calculated for NCA performance across the four age groups (4–5, 5–6, 6–7, and 7–8 years). Continuous and ordinal performance data were summarized using appropriate measures of central tendency and dispersion, including medians and interquartile ranges (IQRs). Item-level performance of the final comprehension sets was examined separately for the two narratives using item means and response distributions, corrected item-total correlations, and Cronbach’s alpha if item deleted. Internal consistency was evaluated using Cronbach’s alpha. A marked floor or ceiling effect was defined as ≥80% of the Phase III sample obtaining the minimum score (0) or maximum score (2), respectively, on an item.
Age-group differences in Picture Walk, Retelling, Literal Comprehension (LC), Inferential Comprehension (IC), and total Prompted Comprehension scores were examined separately for each narrative using the Kruskal–Wallis test. Effect sizes for age-group differences were estimated using epsilon-squared (ε²). Because the final NCA comprised 9 LC items (maximum score = 18) and 11 IC items (maximum score = 22), LC and IC raw scores were converted to percentages of their respective maximum obtainable scores when the two components were compared directly. Within each age group, paired LC and IC percentage scores were compared using the Wilcoxon signed-rank test because the paired score differences did not consistently satisfy normality assumptions.
Guided Story Narration was examined by comparing each participant’s Prompted Comprehension and Guided Story Narration scores using Wilcoxon signed-rank tests. Comparisons were conducted for the overall sample and separately within each age group for both narratives. Test–retest reliability was examined over a three-week interval using a two-way mixed-effects, single-measurement intraclass correlation coefficient (ICC) based on absolute agreement, with 95% confidence intervals reported.
Statistical significance was set at p < .05. Missing data, if any, were examined before analysis and their handling reported. Analyses were interpreted as providing preliminary psychometric and age-related performance evidence rather than normative or standardization data. All statistical analyses were performed using IBM SPSS Statistics, version 30 (IBM Corp., Armonk, NY, USA).
Results
Phase I: Development of the NCA
Phase I resulted in the initial NCA comprising two wordless picture narratives, Mouse Mystery (22 illustrated pages) and Racing Car (19 illustrated pages), together with four sequential assessment components: Picture Walk, Retelling, Prompted Comprehension, and Guided Story Narration. The narratives incorporated sequential events, identifiable characters and actions, temporal and causal relationships, and opportunities for both explicit and inferential interpretation within developmentally and culturally contextualized scenarios.
The initial structured narrative-comprehension question pool comprised 27 items, developed to sample literal and inferential understanding across the narratives. Corresponding expected responses and scoring criteria were developed alongside the items. This complete initial version, including the narratives, illustrations, assessment procedures, question pool, and scoring framework, progressed to expert content evaluation in Phase II.
Phase II: Expert content validation and refinement
Expert evaluation provided content-related validity evidence for the initial NCA and informed refinement of the comprehension-question pool. For the initial 27-item pool, I-CVI values ranged from 0.60 to 1.00. Twenty items met the predefined criterion of I-CVI ≥ 0.80, whereas seven items fell below this criterion and were identified for deletion based on their ratings and associated expert feedback.
Following expert review, seven items were deleted. Of the 20 retained items, four were modified in response to qualitative feedback and 16 were retained unchanged (Table 5). The revised 20-item set subsequently underwent structured expert review, with I-CVI values ranging from 0.80 to 1.00 and all items meeting the predefined criterion. The S-CVI/Ave was 0.93, indicating strong content-related validity evidence for the final question set.
The final question set comprised 9 LC and 11 IC items. The resulting NCA comprised the two wordless narratives, four-component administration and scoring framework, and final 20-item comprehension set. This refined version progressed to Phase III.
Phase III: Pilot application and preliminary psychometric evaluation
Participant characteristics.
Phase III included 80 typically developing children distributed equally across four age groups (n = 20 per group): 4–5, 5–6, 6–7, and 7–8 years. The sample comprised 32 males (40%) and 48 females (60%). All participants had Malayalam as their primary/home language and English as an additional language, had received English-medium education from the preschool level, and were recruited from urban educational settings. Participant characteristics across age groups are presented in Table 6.
Item-level performance and internal consistency
Item-level performance was examined separately for the two narrative-comprehension sets (Table 7). Item means ranged from 0.68 to 1.98 for Mouse Mystery and 0.78 to 2.00 for Racing Car. No items demonstrated a marked floor effect. One item in each narrative showed a marked ceiling effect, indicating limited score variability for these items within the present sample.
Corrected item-total correlations ranged from .10 to .86 for Mouse Mystery. One item showed a correlation below .30. For Racing Car, corrected item-total correlations for items with sufficient response variability ranged from .68 to .84; one item was answered at the maximum score by all participants and therefore had no response variance, precluding calculation of a meaningful item-total correlation.
Internal consistency was high for both narrative-comprehension sets, with Cronbach’s α = .97 for Mouse Mystery and α = .96 for Racing Car. Cronbach’s α if item deleted ranged from .96 to .97 and .96 to .97, respectively. Although deletion of one item in each narrative produced a slight increase in α, the magnitude of change was negligible.
Overall, the item-level analysis identified one very easy item in each narrative with limited discrimination in the present sample. However, removal of these items did not meaningfully improve internal consistency. Given their conceptual role in sampling basic literal narrative information and the preliminary nature of the present evaluation, the items were retained. Their performance should be re-examined in larger and more heterogeneous samples before considering further item reduction.
Performance across NCA components
Performance across the principal NCA components was examined separately for Mouse Mystery and Racing Car across the four age groups. As shown in Table 8, scores generally increased across successive age groups for Picture Walk, Retelling, and Prompted Comprehension. Kruskal–Wallis analyses identified significant age-group differences for all component scores across both narratives (all p < .001).
- Picture Walk: Picture Walk scores showed a clear age-related pattern across both narratives. For Mouse Mystery, median scores increased from 4.0/10 in the 4–5-year group to 7.0, 9.0, and 10.0 in the successive age groups. The overall age-group difference was significant, H(3) = 68.44, p < .001, ε² = .861. A comparable pattern was observed for Racing Car, with median scores increasing from 3.0/10 to 6.0, 8.0, and 10.0, H(3) = 68.57, p < .001, ε² = .863. Thus, substantial age-group differentiation was observed for Picture Walk performance across both narratives. Scores approached the maximum obtainable score in the 7–8-year group, indicating less variability in this component at the upper end of the sampled age range.
- Retelling: Retelling performance also differed significantly across age groups, although the magnitude of the age-group effect was smaller than that observed for Picture Walk. For Mouse Mystery, median scores increased from 2.5/6 at 4–5 years to 3.0, 4.0, and 5.0 across successive age groups, H(3) = 51.50, p < .001, ε² = .638. For Racing Car, corresponding median scores increased from 2.0/6 to 3.0, 4.0, and 5.0, H(3) = 47.93, p < .001, ε² = .591. The Retelling score represented the inclusion of six predefined macrostructural story-grammar elements: Setting, Characters, Goal/Initiating Event, Problem/Episodes, Solution, and Resolution/Ending. The observed differences therefore reflect age-related variation in the representation of these elements during independent story recall. Unlike Picture Walk, Retelling scores remained below the maximum even in the oldest group, with greater score variability retained across the sampled age range.
- Prompted Comprehension: Prompted Comprehension showed pronounced age-group differences for LC, IC, and overall comprehension across both narratives. For Mouse Mystery, median LC scores increased from 8.0/18 at 4–5 years to 10.0, 16.0, and 18.0 across successive age groups. The difference was significant, H(3) = 68.89, p < .001, ε² = .867. IC scores increased from 5.0/22 to 7.0, 15.0, and 20.0, respectively, H(3) = 67.65, p < .001, ε² = .851. Consequently, total Prompted Comprehension scores increased from 13.0/40 to 18.0, 30.5, and 38.0, with a significant age-group effect, H(3) = 70.79, p < .001, ε² = .892. A similar pattern was observed for Racing Car. Median LC scores increased from 8.0/18 to 9.0, 14.0, and 17.0, H(3) = 60.37, p < .001, ε² = .755. Median IC scores increased from 7.0/22 to 10.0, 15.5, and 21.0, H(3) = 68.48, p < .001, ε² = .862. Total Prompted Comprehension scores correspondingly increased from 14.0/40 at 4–5 years to 18.0, 30.0, and 38.5 across successive age groups, H(3) = 68.27, p < .001, ε² = .859. Across both narratives, therefore, significant age-group differences were evident for understanding of explicitly represented narrative information as well as information requiring inferential integration across events. The broadly comparable pattern across Mouse Mystery and Racing Car further indicates that the observed age-group differentiation was not confined to a single narrative stimulus.
Literal versus inferential comprehension
To permit direct comparison between LC and IC despite their different maximum obtainable scores, raw scores were converted to percentages of their respective maxima (LC/18 × 100; IC/22 × 100). Because the paired percentage-score differences did not consistently satisfy normality assumptions across the analyses, LC and IC performance was compared using the Wilcoxon signed-rank test.
For Mouse Mystery, LC percentage scores were significantly higher than IC percentage scores in each age group. As evident from Fig 1, median LC and IC percentages were 44.4% and 22.7%, respectively, at 4–5 years (W = 0.0, p < .001); 55.6% and 36.4% at 5–6 years (W = 7.0, p < .001); 86.1% and 68.2% at 6–7 years (W = 0.0, p < .001); and 100% and 90.9% at 7–8 years (W = 0.0, p < .001).
A similar pattern was observed for Racing Car. As evident from Fig 2, median LC and IC percentage scores were 44.4% and 27.3% at 4–5 years (W = 2.0, p < .001), 52.8% and 36.4% at 5–6 years (W = 8.0, p < .001), 88.9% and 63.6% at 6–7 years (W = 2.5, p < .001), and 100% and 93.2% at 7–8 years (W = 0.0, p < .001).
Thus, after accounting for the unequal numbers of LC and IC items, relative performance remained significantly higher for LC than IC across all four age groups and both narratives. The difference was evident in the younger and middle age groups and became smaller descriptively in the oldest group, in which both LC and IC scores approached their respective maximum scores.
- Guided story Narration: Guided Story Narration was examined to determine whether narrative-comprehension performance differed when children received additional structured examiner support following the initial Prompted Comprehension condition. Prompted and Guided scores were compared within participants using Wilcoxon signed-rank tests. Analyses were conducted for the overall sample and separately across the four age groups for both narratives. Guided Story Narration scores were significantly higher than Prompted Comprehension scores for both narratives (Mouse Mystery: W = 226.5, p < .001; Racing Car: W = 266.0, p < .001), indicating an overall difference in performance following structured examiner guidance. Age-stratified analyses showed that the pattern varied across age groups and narratives (Table 9). For Mouse Mystery, Guided scores were significantly higher than Prompted scores in the 4–5-year (W = 0.0, p < .001), 6–7-year (W = 17.5, p = .002), and 7–8-year groups (W = 3.0, p = .002). The difference in the 5–6-year group did not reach statistical significance (W = 48.0, p = .055). For Racing Car, significant differences between Prompted and Guided performance were observed in the 4–5-year (W = 3.0, p < .001), 5–6-year (W = 14.0, p = .005), and 7–8-year groups (W = 4.5, p < .001). The difference in the 6–7-year group was not statistically significant (W = 43.5, p = .117). Descriptively, the most pronounced increase following guidance was evident in the youngest age group, particularly for Racing Car. Differences between Prompted and Guided scores were generally smaller at the upper end of the age range, where performance was already approaching the maximum obtainable score. Thus, although structured examiner guidance was associated with significantly higher performance across the sample as a whole, the magnitude and consistency of this difference varied according to age group and narrative.
Test–retest reliability
Test–retest reliability was examined during Phase III in all 80 participants following a three-week interval. The same NCA narratives, administration procedures, scoring framework, and examiner were used at both assessments. Reliability was evaluated using a two-way mixed-effects, single-measurement intraclass correlation coefficient (ICC) with absolute agreement.
Test–retest reliability was high for both narratives. Mouse Mystery demonstrated an ICC of .89 (95% CI [.85, .95], p < .001), while Racing Car demonstrated an ICC of .91 (95% CI [.87, .94], p < .001). These findings provided evidence of good temporal stability of NCA performance over the three-week interval.
Overall Phase III findings
Taken together, Phase III demonstrated significant age-group differences across Picture Walk, Retelling, LC, IC, and total Prompted Comprehension scores for both narratives. The magnitude of the age-group effects was large across these outcomes, although Retelling showed comparatively smaller effects (ε² = .591–.638) than Picture Walk (ε² = .861–.863) and Prompted Comprehension outcomes (ε² = .755–.892). Following adjustment for their unequal maximum scores, LC performance was significantly higher than IC performance across all four age groups and both narratives.
Guided Story Narration provided additional information regarding performance under structured examiner support, with Guided scores being significantly higher than Prompted scores across the overall sample for both narratives, although the magnitude and statistical significance of these differences varied across age groups. Test–retest findings further provided evidence of good temporal stability across the three-week interval.
Overall, these findings provide preliminary evidence regarding item functioning, internal consistency, age-group differentiation, relative LC and IC performance, performance under structured examiner support, and temporal stability of the NCA within the present sample. Given the cross-sectional design and geographically restricted pilot sample, the findings are interpreted as preliminary psychometric and age-related performance evidence rather than normative or standardization data or evidence of longitudinal developmental change.
Discussion
The present study provides an initial evaluation of the Narrative Comprehension Assessment (NCA), with findings that warrant consideration in relation to developmental patterns of narrative understanding, the complementary demands of its assessment components, and the methodological requirements for further psychometric evaluation. The present study provides an initial evaluation of the Narrative Comprehension Assessment (NCA), with findings that warrant consideration in relation to developmental patterns of narrative understanding, the complementary demands of its assessment components, and the methodological requirements for further psychometric evaluation. Expert review and preliminary item analysis provided encouraging evidence regarding the content relevance and internal consistency of the comprehension questions. However, the presence of ceiling effects and a comparatively low item-total correlation for one item suggests that further refinement may be necessary to improve discrimination across the intended age range. Although these items were retained because their removal produced negligible changes in internal consistency and accessible items may be appropriate for younger children, very high internal consistency may also indicate item overlap rather than an optimal item set. Continued empirical refinement is therefore warranted, as has occurred with established narrative measures such as MAIN following accumulation of larger datasets [15,24]
Developmental differences in narrative comprehension
Age-related differences across the NCA were consistent with previous picture-based narrative-comprehension research. Paris and Paris [12], using wordless picture books with children from kindergarten through Grade 2, similarly demonstrated significant improvement in narrative comprehension with increasing age and observed comparable developmental patterns across different picture books. Dawes et al. [13] also reported developmental increases in literal and inferential narrative comprehension among typically developing children aged 4–6 years. The present findings extend these observations to children aged 4–8 years using two narratives developed for the NCA. These age-related differences are unlikely to represent growth in a single language skill. Narrative comprehension requires children to identify important events, maintain and integrate information across a sequence, establish causal and temporal relationships, interpret characters’ intentions and internal states, and construct a coherent representation of the story. These abilities become increasingly sophisticated across the preschool and early school years. Filiatrault-Veilleux et al. [25], for example, demonstrated significant age-related growth in narrative inferencing between 3 and 6 years, with children progressively becoming able to infer story problems, internal responses, goals, attempts, and resolutions. The pronounced age differentiation observed in the NCA therefore supports the selection of 4–8 years as a meaningful developmental period for examining changes in narrative understanding.
Differences across NCA components may also reflect their distinct processing demands. Picture Walk primarily involves interpreting sequential visual information, whereas Retelling additionally requires retrieval, organization, and verbal expression of narrative content. Consequently, Retelling may be influenced by expressive language, memory, and discourse-organization abilities, while direct comprehension questions can examine specific aspects of narrative understanding. This distinction supports the complementary use of Picture Walk, Retelling, and Prompted Comprehension rather than relying exclusively on retelling, consistent with established assessments incorporating both narrative production and comprehension [15].
Literal and inferential comprehension
The consistently stronger performance on LC than IC is compatible with the greater processing demands of inferential comprehension. Literal questions primarily require identification of information directly represented within the narrative, whereas inferential questions require integration across events to derive meanings that are not explicitly available, including causal relationships, emotions, intentions, predictions, and connections with existing knowledge. This finding corresponds with research specifically distinguishing literal and inferential narrative comprehension. Dawes et al. [13], using the Squirrel Story Narrative Comprehension Assessment with 132 typically developing children aged 4–6 years, demonstrated developmental increases in both literal and inferential comprehension. Research focusing specifically on inference helps explain why IC may remain more demanding. Filiatrault-Veilleux et al. [25] found that inferential comprehension emerges during the preschool years but continues to develop substantially thereafter; relatively accessible inferences concerning story problems and predictions appeared earlier, whereas inferences involving goals, attempts, and resolution placed greater demands on younger children. Their review similarly indicated that different forms of causal inference follow different developmental trajectories during the preschool period [26].
The smaller LC–IC difference among older children may reflect increasing ability to integrate narrative information and derive implicit meaning. However, because performance also approached maximum scores, reduced variability may have contributed to this pattern. Future refinement should therefore examine whether more demanding inferential items are needed to differentiate higher-performing children at the upper end of the intended age range.
Guided story narration and supported performance
The findings from Guided Story Narration suggest that structured examiner support may allow children to demonstrate narrative information that was not evident during independent prompted performance. However, the variability in responses across age groups and narratives indicates that the contribution of such support may depend on the child’s existing level of narrative comprehension and the demands of the particular story. Importantly, higher scores following guidance should not be interpreted as evidence that comprehension itself improved. This interpretation has parallels with dynamic approaches to narrative assessment. Petersen et al. [27], for example, used a test–teach–test narrative procedure with Spanish–English bilingual children and found that performance following mediation and responsiveness to support provided information beyond initial narrative performance. The Guided Story Narration component of the NCA should not, however, be considered a formal dynamic assessment because the procedure did not systematically quantify modifiability, amount of examiner assistance, or transfer to an untrained narrative. At this stage, its contribution is more appropriately interpreted as providing complementary information about supported performance. Whether the difference between Prompted and Guided performance has clinical or predictive significance remains to be established.
Temporal stability
The good test–retest reliability observed for both narratives provides preliminary evidence of temporal stability over the three-week interval. Paris and Paris [12] similarly reported comparable developmental patterns across different wordless picture books, supporting the broader use of picture-based narratives to examine narrative comprehension. However, the same NCA narratives and procedures were used during retesting, so familiarity or practice effects cannot be excluded. These findings therefore support stability under repeated administration rather than alternate-form reliability.
Cultural and contextual considerations
Cultural contextualization was a central consideration in NCA development. This is particularly relevant in the Indian context, where English may function as a medium of education while children simultaneously participate in multilingual home, school, and community environments. An English-language assessment may therefore be linguistically accessible while still containing settings, activities, social routines, or expectations that are relatively unfamiliar to a child. Such contextual knowledge may be especially relevant to inferential comprehension, where meaning must be constructed by integrating information presented in the narrative with prior knowledge.
Previous evidence supports the potential influence of cultural familiarity on comprehension. Alptekin [17] found that adapting an English narrative to a culturally familiar context facilitated inferential comprehension among second-language readers while having less influence on literal comprehension. Although the population and task differed from those in the present study, this finding illustrates how contextual knowledge may particularly influence information that must be inferred rather than directly retrieved. Attention to cultural and linguistic context is also reflected in MAIN, whose parallel picture-based narratives were developed with consideration of cultural comparability and subsequently adapted across numerous languages [15]. In the Indian context, Madappa et al. [22] described the adaptation of MAIN to Hindi, Kannada, and Malayalam and highlighted features of Indian storytelling and language use that required consideration during adaptation.
However, the present findings do not demonstrate that cultural contextualization itself improved children’s NCA performance or eliminated cultural bias because culturally familiar and unfamiliar versions were not directly compared. Cultural contextualization should therefore be regarded as an intentional design feature supported by expert review and the theoretical rationale for the assessment rather than as an empirically established outcome of Phase III.
Overall interpretation
An important consideration for the continued development of the NCA is whether its multiple assessment components contribute sufficiently distinct and clinically useful information to justify their combined administration. Although the present findings suggest differences in performance across assessment conditions, further investigation is required to establish their incremental value. Formal item-level content and internal-consistency analyses primarily concerned the 20 comprehension questions and should not be interpreted as equivalent psychometric validation of Picture Walk, Retelling, or Guided Story Narration. Future evaluation should therefore examine the contribution of each component and determine whether assessment efficiency can be improved through item reduction or abbreviated procedures without compromising the information obtained.
Limitations and future directions
Several limitations define the scope of the present findings. First, Phase III included 80 typically developing children, with 20 participants per age group, from a geographically restricted urban educational context. All participants had Malayalam as a primary/home language, English as an additional language, and attended English-medium educational settings. The findings therefore cannot be generalized to the linguistic, socioeconomic, educational, and regional diversity of Indian children, and replication across larger multisite samples is required.
Second, no established narrative or broader language assessment was administered alongside the NCA; consequently, convergent and criterion-related validity remain unknown. The sample was also insufficient for robust examination of dimensionality. Larger studies should examine relationships with established narrative and language measures, test the proposed LC–IC structure, and determine whether all 20 comprehension items provide useful discrimination across the intended ability range. Item-response approaches may further inform item refinement.
Third, the age-related findings were cross-sectional and cannot be interpreted as longitudinal developmental trajectories or normative expectations. Near-maximum performance among some older children also suggests possible ceiling effects at the upper end of the intended age range. Normative or standardization work should therefore follow, rather than precede, broader validation.
Fourth, use of the same narratives during test and retest means that familiarity or practice effects cannot be excluded. Administration and scoring by a single examiner also prevented evaluation of inter-rater reliability. Guided Story Narration additionally requires replication with systematic quantification of examiner support before responsiveness to guidance can be considered an established measurement characteristic.
Finally, cultural contextualization was incorporated into NCA development but was not experimentally compared with culturally unfamiliar material, and Phase III included only typically developing children. Broader cultural validity and clinical discrimination therefore remain to be established across diverse Indian populations and among children with communication and developmental disorders.
Future NCA research should consequently progress through larger multisite and item/structural evaluation, convergent validation, broader cultural and linguistic examination, clinical-group evaluation, and ultimately normative standardization.
Conclusion
The present study provides preliminary evidence supporting the NCA as a structured approach to examining narrative comprehension in children aged 4–8 years. Across two wordless narratives, performance differentiated age groups, literal comprehension consistently exceeded inferential comprehension, structured guidance elicited additional performance, and comprehension scores demonstrated high internal consistency and good temporal stability. Together, these findings support continued investigation of the NCA while emphasizing that broader psychometric, cultural, and clinical validation is required before normative or diagnostic use.
Supporting information
S1 File. Mouse Mystery: Wordless picture narrative developed specifically for the Narrative Comprehension Assessment (NCA).
https://doi.org/10.1371/journal.pone.0363483.s001
(PDF)
S2 File. Racing Car: Wordless picture narrative developed specifically for the Narrative Comprehension Assessment (NCA).
https://doi.org/10.1371/journal.pone.0363483.s002
(PDF)
References
- 1.
Subba Rao TA. Manual on Developing Communication Skills in Mentally Retarded Persons. Secunderabad: National Institute for the Mentally Handicapped; 1992.
- 2. Bruner J. The narrative construction of reality. Crit Inq. 1991;18(1):1–21.
- 3. Kintsch W, van Dijk TA. Toward a model of text comprehension and production. Psychol Rev. 1978;85(5):363–94.
- 4.
Cain K, Oakhill J. Children’s comprehension problems in oral and written language: A cognitive perspective. New York: Guilford Press; 2007.
- 5. Van Kleeck A. Providing preschool foundations for later reading comprehension: The importance of and ideas for targeting inferencing in storybook-sharing interventions. Psychol Sch. 2008;45(7):627–43.
- 6. Dodwell K, Bavin EL. Children with specific language impairment: An investigation of their narratives and memory. Int J Lang Commun Disord. 2008;43(2):201–18. pmid:17852521
- 7.
Renfrew C. Bus story test: A test of narrative speech. 4th ed. Bicester: Winslow Press; 1997.
- 8.
Gillam RB, Pearson NA. Test of Narrative Language–Second Edition (TNL-2). Austin, TX: Pro-Ed; 2017.
- 9. Justice LM, Bowles RP, Kaderavek JN, Ukrainetz TA, Eisenberg SL, Gillam RB. The index of narrative microstructure: A clinical tool for analyzing school-age children’s narrative performances. Am J Speech Lang Pathol. 2006;15(2):177–91. pmid:16782689
- 10. Bowles RP, Justice LM, Khan KS, Piasta SB, Skibbe LE, Foster TD. Development of the narrative assessment protocol-2: A tool for examining young children’s narrative skill. Lang Speech Hear Serv Sch. 2020;51(2):390–404. pmid:32073376
- 11. Gillam SL, Gillam RB, Fargo JD, Olszewski A, Segura H. Monitoring indicators of scholarly language: A progress-monitoring instrument for measuring narrative discourse skills. Commun Disord Q. 2017;38(2):96–106.
- 12. Paris AH, Paris SG. Assessing narrative comprehension in young children. Read Res Q. 2003;38(1):36–76.
- 13. Dawes E, Leitão S, Claessen M, Lingoh C. Oral literal and inferential narrative comprehension in young typically developing children and children with developmental language disorder. Int J Speech Lang Pathol. 2019;21(3):275–85. pmid:31159597
- 14. Dawes E, Leitão S, Claessen M, Kane R. A profile of the language and cognitive skills contributing to oral inferential comprehension in young children with developmental language disorder. Int J Lang Commun Disord. 2018;53(6):1139–49. pmid:30294839
- 15. Gagarina N, Klop D, Kunnari S, Tantele K, Välimaa T, Bohnacker U, et al. MAIN: Multilingual assessment instrument for narratives – revised. Zaspil. 2019;63:20.
- 16. Erten İH, Razı S. The effects of cultural familiarity on reading comprehension. RFL. 2009;21(1):60–77.
- 17. Alptekin C. Cultural familiarity in inferential and literal comprehension in L2 reading. System. 2006;34(4):494–508.
- 18. Oliveri ME, Lawless R. The validity of inferences from locally developed assessments administered globally. ETS Research Report Series. 2018;2018(1):1–12.
- 19.
Borooah VK, Sadana Sabharwal N. English as a medium of instruction in Indian education: Inequality of access to educational opportunities. In: Arsel M, Dasgupta A, Storm S. Reclaiming Development Studies: Essays for Ashwani Saith. London: Anthem Press; 2021. p. 197–226.
- 20. Tsimpli IM, Lightfoot A. India: Investigating multilingual classrooms [Internet]. Paris: UNESCO International Institute for Educational Planning; 2020 Nov 25 [cited 2026 Nov 2]. https://www.iiep.unesco.org/en/articles/india-investigating-multilingual-classrooms
- 21. Chimirala UM. Towards a convivial tool for narrative assessment: Adapting MAIN to Gondi (Dantewada, India), Halbi and Hindi for Gondi- and Halbi-Hindi speaking bilinguals. ZAS Papers Linguist. 2020;64:77–99.
- 22. Madappa M, Gurung Y, Raman MG. Spinning a yarn across languages: Adapting MAIN for India. Zaspil. 2020;64:107–16.
- 23. Nimkar S, Joshi S, Kinikar A, Valvi C, Devaleenal DB, Thakur K, et al. Mullen scales of early learning adaptation for assessment of Indian children and application to tuberculous meningitis. J Trop Pediatr. 2021;67(3):fmaa034. pmid:32620972
- 24. Gagarina N, Lindgren J. New language versions of MAIN: Multilingual assessment instrument for narratives – revised. Zaspil. 2020;64:274.
- 25. Filiatrault-Veilleux P, Bouchard C, Trudeau N, Desmarais C. Comprehension of inferences in a narrative in 3- to 6-year-old children. J Speech Lang Hear Res. 2016;59(5):1099–110. pmid:27732719
- 26. Filiatrault-Veilleux P, Bouchard C, Trudeau N, Desmarais C. Inferential comprehension of 3-6 year olds within the context of story grammar: A scoping review. Int J Lang Commun Disord. 2015;50(6):737–49. pmid:26212590
- 27. Petersen DB, Chanthongthip H, Ukrainetz TA, Spencer TD, Steeve RW. Dynamic assessment of narratives: Efficient, accurate identification of language impairment in bilingual students. J Speech Lang Hear Res. 2017;60(4):983–98. pmid:28350892