A hybrid type I randomized effectiveness-implementation trial of a Naturalistic Developmental Behavioral Intervention in the Part C early intervention system: study protocol

Abstract

Background

Participation in Naturalistic Developmental Behavioral Interventions (NDBI) is associated with significant improvements in functioning for toddlers with, and showing early signs of, autism spectrum disorder. The Part C Early Intervention (EI) system, which is publicly funded and available in all U.S. states, offers an optimal infrastructure through which toddlers can receive NDBIs. This study seeks to assess the effectiveness and fit of one NDBI, Caregiver Implemented Reciprocal Imitation Teaching (CI-RIT), within the Part C EI system.

Methods

This hybrid type 1 effectiveness/implementation trial uses a multi-site randomized control design to simultaneously test effectiveness and collect implementation data on CI-RIT in the Part C EI system across four states: Illinois, Massachusetts, Michigan and Washington. Participants include EI providers (target n = 160) who are randomized to either CI-RIT or treatment as usual (TAU), and child/caregiver dyads on their caseloads (target n = 440). Primary effectiveness outcomes focus on (1) child social communication, joint attention, motor imitation; and (2) caregiver responsivity, implementation fidelity of RIT, and self-efficacy, which are all measured at baseline and then 4-months and 9-months after baseline. Implementation outcomes include CI-RIT modifications, treatment acceptability, fidelity of CI-RIT coaching, and RIT session completion.

Discussion

This study represents a critical effort to transport an evidence-based NDBI, CI-RIT, into a national service delivery setting, the Part C EI system. The large, multi-site nature of the trial provides the opportunity to address critical questions about training and intervention effectiveness, which will, in turn, optimize and support CI-RIT implementation at scale.

Trial registration

The trial protocol is registered at ClinicalTrials.gov (NCT05114538; Registration date: 10/28/2021).

Background

Participation in early, developmentally appropriate, specialized intervention in the form of Naturalistic Developmental Behavioral Interventions (NDBI) [1] is associated with significant improvements in social, language, cognitive, and play functioning for toddlers with, and showing early signs of, autism spectrum disorder (ASD) [2,3,4]. NDBIs integrate principles from developmental science and Applied Behavior Analysis (ABA) and are meant to be utilized in the context of meaningful interactions with key interaction partners (including family members). Although NDBIs vary in terms of delivery format, intensity, and complexity, they emphasize similar intervention targets (i.e., language and social communication) and use several common intervention strategies (e.g., following the child’s lead, arranging the environment for success, prompting and natural reinforcement) [1]. Data accumulated over the last two decades support the effectiveness of NDBIs for addressing core impairments in social interaction, imitation, communication, and play, and for improving parent and family functioning in this population [5,6,7,8,9,10,11,12,13]. Despite the promise of NDBIs, there is a substantial gap between the science of early ASD intervention as developed in the lab and its application in community practice.

The Part C Early Intervention (EI) system is federally mandated to serve families of children from birth through three years of age with developmental delays; as such, it offers an optimal existing infrastructure through which toddlers showing early social communication difficulties can begin to receive NDBIs, even before receiving a formal diagnosis of ASD. EI programs are publicly funded, offer services at minimal cost, serve families from diverse backgrounds, and are available in every state. Is it not surprising then, that the EI system may be the first intervention touchpoint for families when ASD is suspected. Unfortunately, the EI system’s current capacity for providing the evidence-based interventions shown to be most effective for this population is severely challenged by the rapid rise in ASD prevalence, significant variability in the training and practice of EI providers, large and clinically diverse caseloads, and high rates of provider turnover within the system [14]. In this context, programs that are highly complex (e.g., include many different intervention strategies), intensive (e.g., require many hours per week for months or years), and involve significant training and ongoing support for providers are likely to fall short. Thus, for an NDBI to be successfully implemented in this setting it must, at a minimum, be easy to learn and use, deliverable in small doses, appropriate for children prior to a formal autism diagnosis, and fit well within the priorities and philosophy of the Part C EI system.

Reciprocal Imitation Teaching (RIT) [7, 15,16,17,18,19,20,21,22,23,24,25] is an NDBI that fits these specific needs. First, it is easy to learn, even for those who have limited backgrounds in ASD and intervention delivery [17], as well as by caregivers and siblings [16, 23, 25, 26]. Further, it can be used at low intensities (e.g., 1–3 h per week) and over short periods of time (e.g., 10–12 weeks) to produce robust changes in pivotal skills [7, 15, ]. Because RIT focuses on key nonverbal social communication skills (i.e., motor imitation and joint attention) it is suitable for most toddlers with social communication delays, regardless of chronological age, language, or developmental level. Finally, RIT is available in a manualized caregiver-coaching model (Caregiver Implemented RIT or CI-RIT), which is well-suited to the priorities of family inclusion and family-centered care within the Part C system.

The Reciprocal Imitation and Social Engagement (RISE) study is a hybrid type 1 effectiveness/implementation trial that aims to test the effectiveness of CI-RIT on child and family outcomes when implemented by EI providers, as well as to explore the broader fit between CI-RIT and the Part C EI system. It is important to note that this trial was originally designed prior to the onset of the COVID-19 pandemic and was funded just months into the acute phase of the pandemic. Our team pivoted the study approach to enable a fully virtual trial capable of running uninterrupted through subsequent waves of the pandemic and beyond [27,28,29,30]. The goal of this paper is to outline the resulting study protocol for this large multi-site trial.

Objectives

The objectives of the RISE study are as follows:

• (1) To test the effectiveness of CI-RIT as delivered by community-based EI providers for improving child- and caregiver/family-level outcomes. We hypothesize that compared to Treatment as Usual (TAU), children working with CI-RIT providers will demonstrate greater improvements in motor imitation joint attention at T2 (4-months post-baseline), and language and social communication at T3 (9-months post-baseline). We also hypothesize that compared to TAU, caregivers working with CI-RIT providers will show greater improvements in contingent responsivity, RIT strategy use, parenting efficacy, and family quality of life.

• (2) To analyse the mechanisms by which CI-RIT improves outcomes. We hypothesize that changes in children’s social communication and language outcomes will be serially mediated by gains in: (a) caregiver contingent responsivity and caregiver RIT strategy use, and (b) children’s motor imitation and joint attention. We hypothesize that changes in caregiver/family outcomes will be mediated by gains in caregiver contingent responsivity, caregiver RIT strategy use, child motor imitation, and child joint attention.

• (3) To prepare for implementation at scale by identifying potential sources of practice variation to inform refinement of RIT training and development of quality assurance protocols. Triangulating evidence from video observations of EI sessions, EI provider self-reports, and qualitative interviews, we will use the Model for Adaptation Design and Impact (MADI) framework [31] to characterize the modifications to RIT that providers make (MADI domain 1), to identify potential mediating or moderating factors of these modifications (e.g., relationship to fidelity, rationale) (MADI domain 2), and to explore whether all or certain modifications influence implementation outcomes (e.g., ongoing fidelity, treatment acceptability) (MADI domain 3). This robust implementation evaluation will provide relevant information for improving the delivery of all NDBIs in the Part C system.

Methods

Overview of trial design

This hybrid type 1 effectiveness/implementation trial uses a multi-site randomized control design to address questions related to the effectiveness of CI-RIT on child and caregiver outcomes when implemented by trained EI providers, as well as factors influencing its implementation by community EI providers. The study employs a nested design, in which caregivers and children are nested within provider, providers are nested within EI agency, and EI agencies are nested within state. The exception to this is in Illinois, where the vast majority (> 80%) of EI providers have independent contracts with the State, and therefore providers are not nested within specific agencies. This multi-site study received approval from the Michigan State University Institutional Review Board (MSU IRB) (Protocol Number STUDY00001960; Clinicaltrials.gov, NCT05114538, Registration date: 10/28/2021). All other participating sites delegated IRB review to the MSU IRB via a formal reliance agreement. This trial is currently considered active. Participant recruitment and data collection began June 2021. Participant recruitment is scheduled to end by June 2025, with final data collected by March 2026. See Fig. 1 for the full CONSORT diagram.

Fig. 1

CONSORT diagram

Study setting

This collaborative trial is conducted within the Part C Early Intervention (EI) system in four states representing distinct regions across the United States: the Northeast (Boston, Massachusetts), the Northwest (Seattle, Washington), and the Midwest (East Lansing, Michigan and Chicago, Illinois). Because the Part C EI system is administered differently across these four states (see Table 1), partnerships with EI agencies and provider recruitment approaches vary by research site. In Boston, Seattle, and East Lansing the research teams have partnered with EI agencies to identify providers who might meet provider eligibility criteria. In Illinois, the research team has partnered with the Chicago-area Child and Family Connections (CFC) programs, which serve as the regional intake centers for families entering the Illinois EI System. CFC leadership has shared information about the RISE study with provider email listservs and at provider continuing education events.

Table 1 Characteristics of Part C early intervention across study states [32]

All recruitment materials and study activities are available in English and Spanish to facilitate recruitment of diverse participants. There are also accommodations for participants with literacy issues, who are assisted with survey completion by the study team.

Participants

Early intervention providers

EI providers are recruited across the four states, with a target n of 160 (approximately 40 per research site). No restrictions are placed on the discipline of intervention providers (e.g., speech therapist, social worker, occupational therapist), although we emphasize that targeting social communication goals should be within a given provider’s scope of practice. Specific inclusion criteria for EI providers are as follows:

• Provider is English speaking

• Provider has an active caseload that is expected to include at least 2 children with social communication delays, suspected ASD or diagnosed ASD by the time they start the trial

• Provider has not received prior training or certification in RIT or other NDBIs

• Provider is willing to receive training in caregiver-implemented RIT

• Provider is willing to be videotaped during EI sessions

• Provider is willing to share the study ‘permission to contact’ (PTC) form and/or give a study handout to all families on their caseload who may meet study eligibility

Caregiver/child dyads

Caregiver/child dyads are enrolled from the caseloads of participating EI providers, with a target n of 440 dyads (approximately 110 per site). EI providers are asked to share a permission to contact (PTC) form with all current families on their caseload who meet following specific criteria:

• Child has a diagnosis of ASD or displays early social communication challenges

• Child is between 16 and 33 months

• Child does not have any significant motor, vision, or hearing impairments

• Child receives ≥ 1 weekly session with the participating provider (not co-treated with another provider)

• Caregiver is present during EI sessions

• Caregiver is the biological parent or custodial guardian

• Caregiver is at least 18 years of age

• Caregiver speaks either English or Spanish

Of note, we include both children with diagnoses of ASD and those with early social communication challenges, given that many children seen in this setting have not yet had the opportunity to participate in formal autism diagnostic evaluations [33, 34].

Informed consent

All participants sign an online consent form through REDCap [35, 36], and copies of the signed consent form are sent to all participants. EI providers meet with research staff to discuss the study details and eligibility criteria. EI providers are given a link to a screening/consent form where they confirm eligibility and review and electronically sign the study consent.

Participating EI providers introduce an electronic PTC form to all families on their caseload who are likely to meet study eligibility criteria. After obtaining permission to contact, research staff send caregivers an online screener to confirm eligibility and to indicate whether they would like to learn more about the RISE study. Caregivers then meet with study staff to review the study and electronically sign the study consent form.

Interventions

Treatment as usual

The comparator for this study is treatment as usual (TAU) within the Part C Early Intervention system. EI providers are asked to continue their typical intervention approaches with study families during the active phase of the trial. While the Code of Federal Regulations mandates a policy that services delivered in EI are based on scientific research, rigorous collection of EI session data will allow for characterization of Part C EI TAU for children with, or showing early indications of, ASD [37].

Caregiver Implemented Reciprocal Imitation Teaching (CI-RIT)

RIT is a manualized, focused intervention that uses an NDBI approach to teach object and gesture imitation to young children with social communication delays. As noted in the introduction, RIT was selected as the preferred intervention for this study because of its strong evidence base, its ease of training and use, its flexibility, and its focus on a core skill area for young children with ASD [38]. One specific advantage of RIT is that, given its low intensity and playful nature, it is suitable for any child with delays in imitation and/or social communication. Moreover, because RIT focuses on skills that emerge early in development (e.g., imitation), and does not require language competency, it can be used with children at very young chronological, language, and developmental levels. In addition, it is an intervention that can be taught to caregivers as well as siblings [16, 17, 23, 25, 26].

RIT employs a systematic method for teaching a child imitation during play and daily routines. In short, the adult and child have an identical set of materials, and the interaction starts with the adult following the child’s lead, imitating his or her actions and vocalizations, and modeling simple and relevant language. Once the child is engaged (or after about 1 to 2 min of imitating the child), the adult shows a new action or gesture with the material that the child is playing with. The adult pairs this action with a descriptive verbal label. The adult shows the same action up to three times. If the child does not spontaneously imitate by the third time, the adult gently helps the child to imitate, provides praise, and then returns to following the child’s lead for another cycle.

EI providers are given access to a sequenced curriculum, referred to as Caregiver Implemented RIT (CI-RIT) which includes 7 core sessions and one optional session to support caregivers in learning and using RIT. Each session uses a systematic approach that was adapted from Ingersoll & Dvortcsak [39], grounded in best practices in caregiver coaching and adult learning theory [40], and can be delivered either in person or via telehealth. Sessions begin with an overview of session goals and structure, review of practice over the prior week, introduction of a new RIT strategy, demonstration (or video review via telehealth), caregiver practice with feedback, and reflection and planning for at home practice.

EI providers have access to several technical assistance materials including electronic and paper versions of the RIT Coach Lesson Guide, RIT Coach Lesson slides (for use in virtual visits), RIT Caregiver Handouts, and Mirror Me, an online companion tool that introduces RIT strategies using video examples and interactive exercises [25]. Caregivers are provided with paper and/or electronic versions of the RIT Caregiver Handouts and access to the Mirror Me companion website.

EI providers are free to determine when, how, and whether they use CI-RIT with study families. Should participating EI providers determine that CI-RIT is not the most appropriate intervention approach for any reason (including family preference or request), they will provide alternative care or discontinue the study intervention altogether as clinically indicated.

EI provider training and consultation in CI-RIT

EI providers in the intervention group participate in a phased training approach that involves (1) completion of the Mirror Me companion website (approximately 2 h), (2) participation in virtual active learning workshops (approximately 12 h spread across 3 days), and (3) participation in a minimum of two individual role-plays with RIT trainers (approximately 1.5 h each). RIT trainers score fidelity of implementation using the CI-RIT Fidelity Checklist during the role plays; providers are required to achieve fidelity ratings of ≥ 80% (with no individual fidelity item score of “1,” meaning the provider does not implement the strategy, or almost all attempts to use the strategy are incorrect/ineffective) across two role-play sessions before using CI-RIT with study families.

EI providers in the intervention condition receive fidelity monitoring and monthly group consultation throughout study participation. Study staff record one EI session per month for each enrolled family, for a maximum of three EI sessions. The RIT Trainer reviews these sessions to monitor fidelity, using the CI-RIT Fidelity Checklist, and sends qualitative feedback about each of these sessions to the provider. If a provider’s CI-RIT fidelity drops below 80%, an RIT Trainer provides individualized consultation and feedback until fidelity is re-established. In addition, EI providers in the intervention condition participate in monthly virtual group consultation meetings with an RIT Trainer to maintain their skill. During these 1-h meetings, providers discuss their use of RIT and engage in joint feedback and problem solving with the RIT Trainer. These sessions are also used to collect data about frequency and intensity of EI sessions for each enrolled family. Family adherence to intervention strategies learned in EI is monitored using the Frequency of Practice Survey, designed for the current trial, and collected at each recorded EI session.

Concomitant care during the trial

There are no restrictions on concomitant care or interventions for study participants. Information about services received in and external to Part C Early Intervention is collected at caregiver/child baseline (T1), four months from baseline (T2), and nine months from baseline (T3). In addition, EI providers are free to attend any additional professional development and/or training workshops during study participation.

Randomization

Randomization occurs at the level of the EI providers, who are allocated to either the CI-RIT condition or TAU condition. EI providers are randomized in groups as they enroll in the study. Within each group, EI providers are matched by site, race, ethnicity, and preferred language. For exact matches, one EI provider is randomly selected for the intervention condition and the other is assigned to waitlist control. For inexact matches, minimization procedures are used by which one EI provider is selected and assigned to intervention using a random number generator (in Microsoft Excel) and a threshold probability equal to the inverse proportion of prior allocations to intervention (e.g., if 4 of 10 prior EI providers reporting Black race had been assigned to intervention, then a new EI provider reporting Black race would have a 6 in 10 chance of being assigned to intervention). Given that most of the EI provider sample reports non-Hispanic ethnicity and White race, exact matches are common. Details of allocation procedures have been withheld from study staff who enroll participants and administer assessments.

Random assignment is overseen by the lead of the Data Management committee, who is provided with de-identified data and returns allocation decisions to a study coordinator. The study coordinator communicates allocation decisions to study staff via email and by documentation in a central database. Study staff are kept unaware of matching variables.

Study outcomes (Table 2)

Table 2 Study Outcomes

Primary effectiveness outcomes are child social communication, joint attention, motor imitation, caregiver responsivity, caregiver fidelity of RIT, and caregiver self-efficacy. Secondary effectiveness outcomes include caregiver report of child social communication, child language, and family quality of life. Implementation outcomes include CI-RIT modifications, treatment acceptability, fidelity of CI-RIT coaching, and RIT session completion. See Fig. 2 for participant timeline.

Fig. 2

Caregiver/child dyad participant timeline

Data collection and management

Plans for assessment and collection of outcomes

This study is single-blinded, meaning that the research staff who conduct the assessments (independent evaluators) are unaware of the status of participant allocation to TAU or CI-RIT. In addition, research staff who code behavioral observation data are also unaware of participant group assignment and timepoint (when feasible). Participants are reminded to avoid revealing which group they are in when interacting with blinded research staff.

All survey data are collected online using REDCap. Interview data are collected via phone or secure video conference and recorded for later transcription and analysis. Direct assessment data are collected in participant homes by trained independent evaluators. Direct assessment data are collected using a remote technology kit developed specifically for this trial to ensure high quality video and audio data, suitable for later behavioral coding, regardless of a family’s internet access or connection quality [29].

Assessors participate in a 4-week standardized training protocol which includes independent review of the manual and prior assessment recordings, participation in a 5-h group training, role play, and supervised administration of assessments (until fidelity is achieved on three administrations in a row). Assessors are each assigned a “mentor” (e.g., postdoctoral fellow) who meets with them on a consistent basis throughout the training period. Twenty percent of assessment videos are checked by the assessment team leads to guard against drift and ensure data quality.

Plans to promote participant retention and complete follow-up

This study employs several strategies for overcoming common barriers to participation retention and data collection. First, study outcome assessments are completed in families’ homes. All participants are provided with a technology kit that allows for both equitable opportunity for any family to participant in the study and collection of high-quality video and audio data for all participants [29]. Study forms can be completed electronically and accessed via email or text message (depending on participant preference); participants also have the choice to complete forms of the phone with study staff as appropriate. Participants are provided with financial incentives for engaging in all study-related activities, and receive study related “swag” (e.g., mugs, cell phone stands) throughout participation. Finally, there are many family- and provider-friendly recruitment and study materials, including a RISE Research Network website, testimonial videos, and animated explainers to help introduce the study to potential participants.

Data management

To ensure data quality and security, data are entered and stored in REDCap [35, 36]. REDCap has HIPAA compliant policies and procedures in place to protect the confidentiality and security of protected health information. In this study, REDCap is used to electronically collect and store consent forms. It is also used to send, collect, and store participant responses to surveys and other data forms. Finally, this project leverages REDCap to facilitate the tracking progress. Project administrators require standard contact log forms for each of the events within the study. These contact logs indicate when contacts were made, the purpose of the contact, as well as any outcomes (e.g. schedule or completion of events). For practical purposes, stored identified information automatically pipes into the contact log of the associated record, thereby linking identifying contact information with the record ID. This centralizes the information needed to make calls for study related purposes.

Because this project is a multi-site study, REDCap users are assigned to Data Access Groups, which are dependent on the site and position they are associated with. This action limits access to participant records to only those entered/created by their specific site/personnel from that site or that are relevant for their study role. Similarly, user rights are set so that designated staff have viewing/data entering privileges based on study role. All REDCap users, regardless of site affiliation, need to complete an end-user license agreement associated with the data management site. New users create a separate REDCap login email and password to access the study's database limiting access privileges to those with an authorized login. Based on REDCap data, a central data team produces monthly reports that are used to track study progress and to identify potential data anomalies for correction (e.g., assessment dates outside expected windows).

Because of the potential sensitivity of video recordings, coding data are stored in separate databases and identified only with research IDs, which are used to merge records with the central database for analyses. These databases support double coding of videos to establish and maintain reliability.

Confidentiality

This study has implemented several precautionary measures to prevent the loss of confidentiality, and clearly articulates these measures to participants in consent forms. All data obtained in this study is confidential to the extent permitted by law. All study staff receive confidentiality and security training, and extensive training on data management and storage, including Collaborative Institutional Training Initiative (CITI) and Good Clinical Practices training. A certificate of confidentiality is in place for this study to further ensure participant confidentiality.

To safeguard the confidentiality of participants, as noted above, data collected during the study are identified with a participant ID number; the only link between the ID number and the name is kept in a password-protected database. Names are not written on video or audio-recordings, questionnaires or other written or digital materials. All data are stored in locked file cabinets and password-protected computers/networks and is only accessible to study staff on a “need to know basis.” In publications, reports, or professional presentations, any descriptions or discussion of intervention sessions will be modified to ensure that the identities of individual participants are altered sufficiently to render them unidentifiable to anyone reading or hearing the publications/reports/presentations, unless there is written consent indicting otherwise.

Sample size and power

Target enrollment is 160 for EI providers and 440 for for child/caregiver dyads across all sites. The study aims to detect at least a moderate effect (Cohen’s d = 0.40), which is both clinically important and consistent with previous work. All estimates assume 80% power and a two-sided alpha of 0.05. Given procedures as planned and assuming approximately 15% attrition for EI providers, we expect 133 EI providers to complete RIT with 400 families. Further, we expect an average of 7 providers per EI agency (ICC estimated at 0.05), and we estimate that each provider would treat an average of 3 families (ICC conservatively estimated at 0.15) and that autocorrelations would equal 0.25. Following published guidelines recommended by NIH, we estimate that these factors will yield a design effect (DE) of 1.95, for an equivalent sample size of 205 children (if clustering were absent). Based on these estimates, we expect to have sufficient power to detect a Cohen’s d between experimental groups of 0.39 (equivalent to approximately 6 standard score points on the Vineland-3 [41]). Notably, studies published shortly before this study was planned found substantially larger effects for proximal outcomes [7, 15], including a meta-analysis which found that similar interventions typically yield larger effects for several primary outcomes, such as expressive and receptive language, cognition, and engagement (e.g., average Hedges’ g = 0.48 for global IQ) [2].

Regarding heterogeneity of treatment effects (HTE) analyses, we estimate power for a regression model with main effects for experimental group, a moderator variable, and an interaction term. Based on the DE and the equivalent sample size noted above, we have followed published guidelines to estimate that the minimum detectable change in R2 (estimated as the square of a partial correlation coefficient) associated with the interaction term is 0.06. Two prior studies estimated effects for interactions between the same moderator and similar outcomes that were substantially larger than this effect [43, 54]. Finally, based on the results of large simulation studies [55], we estimate that our design has 80% power to detect mediation effects assuming at least small-to-medium associations between variables.

Statistical analyses

Effectiveness outcomes

We will use R (4.4.1) (e.g., geepack, lme4, and lavaan packages), Mplus (V.8), and Stata (V.18) to clean and analyze data. Formal hypothesis testing will be two-sided with a nominal type I error rate of 0.05. Results will be reported according to CONSORT guidelines. If data distribution assumptions are not met, we will consider nonparametric procedures or transformation of data. If randomization fails to produce balanced groups, we will add to our models potentially confounding variables for which distributions are unequal across groups​. Descriptive statistics will be used to examine sociodemographic and clinical characteristics of EI provider, caregiver, and child participants. Correlational analyses will explore associations between demographic factors and variables of interest.

To assess the effectiveness of CI-RIT within the Part C EI setting (Aim 1), we will use an intent to treat model to examine group-by-time effects for primary and secondary outcomes. To account for clustering, we will use an appropriate modeling approach (e.g., generalized estimating equations or mixed linear models). We will use a multilevel modeling framework to assess whether the effect of CI-RIT on child language and social communication is serially mediated by a) caregiver responsivity and CI-RIT strategy use, and b) child proximal skills (imitation and joint attention) (Aim 2) [56]. We will create a series of nested models based on our conceptual model in which we will systematically vary model parameters and constraints to test the effect of each potential mediator. Nested models will be compared using difference tests and other standard indices (e.g., adjusted Bayesian Information Criterion, the comparative fit index, the Tucker-Lewis index, and the root mean square error of approximation).

Implementation outcomes

To explore provider modifications to CI-RIT (Aim 3), we will use a convergent mixed-methods design. Data will be integrated in multiple ways, at both the data analysis and the data interpretation phase. Our qualitative study sample is a subset of the participants providing quantitative data (i.e., connected integration). In addition, data sources will initially be coded/analyzed separately and combined using merged integration to gain a more nuanced understanding of the scope of provider modifications. Data interpretation will involve using joint displays to present mixed methods results in an integrative way. This analysis prioritizes quantitative evidence, while also considering qualitative findings.

Additional analyses

We will examine each primary measure for scalar measurement invariance, both across timepoint and across subgroups (e.g., language spoken, race/ethnicity, and child gender), using differential item functioning testing with multiple indicators, multiple causes models.

We will analyse our primary measures independently. In addition, we will consider aggregating primary and secondary measures depending on the correlations among the variables within domains of interest. Specifically, we will consider aggregating when variables are correlated at a minimum threshold of 0.4 [57].

Missing data

Because we are using an intent-to-treat model, we will account for attrition and other missing data in our analyses. We will estimate associations between missingness and observed variables and test whether observed data are consistent with a missing completely at random or missing at random hypothesis. If missing at random is plausible, we will proceed with multiple imputation or the use of a maximum likelihood estimator.

Data sharing

This trial will submit data on standardized and widely used measures to the National Database for Autism Research (NDAR) and publish statistical code on a suitable open-source repository.

Study oversight and monitoring

The Trial Steering Committee (Leadership Team) includes the Principal Investigators, project manager, and select Co-Investigators who meet on a weekly basis to review study operations, protocol modifications, data collection, data quality, and potential risks to participants.

This trial also includes several other committees that meet weekly and contribute to the day-to-day operations of the study [28]. The Coordinator/Liaison committee oversees recruitment and cross-site coordination. The Data Management committee is responsible for overseeing development and maintenance of REDCap databases, data-related processes and timelines, data quality assurance procedures, and data pulls. The Family Assessment committee is responsible for guiding families through their participation in the study and for data collection at the child and caregiver levels. The Provider Training committee is responsible for guiding providers through their participation in the study, including training and consultation in CI-RIT, and for data collection at the level of the provider. The Spanish Language committee oversees translation of materials from English to Spanish and supports trial participation and data collection for Spanish-speaking participants. The Implementation committee is responsible for overseeing all aspects of the study implementation aim.

Data and safety monitoring board and plan

In addition to the Trial Steering Committee, an independent Data and Safety Monitoring Board (DSMB) consisting of four external scientists with expertise in clinical trials and autism has been convened for this study. Two of the four members have personal connections to children on the autism spectrum. The Trial Steering committee presents to the external DSMB for consultation regarding ongoing data collection procedures, data quality, timeliness, participation retention, participant risk versus benefit, performance of intervention sites, and other factors that can affect study outcomes.

The project has also established a Data and Safety Monitoring Plan in which unanticipated problems or adverse events, if identified, are documented through a centralized database system, investigated, followed up under the direction of the respective site Principal Investigators, and reported to the MSU IRB. Such events are also presented to the DSMB.

All modifications to the study protocol are formally documented and submitted to the MSU IRB for approval.

Dissemination plans

We will disseminate the knowledge gained from this study through several channels. First, we have registered this trial on ClinicalTrials.gov (NCT05114538, Registration date: 10/28/2021) and will report results on this platform within the recommended timeframe. In addition, we will present our findings at several venues, including scientific conferences, as well as national and state-wide professional development conferences for EI providers and Part C policy makers. We plan to publish our findings in both scholarly and lay publications, and to send participants a summary of findings, to reach a range of key partners. Importantly, data reporting from this trial will adhere to CONSORT guidelines.

Discussion

The current study represents a critical effort to transport an evidence-based NDBI into a national service delivery setting, the Part C Early Intervention system. To date, it is one of the largest randomized control trials of an NDBI in any community setting, allowing this study to address critical questions of training and intervention effectiveness, and develop a more nuanced understanding of critical factors necessary to support CI-RIT implementation at scale.

It is critical to note that the COVID-19 pandemic completely disrupted the already stressed Part C System, leading to suspension of clinical services, significant service delays, and missed Early Intervention evaluations for many families [58]. As services resumed, EI providers were faced with unprecedented changes to service delivery (e.g., conducting services remotely, supporting families in using technology, using different evaluation and progress monitoring tools, billing for telehealth), often without formal guidance for how to navigate these changes [58]. For example, 85% of EI providers in Illinois reported significant disruptions to their clinical practice, with the number of sessions and number of children on providers’ caseloads decreasing significantly in the first year of the pandemic [59].

It is within this broader context that the current trial has had to creatively navigate agency, provider, and family recruitment to meet the study aims while also supporting the diverse and substantial needs of our partners and participants. Unplanned, but fruitful, efforts to recruit and support participants included early listening sessions with EI leadership and providers, establishing local EI leader advisory boards, offering non-RISE study related professional development opportunities (e.g., lectures on relevant clinical topics), meeting individually with enrolled EI providers to review caseloads on a monthly basis, offering “on site” and flexible scheduling of assessments for families, and developing local resource banks to facilitate family connections with relevant community supports. As families of young children, as well as the broader EI system, continue to grapple with lasting impacts of the pandemic, our study team remains committed to identifying practical and creative ways to support participants in enrolling and engaging with the RISE study.

Given the goals of the trial and the context in which it is taking place, the RISE study has emphasized pragmatism through decisions such as establishing broad inclusion criteria, including standard EI services as our treatment as usual control condition, allowing for flexible CI-RIT delivery, and employing an intent to treat model for statistical analyses. While these pragmatic decisions could be viewed as potential limitations, we believe that this approach is required to demonstrate the real-world effectiveness of CI-RIT within the EI system. If proven effective in this trial, CI-RIT is poised for scaling across the national Part C EI system to address the pressing need for interventions supporting pivotal skills and family functioning for toddlers showing early social communication challenges.