This paper is in the following e-collection/theme issue:

Machine Learning (1501) Autism Spectrum Disorder (ASD) (112) Artificial Intelligence (1421)

Published on in Vol 9, No 6 (2021): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/27793, first published .
Unsupervised Machine Learning for Identifying Challenging Behavior Profiles to Explore Cluster-Based Treatment Efficacy in Children With Autism Spectrum Disorder: Retrospective Data Analysis Study

Unsupervised Machine Learning for Identifying Challenging Behavior Profiles to Explore Cluster-Based Treatment Efficacy in Children With Autism Spectrum Disorder: Retrospective Data Analysis Study

Unsupervised Machine Learning for Identifying Challenging Behavior Profiles to Explore Cluster-Based Treatment Efficacy in Children With Autism Spectrum Disorder: Retrospective Data Analysis Study

Authors of this article:

Julie Gardner-Hoag1 Author Orcid Image ;   Marlena Novack2 Author Orcid Image ;   Chelsea Parlett-Pelleriti3 Author Orcid Image ;   Elizabeth Stevens3 Author Orcid Image ;   Dennis Dixon2 Author Orcid Image ;   Erik Linstead3 Author Orcid Image
Article Authors Cited by (5) Tweetations (10) Metrics

    Original Paper

    1Schmid College of Science and Technology, Chapman University, Orange, CA, United States

    2Center for Autism and Related Disorders, Woodland Hills, CA, United States

    3Fowler School of Engineering, Chapman University, Orange, CA, United States

    *these authors contributed equally

    Corresponding Author:

    Erik Linstead, BSc, MSc, PhD

    Fowler School of Engineering

    Chapman University

    One University Drive

    Orange, CA, 92866

    United States

    Phone: 1 714 289 3159

    Email: linstead@chapman.edu


    Background: Challenging behaviors are prevalent among individuals with autism spectrum disorder; however, research exploring the impact of challenging behaviors on treatment response is lacking.

    Objective: The purpose of this study was to identify types of autism spectrum disorder based on engagement in different challenging behaviors and evaluate differences in treatment response between groups.

    Methods: Retrospective data on challenging behaviors and treatment progress for 854 children with autism spectrum disorder were analyzed. Participants were clustered based on 8 observed challenging behaviors using k means, and multiple linear regression was performed to test interactions between skill mastery and treatment hours, cluster assignment, and gender.

    Results: Seven clusters were identified, which demonstrated a single dominant challenging behavior. For some clusters, significant differences in treatment response were found. Specifically, a cluster characterized by low levels of stereotypy was found to have significantly higher levels of skill mastery than clusters characterized by self-injurious behavior and aggression (P<.003).

    Conclusions: These findings have implications on the treatment of individuals with autism spectrum disorder. Self-injurious behavior and aggression were prevalent among participants with the worst treatment response, thus interventions targeting these challenging behaviors may be worth prioritizing. Furthermore, the use of unsupervised machine learning models to identify types of autism spectrum disorder shows promise.

    JMIR Med Inform 2021;9(6):e27793

    doi:10.2196/27793

    Keywords

    autism spectrum disorder; challenging behaviors; unsupervised machine learning; subtypes; treatment response; autism; treatment; behavior; machine learning; impact; efficacy; disorder; engagement; retrospective 


    Autism spectrum disorder is a neurodevelopmental disorder characterized by deficits in social communication and social interaction, as well as the presence of restricted, repetitive patterns of behavior, interests, and activities [1]. With the exception of restricted, repetitive behaviors (eg, stereotypy, perseveration), challenging behaviors are not classified as a core symptom of autism spectrum disorder; however, these behaviors are prevalent among individuals with autism spectrum disorder. As many as 94% of children with autism spectrum disorder engage in some type of challenging behavior, often including stereotypy (eg, self-stimulatory or persistent repetitive motor or vocal behavior), aggression, tantrums, and self-injurious behavior [2,3]. Challenging behaviors may pose risk of injury to the individual or others and may inhibit learning opportunities and social interactions [4]. Furthermore, challenging behaviors may negatively impact family functioning and contribute to caregiver stress [5,6].

    Various risk factors for engagement in challenging behaviors have been investigated in individuals with autism spectrum disorder. Symptom severity has been found to predict challenging behaviors, with greater symptom severity associated with engagement in higher numbers of challenging behaviors at stronger intensities [2,3]. Intellectual functioning has also been linked to challenging behaviors in individuals with autism spectrum disorder, with greater deficits in intellectual functioning predicting greater frequencies of stereotypy [7,8], aggression [8], and self-injurious behavior [8,9]. In addition, deficits in adaptive skills [10,11] and expressive language skills [11] have been associated with engagement in challenging behaviors in individuals with autism spectrum disorder, but studies [8-12] that investigated the relationship between gender and challenging behaviors found no significant differences in engagement in challenging behaviors between boys and girls with autism spectrum disorder.

    Applied behavior analysis interventions, which involve the application of principles and procedures of learning and motivation to alter behavior [13,14], may be used to reduce challenging behaviors and increase appropriate behaviors in individuals with autism spectrum disorder. Specific challenging behaviors that are commonly addressed in treatment include stereotypy, noncompliance, and aggression [15]. Outcome studies for children with autism spectrum disorder have not often included challenging behaviors as an outcome measure [4,16]. Several group design studies [17-19] have found evidence to support the use of caregiver training to manage challenging behaviors. Furthermore, there is an abundance of single-individual research evaluating the effectiveness of behavioral interventions for challenging behaviors in individuals with autism spectrum disorder, and reviews of this research have found behavioral interventions, particularly those implementing pretreatment functional assessments, to be effective in reducing challenging behaviors [20-22].

    Applied behavior analysis–based therapy is considered to be well-established for the treatment of autism spectrum disorder [23,24]. While ample research demonstrates the effectiveness of applied behavior analysis–based treatment [25,26] research also reveals variability in individual response to treatment [27,28]. Treatment-related variables including greater treatment intensity [27,29-32], longer treatment duration [30-32], and greater total intervention time [33,34] have been linked to superior treatment outcomes. Furthermore, many patient-related variables have been associated with greater treatment gains. These include younger age [29,32,34-38], lower autism spectrum disorder symptom severity [35,36,38,39], and greater intellectual functioning [27,36,38-45].

    Research evaluating the impact of challenging behaviors on treatment response in individuals with autism spectrum disorder is limited. Eikeseth and colleagues [46] investigated whether challenging behaviors, among other intake measures, were associated with treatment outcomes for adaptive behavior and autism spectrum disorder symptoms in children with autism spectrum disorder; however, challenging behaviors were not found to be a predictor of treatment outcome. Conversely, Remington and colleagues [39] found that higher rates of challenging behaviors at intake were associated with superior response to treatment and suggested that their counterintuitive findings may possibly be attributed to the sensitivity of the measure used to assess challenging behaviors. Given the prevalence of challenging behaviors among individuals with autism spectrum disorder, additional research is needed to investigate the impact of these behaviors on treatment response.

    To account for the heterogeneity observed across individuals with autism spectrum disorder, researchers have investigated types of autism spectrum disorder [47]. Preliminary research has found behavioral types of autism spectrum disorder to have differences in gene expression [48-50], developmental trajectory [51-54], and treatment response [55]. In a recent study, Stevens and colleagues [55] used an unsupervised machine learning model to identify behavioral types of autism spectrum disorder and evaluate differences in treatment response across types. Participants included 2400 children with autism spectrum disorder. Data from a comprehensive assessment of skill deficits and treatment progress data were analyzed. A total of 16 autism spectrum disorder groups were identified using a Gaussian mixture model. Using a linear regression model, relationships between treatment hours and skill mastery were found to be strong within groups, accounting for 64% to 75% of variance. These findings are a preliminary step toward advancing targeted treatments and improving outcomes for individuals with autism spectrum disorder based on type membership.

    Autism spectrum disorder types may also be identified based upon profiles of challenging behavior. Stevens and colleagues [56] conducted an analysis of challenging behaviors in a large sample of children with autism spectrum disorder (n=2116). Using k-means clustering, 8 diverse profiles, in which a single dominant challenging behavior was apparent, were identified. Furthermore, gender differences were observed when cluster analyses were performed separately for male and female participants. While all of the male clusters were found to exhibit a single dominant challenging behavior, 2 of the female clusters indicated equal engagement in 2 dominant challenging behaviors. These findings suggest that gender may play a role in the presentation of challenging behaviors in individuals with autism spectrum disorder. Further investigations into autism spectrum disorder types based on challenging behaviors are warranted.

    The study of challenging behaviors across types of autism spectrum disorder may help explain some of the variation observed in treatment outcomes across individuals with autism spectrum disorder and may advance efforts to develop targeted treatments to maximize outcomes. Preliminary evidence indicates there are autism spectrum disorder types based on challenging behaviors; however, little is known about how challenging behaviors impact treatment response. The purpose of this study was to identify types of autism spectrum disorder based on engagement in different challenging behaviors and evaluate differences in treatment response between groups and across gender.


    Data Set

    Deidentified retrospective treatment data for a large sample of children with autism spectrum disorder were used in this study. Data on the frequency of challenging behaviors and treatment progress were obtained from the Skills system software (Skills Global LLC [57]). Skills includes a thorough assessment of skill deficits with demonstrated reliability [58] and validity [59], a comprehensive curriculum to build individualized treatment plans, and tracking capabilities for challenging behaviors and ongoing treatment progress. In addition to Skills data, operational data on treatment hours were used in this study.

    Participants included children with autism spectrum disorder who were receiving applied behavior analysis treatment from a community-based provider. A total of 2116 clinical records were reviewed based on the following inclusion criteria: (1) were between the ages 18 months and 12 years old; (2) had a diagnosis of autism spectrum disorder, autistic disorder, pervasive developmental disorder–not otherwise specified, or Asperger syndrome by an independent licensed clinician (eg, psychologist, pediatrician, etc); (3) received at least 20 hours of treatment per month; and (4) had at least 1 month of continuous services; (5) demonstrated repeated instances of challenging behavior as documented in their treatment history; and (6) had available treatment response data over the course of treatment. Parameters with respect to age were set based on the age range predominately represented in the data set to avoid potential outliers that may have affected the cluster analysis. Likewise, parameters regarding treatment intensity and duration were established so that each participant had adequate treatment response data to include in the analysis. After applying these criteria, a sample of 854 participants were included. Of the participants, 706 were male and 148 were female. The average age of participants was 7.59 (SD 2.17) years old, ranging from 2.74 years to 12 years. Participants resided in the states of Arizona, California, Colorado, Illinois, Louisiana, New York, Texas, and Virginia. The data used for this study were collected during a 36-month period (January 1, 2014 through December 31, 2016).

    Measures

    Data on engagement in challenging behaviors were used to identify potential clusters. While the classification of challenging behaviors is subjective in nature, there is agreement among the literature regarding operational definitions for common topographies of challenging behaviors exhibited by individuals with autism spectrum disorder [4]. While this may not be exhaustive, data were examined for the following topographies of challenging behaviors: aggression (eg, hitting, kicking), disruption (eg, interrupting, yelling), elopement (eg, wandering, bolting), noncompliance (eg, defiant behavior, refusing), obsessive behavior (eg, repeatedly talking about the same topic, preservation), self-injurious behavior (eg, head banging, hand biting), stereotypy (eg, hand flapping, toe walking, vocal stereotypy), and tantrums (eg, crying, falling). Skills is implemented as a relational database, which allows behavior interventionists to record observations in real time during a therapy session using an iPad and the corresponding Skills app. In the case of challenging behaviors, when such a behavior is observed, the therapist marks the type of behavior and provides a textual description of its context. This information is then timestamped and then stored in the underlying relational database. Aggregation of challenging behavior data for each patient can then be easily achieved using a simple database query (SQL format). An extra validation step was taken to compare identified challenging behaviors to the textual description provided by the behavior interventionist to ensure no challenging behavior observations were misidentified.

    Data on mastered learning objectives were used to evaluate treatment response. Mastery criteria for learning objectives were determined by the patient’s clinician and individualized to the patient. Typically, mastery was defined as 80% accuracy or greater for a minimum of 2 treatment sessions across 2 days.

    Treatment

    Participants received individualized applied behavior analysis–based treatment. Treatment comprehensively targeted deficits across developmental domains, including language, social, adaptive, cognitive, executive function, academic, play, and motor skills. Services were provided in the participant’s home, clinic, school, community, or a variety of settings. Treatment was provided according to the Center for Autism and Related Disorders model [60].

    Participants’ treatment programs addressed skill acquisition and targeted the reduction of challenging behaviors. Interventions for challenging behaviors varied based on the target behavior’s topography and function (determined using functional assessment). Possible interventions implemented by a participant’s clinician included: antecedent-based interventions (ie, manipulations to the environment to reduce the target behavior) such as noncontingent reinforcement, demand fading, task modification, and choice; replacement behavior interventions including functional communication training, differential reinforcement of alternative behavior, and differential reinforcement of incompatible behavior; and consequence-based interventions (ie, manipulations to the events following the target behavior to reduce the likelihood of its reoccurrence) such as extinction, differential reinforcement of other behavior, differential reinforcement of low rate behavior, and response interruption and redirection.

    Data Analysis

    Clustering

    This analysis expanded on the work of Stevens and colleagues [56] to explore differences in treatment response across identified challenging behavior clusters in individuals with autism spectrum disorder. Patients were clustered based on relative frequency of 8 challenging behaviors (aggression, disruption, elopement, noncompliance, obsessive behavior, self-injurious behavior, stereotypy, and tantrums) using a k-means machine learning algorithm. This was achieved by creating an 8-dimensional feature vector for each patient. Relative frequency was calculated by finding the proportion of all challenging behaviors for each of the 8 categories for each patient. Duration and severity of the challenging behaviors were not taken into consideration for this value. Each vector element corresponded to the relative frequency of a specific challenging behavior observed for that patient. The 8-dimensional vectors were fed directly to the clustering algorithm without the use of feature selection because the dimensionality of the data was relatively small, and it was important to preserve each of the challenging behaviors in the final cluster model. Once clusters were identified using the k-means algorithm, multiple linear regression was performed to evaluate interactions between cluster assignment, treatment response, and gender.

    The goal of clustering is to find latent groups, or clusters, in data. Patients within the same cluster will have more similar challenging behaviors profiles than patients in different clusters [61]. The k-means methods was selected for clustering because it is computationally efficient, easily implemented, and is a widely used prototype-based clustering algorithm, wherein each cluster is represented by a prototype. This prototype can either be the centroid of data points with similar continuous features or the medoid in the case of categorical features. This data set involved continuous features; therefore, each cluster had a centroid.

    The k-means algorithm was implemented with 5 steps. (1) The best value of k (ie, the number of clusters) was identified by incrementally testing values between 2 and 20. (2) For each of these values, the algorithm picked k sample points from the data at random, which are the initial centroids (c1, c2, ..., ck). (3) Each 8-dimensional data point di was assigned to the nearest centroid ck using Euclidean distance to measure the distance between the point and the centroid. (4) The algorithm recalculated the centroids by taking the mean value (for each behavior) from all the data points currently in the cluster. (5) The algorithm repeated steps 3 and 4 until the cluster assignments did not change or a maximum number of iterations was reached.

    To find the distance between the data points and the centroids in the data set, squared Euclidean distance was used. Similarity between data points is defined as the opposite of distance. A commonly used metric for finding the distance between data points x and y in m-dimensional space is the squared Euclidean distance.

    Once similarities are measured, clustering becomes an optimization problem. An iterative approach was used to minimize the within-cluster sum of squared errors or cluster inertia. Once these errors were calculated, a graph of the errors were examined using the elbow method to find the best value for k. The elbow method involves examining the plot (ie, the arm) to determine the point at which diminishing returns are observed (ie, the elbow). As k increases, the sum of squared errors gets smaller. When k is equal to the number of points in the data set, the sum of squared errors is 0 and every point is its own cluster. Choosing k to correspond to the elbow in the graph thus provides an effective measure by which to prevent overfitting. The chosen k indicates the optimal number of clusters that are both cohesive and separate.

    Linear Regression

    A multiple linear regression model was used to evaluate the relationships between the target variable (skill mastery) and explanatory variables (treatment intensity, cluster assignment, and gender).

    In univariate linear regression, the relationship between a single explanatory variable x and a response or target variable y is modeled. The equation used for linear models with only 1 predictor variable is defined as yi=β0+ β1xi+εi, where the weight β0 represents the y-axis intercept and β1 is the coefficient of the explanatory variable. In simple linear regression, the goal is to find the weights of the equation to explain the relationship between the explanatory variable and the target variable. From this, the responses of new data points that were not part of the observed data may be predicted and coefficients of the model may be interpreted. The simple linear regression equation may be generalized to produce an equation for multiple linear regression that involves multiple explanatory variables.

    Linear regression works by taking the explanatory variables and the response variable, and fitting a straight hyperplane to the data that minimizes the distance between an observed point and the fitted model. The explanatory variables were treatment intensity, cluster assignment, and gender, and the response variable was skill mastery.

    An efficient way to quantitatively measure a model's performance is the mean squared error, which measures the average squared error between the model’s prediction and the actual values. Mean squared error may be used to compare different regression models with the same outcome.

    R2 (another measure of model fit) is bounded between 0 and 1, with 1 indicating a perfect relationship between x and y and mean squared error is equal to 0. R allows for the specification of interaction terms in regression formulas. An interaction occurs when the product of 2 predictor variables is also a significant predictor [62]. In this study, there were 3 explanatory variables (treatment intensity, cluster assignment, and gender), and all interactions were included in the model.

    Tukey Posthoc

    Results from the regression model indicated that there was a significant difference between treatment hours, cluster assignment, and the interaction between cluster assignment and gender. Posthoc analysis was conducted to determine which pairs of clusters significantly differed. The Tukey honestly significant difference method was used to correct for multiple comparisons.

    The Tukey posthoc test assesses all the pairwise comparisons using the Tukey honestly significant difference formula

    where Mi – Mj is the difference between the pairs of means, MSw is the mean square within, and n is the number of clusters.


    Clustering

    Figure 1 shows within-group sum of squared errors for patients. The optimal value of k (the number of distinct challenging behavior profiles) was found to be 7, confirmed both by the elbow method and by silhouette score (the highest indicates most cohesive and separate). Each cluster corresponds to a phenotype and can be quantitatively represented with its centroid (the mean relative frequency for each of the 8 challenging behaviors for patients in that cluster). The dimensionality of each centroid is identical to the input feature space, which is preserved during the clustering process.

    ‎
    Figure 1. Within-cluster sum of squared errors for all patients, both male and female. The elbow method indicates that the best value of k is 7, meaning there are 7 clusters.
    View this figure

    Seven phenotypes of autism spectrum disorder, most of which demonstrated 1 dominant challenging behavior, were identified based on average frequency (centroid) of 8 challenging behaviors (ie, aggression, disruption, elopement, noncompliance, obsessive behavior, self-injurious behavior, stereotypy, and tantrums) calculated for each cluster (Table 1). It is important to reiterate that the machine learning process is unsupervised. The phenotypes are identified by the algorithm without the need for human labels, which are required for supervised learning (classification).

    Table 1. Breakdown of identified clusters.
    ClusterDominant challenging behaviorBoys, nGirls, nAll, n
    1Tantrums601474
    2Self-injurious behavior79887
    3Elopement781896
    4Stereotypy (low rate)17037207
    5Noncompliance8726113
    6Aggression11326139
    7Stereotypy (high rate)11919138

    The radar graphs shown in Figure 2 and Figure 3 provide visual representations of each phenotype’s engagement in the 8 challenging behaviors. The radar charts in Figure 3 were scaled from 0 to the average frequency of the dominant challenging behavior. For example, Cluster 1 was scaled from 0 to 0.6, to which tantrums extend. Cluster 4 was scaled from 0 to 0.4, to which stereotypy extends. It is worth noting that Cluster 4 and Cluster 7 both have stereotypy as their dominant challenging behavior, but their frequencies were different. Cluster 4 was found to engage in stereotypy at a lower rate than Cluster 7.

    ‎
    Figure 2. Radar graphs depicting engagement in challenging behaviors across clusters.
    View this figure
    ‎
    Figure 3. Radar graphs showing the dominant challenging behavior for each cluster. Note that the maximum varies between the clusters, particularly Cluster 4 and Cluster 7, in which patients demonstrate the same dominant challenging behavior.
    View this figure

    Linear Regression

    The R2 value was found to be 0.67. The value for R2 is the fraction of the variance of exemplar mastery that is explained by the model. Thus, the model explained 67% of the variance of exemplar mastery. The model was significantly predictive of mastery (F27,826=61.05, P<.001).

    Figure 4 shows the regression lines for male and female patients in each of the different clusters. The mean squared error for each cluster is shown in Table 2.

    ‎
    Figure 4. The line of best fit for each gender and cluster.
    View this figure
    Table 2. Mean squared error comparison across clusters.
    ClusterDominant challenging behaviorMean squared error
    1Tantrums0.30
    2Self-injurious behavior0.34
    3Elopement0.23
    4Stereotypy (low rate)0.37
    5Noncompliance0.30
    6Aggression0.24
    7Stereotypy (high rate)0.29

    Box plots (Figure 5) for each of the 7 clusters depicts differences across clusters with respect to exemplar mastery and show the range of the exemplars mastered for each cluster, where the whiskers represent the minimum and maximum values (or 1.5 × the interquartile range, if outliers were present).

    ‎
    Figure 5. Box plots for each cluster. The box plots show the range of the exemplars mastered for each cluster, where the whiskers represent the minimum and maximum values. The line across each box is the median. The top of the box represents the third quartile. The bottom of the box represents the first quartile. Any points on the graph represent outliers in the clusters.
    View this figure

    Increased treatment hours were associated with a significant increase in mastery (P<.001), and there were significant differences in mastery between clusters (P=.002); however, the interaction between treatment hours and cluster assignment was not significant (P=.28). Gender was nonsignificant (P=.051); however, the interaction between gender and cluster assignment did have a significant relationship with exemplar mastery (P=.018) (Table 3).

    Table 3. Explanatory variables.
    VariableP value
    Therapy hours<.001
    Gender.051
    Cluster.002
    Therapy hours × gender.67
    Therapy hours × cluster.28
    Gender and cluster.02
    Therapy hours, gender, and cluster.63

    Table 4 shows the averages for treatment hours and exemplars mastered for male, female, and combined clusters. Cluster 4 had the highest and Cluster 3 had the second-highest average number of exemplars mastered. Cluster 2 had the lowest average number of exemplars mastered.

    Table 4. Average treatment hours and exemplars mastered across male, female, and combined gender clusters.
    Cluster and dominant challenging behaviorTreatment hoursExemplars mastered
    Male


    		Tantrums6.615.33

    		Self-injurious behavior6.645.05

    		Elopement6.715.41

    		Stereotypy (low rate)6.925.50

    		Noncompliance6.475.26

    		Aggression6.545.27

    		Stereotypy (high rate)6.725.41
    Female


    		Tantrums6.495.37

    		Self-injurious behavior6.725.43

    		Elopement6.665.50

    		Stereotypy (low rate)6.905.80

    		Noncompliance6.575.39

    		Aggression6.585.15

    		Stereotypy (high rate)6.665.13
    Combined


    		Tantrums6.585.34

    		Self-injurious behavior6.655.08

    		Elopement6.705.43

    		Stereotypy (low rate)6.915.55

    		Noncompliance6.505.29

    		Aggression6.555.24

    		Stereotypy (high rate)6.715.38

    Tukey Posthoc

    Significant differences were found between Cluster 4 (low frequency stereotypy and moderate frequencies of other challenging behaviors) and Cluster 2 (self-injurious behavior) (P=.003) and between Cluster 4 and Cluster 6 (aggression) (P=.047). Overall, Cluster 4 had the highest rate of mastery while Cluster 2 had the lowest (Table 4); there was a significant difference between the clusters.

    The interaction between gender and cluster assignment is depicted in Figure 6. Girls (P=.005) and boys (P=.03) in Cluster 4 mastered significantly more exemplars than the boys in Cluster 2. There was no significant difference between the girls in Cluster 2 and the girls and boys in Cluster 4. There were also no significant differences within clusters between genders (P=.003).

    ‎
    Figure 6. Box plots for each cluster and gender. The whiskers represent the minimum and maximum values. The line across each box is the median. The top of the box represents the third quartile. The bottom of the box represents the first quartile. Any points on the graph represent outliers in the clusters.
    View this figure

    The purpose of this study was to identify types of autism spectrum disorder based on engagement in 8 challenging behaviors (ie, aggression, disruption, elopement, noncompliance, obsessive behavior, self-injurious behavior, stereotypy, and tantrums) as well as examine group and gender differences in treatment response; k-means clustering analyses performed on male, female, and blended samples revealed 7 unique clusters. These findings differ from those of Stevens and colleagues [56], in which 8 male and female clusters were identified based on engagement in challenging behaviors. Similar to those found by Stevens and colleagues [56], the clusters in our study were found to have a single dominant challenging behavior. Only 2 of the measured challenging behaviors (ie, disruption and obsessive behaviors) did not appear as a dominant challenging behavior across the identified clusters. Furthermore, relatively low rates of disruption and obsessive behaviors were also observed across all the clusters. Cluster 1 had tantrums as its dominant challenging behavior, Cluster 2 had self-injurious behavior as its dominant challenging behavior, Cluster 3 had elopement as its dominant challenging behavior, Cluster 4 had stereotypy (low rate compared to cluster 7) as its dominant challenging behavior, Cluster 5 had noncompliance as its dominant challenging behavior, Cluster 6 had aggression as its dominant challenging behavior, and Cluster 7 had stereotypy (at a higher rate than Cluster 4) as its dominant challenging behavior. Neither obsessive behavior nor disruption appeared as a dominant behavior in any of the clusters.

    To explore the relationship between skill mastery, treatment hours, cluster assignment, and gender, multiple linear regression was performed. Interactions between all the explanatory variables were also evaluated. In line with previous findings [30,31], the relationship between skill acquisition and treatment hours was found to be significant in our study (P<.001).

    In addition to treatment hours, cluster assignment was found to be significantly related to skill mastery (P=.002). Results from the Tukey posthoc test revealed that Cluster 4, characterized by the dominant behavior stereotypy with moderate frequencies of other challenging behaviors, was found to have significantly stronger levels of skill mastery than both Cluster 2, characterized by self-injurious behavior, and Cluster 6, characterized by aggression (P=.003). These findings suggest that treatment response varies across individuals with autism spectrum disorder that engage in different topographies of challenging behaviors. In particular, participants who engaged in self-injurious behavior and aggression were found to have poorer response to treatment compared to those with low levels of stereotypy. It is likely that prioritizing the treatment of self-injurious behavior and aggression using appropriate behavior interventions based on the identified function of the behavior [63] will result in better long-term treatment outcomes for these individuals.

    The only interaction between explanatory variables that was found to be significant in this study was cluster assignment and gender (P=.018). No significant gender differences were found, with respect to skill acquisition, within the same cluster. That is, boys and girls in the same cluster were found to have similar rates of skill acquisition (Table 4). Significant gender differences were found across clusters, however. Specifically, both girls and boys in Cluster 4 (stereotypy) displayed stronger rates of skill mastery than boys in Cluster 2 (self-injurious behavior); however, no significant differences were found between boys and girls in Cluster 4 and girls in Cluster 2. In previous research, gender was found to be a risk factor for the occurrence of challenging behaviors in individuals with autism spectrum disorder [8-12]. While the role of gender is unclear, this finding provides further support for the significant differences in treatment response across clusters, particularly for Cluster 4 and Cluster 2.

    This study has several limitations that are important to consider. As a retrospective study, the analysis was limited to the existing data in the data set. Data on race and ethnicity were not available in the data set; therefore, representation across those demographics and any potential disparities in this sample are unknown. Furthermore, variables such as autism spectrum disorder symptom severity and IQ were not measured. Both symptom severity and IQ have been found to be related to engagement in challenging behaviors [2,3,7-9] as well as related to treatment response [27,35,36,38-45]. In particular, aggression and self-injurious behavior, the behaviors associated with slower skill acquisition in our study, have been linked to low IQ scores [8]. It would be worth exploring these variables in future research. In addition, the method used to aggregate the data for clustering results in a relatively small feature space of only 8 dimensions. These dimensions correspond to broad categories of challenging behaviors but do not capture other aspects related to those behaviors such as function. A future study could improve on this work by starting with a higher-dimension behavioral feature space, including functions of behavior, and then utilizing contemporary feature selection algorithms to derive the most meaningful subset of features to be fed to the unsupervised learning algorithm. In this case, use of a clustering algorithm that is more sophisticated may be warranted; the k-means algorithm takes a simple approach to clustering that relies upon regularly shaped clusters throughout the feature space. Finally, we note that additional studies to validate the clusters identified here would be valuable. In particular, the use of an additional cohort of participants which could be assigned to clusters and then have their assignments verified by clinicians using a broader set of medical records would be important to verify that the clusters identified here are generalizable beyond the study population.

    This study is among the first to investigate types of autism spectrum disorder based on engagement in challenging behaviors and the impact of challenging behaviors on treatment response. Findings suggest that challenging behaviors do impact treatment response with specific topographies (ie, self-injurious behavior, aggression) being particularly detrimental. In future investigations, it would be worthwhile to map the function of the behavior (eg, attention, escape, tangible, automatic), in addition to the topography, and explore its impact on treatment response. Future research should also explore targeted interventions to improve skill acquisition based on cluster assignment, particularly for the clusters characterized by self-injurious behavior and aggression. Until such investigations are conducted, treatment providers should be aware that these behaviors seem to have a particularly negative impact on skill acquisition and interventions addressing these behaviors may be worth prioritizing in treatment. To further improve outcomes across individuals with autism spectrum disorder, attention must be given to segmentation within the disorder. Investigations, such as these, show the promise of unsupervised machine learning models in identifying types of autism spectrum disorder so that targeted treatments based on type membership may be explored. We recommend that clinicians who are interested in further exploring latent structural features of the autism spectrum, including challenging behaviors, proactively collect data to the greatest extent that is practical and unobtrusive. Such data, especially in aggregate, will be essential for gaining additional insights into autism spectrum disorder types with the ultimate goal of personalizing and optimizing treatment plans.

    Acknowledgments

    CPP was supported by the National Science Foundation (grant GRFP1849569).

    Authors' Contributions

    EL, ES, and DD conceived the research design. JGH, CPP, and EL conducted the analysis. ES, MN, and JGH performed the literature review. All authors contributed to drafting and revising the manuscript.

    Conflicts of Interest

    None declared.

    1. American Psychiatric Association DSM-5 Task Force. Diagnostic and Statistical Manual of Mental Disorders Fifth Edition. Washington, DC: American Psychiatric Publishing Inc; 2013.
    2. Jang J, Dixon DR, Tarbox J, Granpeesheh D. Symptom severity and challenging behavior in children with ASD. Res Autism Spectr Disord 2011 Jul;5(3):1028-1032. [CrossRef]
    3. Matson JL, Wilkins J, Macken J. The relationship of challenging behaviors to severity and symptoms of autism spectrum disorders. J Ment Health Res Intellect Disabil 2008 Dec 29;2(1):29-44. [CrossRef]
    4. Matson JL, Nebel-Schwalm M. Assessing challenging behaviors in children with autism spectrum disorders: a review. Res Dev Disabil 2007 Nov;28(6):567-579. [CrossRef] [Medline]
    5. Lecavalier L, Leone S, Wiltz J. The impact of behaviour problems on caregiver stress in young people with autism spectrum disorders. J Intellect Disabil Res 2006 Mar;50(Pt 3):172-183 [FREE Full text] [CrossRef] [Medline]
    6. Sikora D, Moran E, Orlich F, Hall T, Kovacs E, Delahaye J, et al. The relationship between family functioning and behavior problems in children with autism spectrum disorders. Res Autism Spectr Disord 2013 Feb;7(2):307-315 [FREE Full text] [CrossRef]
    7. Bishop SL, Richler J, Lord C. Association between restricted and repetitive behaviors and nonverbal IQ in children with autism spectrum disorders. Child Neuropsychol 2006 Aug;12(4-5):247-267. [CrossRef] [Medline]
    8. McTiernan A, Leader G, Healy O, Mannion A. Analysis of risk factors and early predictors of challenging behavior for children with autism spectrum disorder. Res Autism Spectr Disord 2011 Jul;5(3):1215-1222. [CrossRef]
    9. Murphy O, Healy O, Leader G. Risk factors for challenging behaviors among 157 children with autism spectrum disorder in Ireland. Res Autism Spectr Disord 2009 Apr;3(2):474-482 [FREE Full text] [CrossRef]
    10. Baghdadli A, Pascal C, Grisi S, Aussilloux C. Risk factors for self-injurious behaviours among 222 young children with autistic disorders. J Intellect Disabil Res 2003 Nov;47(Pt 8):622-627. [CrossRef] [Medline]
    11. Hartley S, Sikora D, McCoy R. Prevalence and risk factors of maladaptive behaviour in young children with autistic disorder. J Intellect Disabil Res 2008 Oct;52(10):819-829 [FREE Full text] [CrossRef] [Medline]
    12. Kozlowski AM, Matson JL, Rieske RD. Gender effects on challenging behaviors in children with autism spectrum disorders. Res Autism Spectr Disord 2012 Apr;6(2):958-964. [CrossRef]
    13. Cooper J, Heron T, Heward W. Applied Behavior Analysis, Second Edition. Upper Saddle River, NJ: Pearson; 2007.
    14. Granpeesheh D, Tarbox J, Dixon DR. Applied behavior analytic interventions for children with autism: a description and review of treatment research. Ann Clin Psychiatry 2009;21(3):162-173. [Medline]
    15. Hong E, Dixon DR, Stevens E, Burns CO, Linstead E. Topography and function of challenging behaviors in individuals with autism spectrum disorder. Adv Neurodev Disord 2018 Apr 30;2(2):206-215. [CrossRef]
    16. Machalicek W, Raulston T, Knowles C, Ruppert T, Carnett A, Alresheed F. Challenging behavior. In: Matson J, editor. Comorbid Conditions Among Children with Autism Spectrum Disorders. New York: Springer; Oct 2015.
    17. Bearss K, Johnson C, Smith T, Lecavalier L, Swiezy N, Aman M, et al. Effect of parent training vs parent education on behavioral problems in children with autism spectrum disorder: a randomized clinical trial. JAMA 2015 Apr 21;313(15):1524-1533 [FREE Full text] [CrossRef] [Medline]
    18. Aman M, Mcdougle C, Scahill L, Handen B, Arnold L, Johnson C, et al. Medication and parent training in children with pervasive developmental disorders and serious behavior problems: results from a randomized clinical trial. J Am Acad Child Adolesc Psychiatry 2009 Dec;48(12):1143-1154 [FREE Full text] [CrossRef]
    19. Lindgren S, Wacker D, Suess A, Schieltz K, Pelzel K, Kopelman T, et al. Telehealth and autism: treating challenging behavior at lower cost. Pediatrics 2016 Feb;137 Suppl 2:S167-S175 [FREE Full text] [CrossRef] [Medline]
    20. Campbell JM. Efficacy of behavioral interventions for reducing problem behavior in persons with autism: a quantitative synthesis of single-subject research. Res Dev Disabil 2003;24(2):120-138. [CrossRef] [Medline]
    21. Didden R, Korzilius H, van Oorsouw W, Sturmey P. Behavioral treatment of challenging behaviors in individuals with mild mental retardation: meta-analysis of single-subject research. Am J Ment Retard 2006 Jul;111(4):290-298. [CrossRef] [Medline]
    22. Heyvaert M, Saenen L, Campbell JM, Maes B, Onghena P. Efficacy of behavioral interventions for reducing problem behavior in persons with autism: an updated quantitative synthesis of single-subject research. Res Dev Disabil 2014 Oct;35(10):2463-2476. [CrossRef] [Medline]
    23. Rogers SJ, Vismara LA. Evidence-based comprehensive treatments for early autism. J Clin Child Adolesc Psychol 2008 Jan 03;37(1):8-38 [FREE Full text] [CrossRef] [Medline]
    24. Smith T, Iadarola S. Evidence base update for autism spectrum disorder. J Clin Child Adolesc Psychol 2015 Oct 02;44(6):897-922. [CrossRef] [Medline]
    25. Eldevik S, Hastings RP, Hughes JC, Jahr E, Eikeseth S, Cross S. Meta-analysis of early intensive behavioral intervention for children with autism. J Clin Child Adolesc Psychol 2009 May 19;38(3):439-450. [CrossRef] [Medline]
    26. Reichow B. Overview of meta-analyses on early intensive behavioral intervention for young children with autism spectrum disorders. J Autism Dev Disord 2012 Apr 15;42(4):512-520. [CrossRef] [Medline]
    27. Eldevik S, Hastings R, Hughes JC, Jahr E, Eikeseth S, Cross S. Using participant data to extend the evidence base for intensive behavioral intervention for children with autism. Am J Intellect Dev Disabil 2010 Sep;115(5):381-405. [CrossRef] [Medline]
    28. Howlin P, Magiati I, Charman T. Systematic review of early intensive behavioral interventions for children with autism. Am J Intellect Dev Disabil 2009;114(1):23-41. [CrossRef]
    29. Granpeesheh D, Dixon DR, Tarbox J, Kaplan AM, Wilke AE. The effects of age and treatment intensity on behavioral intervention outcomes for children with autism spectrum disorders. Res Autism Spectr Disord 2009 Oct;3(4):1014-1022. [CrossRef]
    30. Linstead E, Dixon DR, French R, Granpeesheh D, Adams H, German R, et al. Intensity and learning outcomes in the treatment of children with autism spectrum disorder. Behav Modif 2017 Mar 21;41(2):229-252. [CrossRef] [Medline]
    31. Linstead E, Dixon DR, Hong E, Burns CO, French R, Novack MN, et al. An evaluation of the effects of intensity and duration on outcomes across treatment domains for children with autism spectrum disorder. Transl Psychiatry 2017 Sep 19;7(9):e1234-e1234 [FREE Full text] [CrossRef] [Medline]
    32. Makrygianni M, Reed P. A meta-analytic review of the effectiveness of behavioural early intervention programs for children with Autistic Spectrum Disorders. Res Autism Spectr Disord 2010 Oct;4(4):577-593 [FREE Full text] [CrossRef]
    33. Virués-Ortega J. Applied behavior analytic intervention for autism in early childhood: meta-analysis, meta-regression and dose-response meta-analysis of multiple outcomes. Clin Psychol Rev 2010 Jun;30(4):387-399 [FREE Full text] [CrossRef] [Medline]
    34. Virués-Ortega J, Rodríguez V, Yu C. Prediction of treatment outcomes and longitudinal analysis in children with autism undergoing intensive behavioral intervention. Int J Clin Health Psychol 2013 May;13(2):91-100 [FREE Full text] [CrossRef]
    35. Ben Itzchak E, Zachor D. Who benefits from early intervention in autism spectrum disorders? Res Autism Spectr Disord 2011 Jan;5(1):345-350 [FREE Full text] [CrossRef]
    36. Eldevik S, Hastings R, Jahr E, Hughes JC. Outcomes of behavioral intervention for children with autism in mainstream pre-school settings. J Autism Dev Disord 2012 Feb;42(2):210-220 [FREE Full text] [CrossRef] [Medline]
    37. Flanagan H, Perry A, Freeman N. Effectiveness of large-scale community-based Intensive Behavioral Intervention: a waitlist comparison study exploring outcomes and predictors. Res Autism Spectr Disord 2012 Apr;6(2):673-682 [FREE Full text] [CrossRef]
    38. Perry A, Cummings A, Geier J, Freeman N, Hughes S, Managhan T, et al. Predictors of outcome for children receiving intensive behavioral intervention in a large, community-based program. Res Autism Spectr Disord 2011 Jan;5(1):592-603 [FREE Full text] [CrossRef]
    39. Remington B, Hastings R, Kovshoff H, degli Espinosa F, Jahr E, Brown T, et al. Early intensive behavioral intervention: outcomes for children with autism and their parents after two years. Am J Ment Retard 2007 Nov;112(6):418-438. [CrossRef] [Medline]
    40. Ben-Itzchak E, Zachor D. The effects of intellectual functioning and autism severity on outcome of early behavioral intervention for children with autism. Res Dev Disabil 2007;28(3):287-303 [FREE Full text] [CrossRef] [Medline]
    41. Eikeseth S, Smith T, Jahr E, Eldevik S. Intensive behavioral treatment at school for 4- to 7-year-old children with autism. a 1-year comparison controlled study. Behav Modif 2002 Jan;26(1):49-68 [FREE Full text] [CrossRef] [Medline]
    42. Eikeseth S, Smith T, Jahr E, Eldevik S. Outcome for children with autism who began intensive behavioral treatment between ages 4 and 7: a comparison controlled study. Behav Modif 2007 May;31(3):264-278 [FREE Full text] [CrossRef] [Medline]
    43. Hayward D, Eikeseth S, Gale C, Morgan S. Assessing progress during treatment for young children with autism receiving intensive behavioural interventions. Autism 2009 Nov;13(6):613-633 [FREE Full text] [CrossRef] [Medline]
    44. Magiati I, Charman T, Howlin P. A two-year prospective follow-up study of community-based early intensive behavioural intervention and specialist nursery provision for children with autism spectrum disorders. J Child Psychol Psychiatry 2007 Aug;48(8):803-812 [FREE Full text] [CrossRef] [Medline]
    45. Magiati I, Moss J, Charman T, Howlin P. Patterns of change in children with autism spectrum disorders who received community based comprehensive interventions in their pre-school years: A seven year follow-up study. Res Autism Spectr Disord 2011 Jul;5(3):1016-1027 [FREE Full text] [CrossRef]
    46. Eikeseth S, Klintwall L, Jahr E, Karlsson P. Outcome for children with autism receiving early and intensive behavioral intervention in mainstream preschool and kindergarten settings. Res Autism Spectr Disord 2012 Apr;6(2):829-835 [FREE Full text] [CrossRef]
    47. Beglinger L, Smith T. A review of subtyping in autism and proposed dimensional classification model. J Autism Dev Disord 2001 Aug;31(4):411-422. [CrossRef] [Medline]
    48. Bruining H, de Sonneville L, Swaab H, de Jonge M, Kas M, van Engeland H, et al. Dissecting the clinical heterogeneity of autism spectrum disorders through defined genotypes. PLoS One 2010 May 28;5(5):e10887-e10887 [FREE Full text] [CrossRef] [Medline]
    49. Hu V, Lai Y. Developing a predictive gene classifier for autism spectrum disorders based upon differential gene expression profiles of phenotypic subgroups. N Am J Med Sci (Boston) 2013;6(3):1-18 [FREE Full text] [CrossRef] [Medline]
    50. Veatch O, Veenstra-Vanderweele J, Potter M, Pericak-Vance MA, Haines JL. Genetically meaningful phenotypic subgroups in autism spectrum disorders. Genes Brain Behav 2014 Mar;13(3):276-285 [FREE Full text] [CrossRef] [Medline]
    51. Fein D, Stevens M, Dunn M, Waterhouse L, Allen D, Rapin I, et al. Subtypes of pervasive developmental disorder: clinical characteristics. Child Neuropsychol 2010 Aug 09;5(1):1-23 [FREE Full text] [CrossRef]
    52. Lord C, Bishop S, Anderson D. Developmental trajectories as autism phenotypes. Am J Med Genet C Semin Med Genet 2015 Jun;169(2):198-208 [FREE Full text] [CrossRef] [Medline]
    53. Pickles A, Anderson D, Lord C. Heterogeneity and plasticity in the development of language: a 17-year follow-up of children referred early for possible autism. J Child Psychol Psychiatry 2014 Dec;55(12):1354-1362 [FREE Full text] [CrossRef] [Medline]
    54. Stevens M, Fein D, Dunn M, Allen D, Waterhouse LH, Feinstein C, et al. Subgroups of children with autism by cluster analysis: a longitudinal examination. J Am Acad Child Adolesc Psychiatry 2000 Mar;39(3):346-352 [FREE Full text] [CrossRef] [Medline]
    55. Stevens E, Dixon D, Novack MN, Granpeesheh D, Smith T, Linstead E. Identification and analysis of behavioral phenotypes in autism spectrum disorder via unsupervised machine learning. Int J Med Inform 2019 Sep;129:29-36 [FREE Full text] [CrossRef] [Medline]
    56. Stevens E, Atchison A, Stevens L, Hong E, Granpeesheh D, Dixon D, et al. A cluster analysis of challenging behaviors in autism spectrum disorder. 2017 Dec Presented at: 16th IEEE International Conference on Machine Learning and Applications; Devember 18-21; Cancun, Mexico p. 661-666   URL: https://doi.org/10.1109/ICMLA.2017.00-85 [CrossRef]
    57. Skills software. Skills Global LLC.   URL: https://www.skillsforautism.com/ [accessed 2019-02-01]
    58. Dixon D, Tarbox J, Najdowski A, Wilke A, Granpeesheh D. A comprehensive evaluation of language for early behavioral intervention programs: the reliability of the SKILLS language index. Res Autism Spectr Disord 2011 Jan;5(1):506-511 [FREE Full text] [CrossRef]
    59. Persicke A, Bishop M, Coffman C, Najdowski A, Tarbox J, Chi K, et al. Evaluation of the concurrent validity of a skills assessment for autism treatment. Res Autism Spectr Disord 2014 Mar;8(3):281-285 [FREE Full text] [CrossRef]
    60. Granpeesheh D, Tarbox J, Najdowski A, Kornack J. Evidence-Based Treatment for Children with Autism: The CARD Model. New York: Elsevier; 2015.
    61. Raschka S. Python Machine Learning. Birmingham, United Kingdom: Packt Publishing; 2015.
    62. Teetor P. R Cookbook: Proven Recipes for Data Analysis, Statistics, and Graphics. Sebastopol, CA: O'Reilly Media; 2011.
    63. Napolitano D, Knapp V, Speares E, McAdam D, Brown H. The role of functional assessment in treatment planning. In: Matson JL, editor. Functional Assessment for Challenging Behaviors. New York: Springer; 2012:195-211.

    Edited by G Eysenbach; submitted 06.02.21; peer-reviewed by S Iadarola, Q Zhan; comments to author 01.03.21; revised version received 23.04.21; accepted 29.04.21; published 02.06.21

    Copyright

    ©Julie Gardner-Hoag, Marlena Novack, Chelsea Parlett-Pelleriti, Elizabeth Stevens, Dennis Dixon, Erik Linstead. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 02.06.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.