Ramdoss et al. (2012)

SECONDARY REVIEW CRITIQUE

Key:

 

ASD = Autism Spectrum Disorders

d = standardized mean difference

CBI = computer-based interventions

NA = not applicable

NAP = Non-overlapping of All Pairs

P = participant

pmh = Patricia Hargrove, blog developer

SR = systematic review

Source: Ramdoss, S., Machalicek, W., Rispoli, M., Mulloy, A. Russell Lang, R., & O’Reilly, M. (2012). Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: A systematic review. Developmental Neurorehabilitation , 15, 119-135.

 

Reviewer(s): pmh

 

Date: April 30, 2015

 

Overall Assigned Grade: B   (Highest possible grade, based on the design of the paper, is B.)

 

Level of Evidence: B

 

Take Away: Ramdoss et al. (2012) focused on a variety of outcomes and treatment procedures, only outcomes and treatment procedures concerned with prosody will be discussed in this review. The systematic review (SR) summarized and analyzed the literature pertaining to the use of computer-based interventions (CBI) to treat social and emotional outcomes for children, adolescents, and adults with Autism Spectrum Disorders. All the direct treatments of recognition of prosodic emotion employed Mind Reading software. Gains were moderate to large.

 

What type of secondary review? Narrative Systematic Review

  • Classic Systematic Review

 

  1. Were the results valid? Yes

– Was the review based on a clinically sound clinical question? Yes

 

– Did the reviewers clearly describe reasonable criteria for inclusion and exclusion of literature in the review (i.e., sources)? Yes

 

– Authors noted that they reviewed the following resources:

  • electronic based databases
  • references from identified literature

 

– Did the sources involve only English language publications? Yes

– Did the sources include unpublished studies? No

– Was the time frame for the publication of the sources sufficient? Yes

– Did the reviewers identify the level of evidence of the sources? Yes. They classified each of the sources as either suggestive, preponderant, or conclusive.

– Did the reviewers describe procedures used to evaluate the validity of each of the sources? Yes

– Was there evidence that a specific, predetermined strategy was used to evaluate the sources? Yes

– Did the reviewers or review teams rate the sources independently? Yes

– Were interrater reliability data provided? Yes. In addition, the discussed disagreements and came to a consensus on all disagreements.

– If the reviewers provided interrater reliability data, list them: Not Applicable

 

– If there were no interrater reliability data, was an alternate means to insure reliability described? Not Applicable

 

– Were assessments of sources sufficiently reliable? Yes

– Was the information provided sufficient for the reader to undertake a replication? Yes

 

– Did the sources that were evaluated involve a sufficient number of participants? Variable. Numbers of participants (P) in the sources ranged from 4 to 79 with a mean of 28 Ps.

 

– Were there a sufficient number of sources? Yes. The overall number of sources was 11 and there were 12 experiments. The number of sources concerned with some aspect of prosody was 6.

  1. Description of outcome measures:

NOTE: Only procedures concerned with prosody as an outcome or as a means to treating other outcomes will be described here.

The outcome measures were

  • Outcomes Associated with Procedure #1 (Beaumont & Sofronoff, 2008; prosody was part of the intervention, not an outcome): improved reciprocal positive interaction, social responsiveness, initiating and maintaining conversations, interactive play, interpreting facial expressions and body postures, knowledge of anger and anxiety management strategies.
  • Outcome Associated with Procedure #2 (Golan & Baron-Cohen, 2006; Experiment 1): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations
  • Outcome Associated with Procedure #3 (Golan & Baron-Cohen, 2006; Experiment 2): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations

 

  • Outcome Associated with Procedure #4 (Lacava et al., 2007): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations
  • Outcomes Associated with Procedure #5 (Lacava et al., 2010): improved comprehension/recognition of complex emotions as represented in facial representations and positive social interaction

 

  • Outcome Associated with Procedure #6 (Silver & Oakes, 2001; the intervention is likely to have involved prosody because the authors noted that one of the components of the treatment consisted of interpreting emotions from narratives): improved recognition of emotions from photographs of faces and from cartoons. NOTE: I am assuming that the narratives were presented outloud. If they were written, this would not be a prosody related intervention (pmh.)

 

  1. Description of results: (information is found in tables and in the prose; be sure to review both)

– What evidence-based practice (EBP) measures were used to represent the magnitude of the treatment/effect size?

  • standardized mean difference (d) effect size for group analyses
  • Non-overlapping of All Pairs (NAP) for single case studies
  • Following the calculation of the measures, the authors sorted the experiments on the basis of design/methodology as

– suggestive

– preponderant

– conclusive (see p. 122 for criteria for categorization)

– Summarization overall findings of the secondary review:

  • There was only one conclusive experiment (the authors’ highest level of evidence) from the 12 possible experiments. That investigation was not concerned with prosody and will not be discussed here.
  • For the investigations concerned with prosody/voice representations of emotion, improvement across the relevant investigations was in a positive direction with small to moderate improvement.
  • The authors noted that while there is as yet insufficient evidence to support the overall use of CBI to teach social/emotional skills to students with ASD, the research provides helpful guidelines:

– Golan & Baron-Cohen (2006) determined that there were no significant differences between CBI and face-to-face interventions. This should be considered to be positive support for the use of CBI to improve emotion recognition.

– Some of the research indicated that progress in emotion recognition was correlated with the number of intervention sessions.

  • The effect sizes/NAPs were categorized as ineffective, small, moderate, or large. The overall effect sizes for the outcomes or treatments associated with improving prosody/voice emotion recognition were small and moderate. The quality of improvement and the level of evidence for the Outcomes Associated with the different prosody related (treating prosody or using prosody to treat another aspect of communication) experiments is listed below:
  • Outcomes Associated with Procedure #1 (Beaumont & Sofronoff, 2008; prosody was part of the intervention, not an outcome): improved reciprocal positive interaction, social responsiveness, initiating and maintaining conversations, interactive play, interpreting facial expressions and body postures, knowledge of anger and anxiety management strategies: Large effect for social outcomes; [Certainty of evidence = preponderant]

 

  • Outcome Associated with Procedure #2 (Golan & Baron-Cohen, 2006; Experiment 1): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations: Moderate effect for faces and voices outcomes; No significant differences on reading the mind tasks [Certainty of evidence = suggestive]
  • Outcome Associated with Procedure #3 (Golan & Baron-Cohen, 2006; Experiment 2): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations: No significant between 2 treatment groups (CBI vs face-to face treatment) [Certainty of evidence = suggestive]

 

  • Outcome Associated with Procedure #4 (Lacava et al., 2007): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations: Moderate effect size for Faces and Voices (prosody) subtests [Certainty of evidence = preponderant]

 

  • Outcomes Associated with Procedure #5 (Lacava et al., 2010): improved comprehension/recognition of complex emotions as represented in facial representations and positive social interaction: Small effect for social interaction outcomes; Large effects for Faces and Voices subtests [Certainty of evidence = preponderant]

 

  • Outcome Associated with Procedure #6 (Silver & Oakes, 2001; the intervention is likely to have involved prosody because the authors noted that one of the components of the treatment consisted of interpreting emotions from narratives): improved recognition of emotions from photographs of faces and from cartoons. Large effects [Certainty of evidence = suggestive] NOTE: I am assuming that the narratives were presented outloud. If they were written, this would not be a prosody related intervention (pmh).

 

– Were the results precise? Unclear

– If confidence intervals were provided in the sources, did the reviewers consider whether evaluations would have varied if the “true” value of metrics were at the upper or lower boundary of the confidence interval? Not Applicable

 

– Were the results of individual studies clearly displayed/presented? Yes

– For the most part, were the results similar from source to source? Yes, all the research concerned with prosodic recognition of emotion was in the positive direction.

–  Were the results in the same direction? Yes

–  Did a forest plot indicate homogeneity? Not Applicable

 

  1. Was heterogeneity of results explored? Yes
  1. Were the findings reasonable in view of the current literature? Yes
  2. Were negative outcomes noted? Yes

           

                                                                                                                   

  1. Were maintenance data reported? No

 

  1. Were generalization data reported? Yes. The investigations also were concerned with skills other than the prosodic recognition. These could be considered generalizations. The will not be discussed here.

 

 

SUMMARY OF INTERVENTION

 

Population:   ASD; children, adolescents, adults

 

Prosodic Targets: affect recognition (comprehension/receptive) as noted in Outcomes #2, 3, 4, and 5.

Nonprosodic Targets: A variety of social interaction skill and facial (Outcomes #1 and 6) and body posture recognition of emotions (Outcome #1)

Aspects of Prosody Used in Treatment of Nonprosodic Targets: Affective prosody (Outcome #1) and overall prosody in narratives (Outcome #6)

Description of Procedure/Source #1—Outcomes Associated with Procedure #1 (Beaumont & Sofronoff, 2008; prosody was part of the intervention, not an outcome): improved reciprocal positive interaction, social responsiveness, initiating and maintaining conversations, interactive play, interpreting facial expressions and body postures, knowledge of anger and anxiety management strategies

PROCEDURE #1

  • The investigators used Junior Detective Training Program software
  • The intervention comprised 2 phases:

– Phase 1: Using computer animation, Ps learned to interpret facial expression, body postures, and prosody of human characters

– Phase 2: Using cartoon characters, Ps learned to interpret emotions in a variety of contexts using nonverbal (including prosody?) and environmental cues.

EVIDENCE SUPPORTING PROCEDURE/SOURCE #1

  • Large improvements on measures of social interaction and emotion management.

EVIDENCE CONTRAINDICATING PROCEDURE/SOURCE #1

  • Measures of recognition of facial and body posture representations of emotion did not improve significantly.

Description of Procedure/Source #2— Outcome Associated with Procedure #2 (Golan & Baron-Cohen, 2006; Experiment 1): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations

PROCEDURE #2

  • Investigators used Mind Reading software
  • Ps used the software at home for about 2 hours a week for 10-15 weeks.
  • The software contained a emotion library, games, and instructional logs. The software represented 24 emotion groups at 4 developmental levels.

EVIDENCE SUPPORTING PROCEDURE/SOURCE #2

  • Moderate, significant improvement on interpreting facial expression

EVIDENCE CONTRAINDICATING PROCEDURE/SOURCE #2

  • No significant improvements in reading the mind in eyes, voices, or films

Description of Procedure/Source #3— Outcome Associated with Procedure #3 (Golan & Baron-Cohen, 2006; Experiment 2): improved comprehension/recognition of complex emotions as represented in prosody and in facial representation

PROCEDURE #3

  • Two sets of procedures were compared: face-to-face social skills teaching procedures and CBI plus social skill course procedures. (The nature of the control group is confusing. In the prose it is referenced as tutoring and in Table 1, it is referenced as a social skills course.)
  • The authors of the SR only described the CBI procedures: the used the same Mind Reading Software and procedures as Golan & Baron-Cohen (2006), Experiment 1.

EVIDENCE SUPPORTING PROCEDURE/SOURCE #3

  • There were no significant differences in the outcomes of the CBI plus adult tutors procedures and face-to-face social skills teaching groups.

Description of Procedure/Source #4— Outcome Associated with Procedure #4 (Lacava et al., 2007): improved comprehension/recognition of complex emotions as represented in prosody and in facial representations

PROCEDURE #4

  • The investigators used Mind Reading software.
  • The software contains an emotions library, a learning center, and games. Ps were restricted in their use of games to 33% of the time they were engaged with the software.

 

EVIDENCE SUPPORTING PROCEDURE/SOURCE #4

  • The results indicated significant and moderate effect sizes for Faces and Voices (prosody) subtests of a measure of emotion recognition.

EVIDENCE CONTRAINDICATING PROCEDURE/SOURCE #4

  • The group size was small and the group assignment was not random.

Description of Procedure/Source #5— Outcomes Associated with Procedure #5 (Lacava et al., 2010): improved comprehension/recognition of complex emotions as represented in facial representations and positive social interaction

PROCEDURE #5

  • The investigators used Mind Reading Software.
  • Adult tutors who sat next to the Ps during the use of the software. The tutors prompted Ps and discussed emotions encountered in daily living.

EVIDENCE SUPPORTING PROCEDURE/SOURCE #5

  • Small, significant effect for social interaction outcomes.
  • Large, significant effects for Faces and Voices subtests for recognizing emotion.

EVIDENCE CONTRAINDICATING PROCEDURE/SOURCE #5

  • Small N.

Description of Procedure/Source #6— Outcome Associated with Procedure #6 (Silver & Oakes, 2001; the intervention is likely to have involved prosody because the authors noted that one of the components of the treatment consisted of interpreting emotions from narratives): improved recognition of emotions from photographs of faces and from cartoons

PROCEDURE #6—

  • The investigators used Emotion Trainer software.
  • Ps used Emotion Trainer software to interpret emotions from photos, physical situations (?), and narratives.
  • NOTE: I am assuming that the narratives were presented outloud. If they were written, this would not be a prosody related intervention (pmh).
  • Ps selected the represented emotion from 4 possibilities and was reinforced with the written message “well done” when correct.

EVIDENCE SUPPORTING PROCEDURE/SOURCE #6

  • Large, significant effects for recognizing mental/emotional states in narratives and cartoons

EVIDENCE CONTRAINDICATING PROCEDURE/SOURCE #6

  • No significant effect for recognizing emotion from facial expression.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: