Simmons et al. (2016)

EBP THERAPY ANALYSIS

Treatment Groups

Note: Scroll about two-thirds of the way down the page to read the summary of the procedure(s).

Key:

C = Clinician

EBP = evidence-based practice

f = female

G = grade level

m = male

NA = not applicable

P = Patient or Participant

pmh = Patricia Hargrove, blog developer

Ss = students who were enrolled in speech-language therapy

SLPs = speech–language pathologists

 

 

SOURCE:  Simmons, E. S., Paul, R., & Shic, F. (2016.) A mobile application to treat prosodic deficits in autism spectrum disorder and other communication impairments. Journal of Autism and Developmental Disorders, 46, 320-327.

 

REVIEWER(S): pmh

 

DATE: December 21, 2016

 

ASSIGNED GRADE FOR OVERALL QUALITY: C (The highest possible overall quality grade for this investigation was C+, Prospective Single Group with Pre and Post Testing)

 

TAKE AWAY: This preliminary investigation explores whether a software program (SpeechPrompts) has potential for treating prosodic problems in children and adolescents. Basic issues (e.g., student engagement, use pattern, perceive improvement, ease of use) about the intervention were investigated and it was determined that speech-language pathologists (SLPs) found SpeechPrompts enjoyable and easy to use in a school setting and they perceived that it resulted in perceived improvements in the Ss’ ability to produce stress, loudness, and intonation, but not speaking rate. These improvements were reported despite a low dose of the intervention over an 8 week period.

 

  1. What type of evidence was identified?

                                                                                                           

  • What was the type of evidence? Prospective, Single Group with Pre- and Post-Testing

                                                                                                          

  • What was the level of support associated with the type of evidence? Level = C+

 

                                                                                                           

  1. Group membership determination:

                                                                                                           

  • If there was more than one group, were participants (Ps) randomly assigned to groups? Not Applicable (NA), there was only one group.

 

  1. Was administration of intervention status concealed?

                                                                                                           

  • from participants (Ps)? No

                                                                    

  • from clinicians? No

                                                                    

  • from analyzers? No

                                                                    

 

  1. Were the groups adequately described? Yes

 

–           How many Ps were involved in the study?

 

  • total # of Ps: 40 students (Ss); 10 speech language pathologists (SLPs) [The 2 major questions involved different sets of participants: students who were enrolled in speech-language therapy; SLPs]
  • # of groups: 3 groups:

∞ treatment group (consisting of participants with a range of diagnosis, although most had the diagnosis of ASD);

∞ a subset of the treatment group with the diagnosis of ASD who the investigators had permission to link their diagnosis with their performance on the investigation task

∞ SLPs

  • List names of groups and the # of participants in each group:

     ∞ SLPs (speech-language pathologists) = 10

     ∞ Ss (students receiving treatment) = 40

  • Autism Spectrum Disorder (ASD) Subset = 12 (for this group of participants, the investigators were able to link their data and their diagnoses)

 

– List the P characteristics that are controlled (i.e., inclusion/exclusion criteria) or described. Provide data for each characteristic.

 

SLPs

 

CONTROLLED:

  • Licensure: All SLPs were licensed by the State of Connecticut
  • Certification: All SLPs were certified by the American Speech-Language-Hearing Association
  • caseload: All SLPs had students on their caseload with prosodic deficits.

 

DESCRIBED

  • caseload: (NOTE: The total is more than 10 because some SLPs worked in more than 1 setting)

– Preschool = 3

     – Elementary School = 8

     – Middle School = 4

     – High School = 2

  • years in current position:

– 0 to 5 years: 2

     – 6 – 10 years: 4

     – 11-15 years: 2

     – 16-20 years: 0

     – 21 or more years : 2

  • experience with iPads:

– minimal: 2

     – some experience: 2

     – significant experience: 60

 

Ss

CONTROLLED

  • expressive language:

All Ss spoke in full sentences.

     – All Ss exhibited prosodic problems

  • enrollment in speech-language therapy

– All Ss were enrolled in speech-language therapy as part of special education

 

DESCRIBED:

  • age: 5 years to 19 years (mean = 9.63)
  • gender: 31m, 9f
  • educational level of students (Ss):

– Elementary (PreK to Grade 4, G4) = 22

     – Middle School (G5 to G8) = 13

     – High School (G9 to G12) = 5

  • diagnosis:

– Autism Spectrum Disorder = 67.5% (27 Ss)

     – Speech and language impairment = 17.5% (7 Ss)

     – Intellectual disability = 7.5% (3 Ss)

     – Multiple disabilities = 2.5% (1 S)

     – Traumatic brain injury = 2.5% (1 S)

     – Other health problems = 2.5% (1 S)

 

ASD Subset

(investigators had permission to link diagnosis and task performance)

CONTROLLED

  • expressive language:

All Ss spoke in full sentences.

     – All Ss exhibited prosodic problems

  • enrollment in speech-language therapy

– All Ss were enrolled in speech-language therapy as part of special education

  • diagnosis: All ASD Subset Ps were diagnosed with ASD and the investigators were able to link their data with them

 

DESCRIBED:

  • age: 6 to 12 years; mean = 8.25
  • gender: 11m; 1f

 

– Were the groups similar before intervention began? NA, there was only one group

                                                         

– Were the communication problems adequately described? No

 

  • disorder type: (only prosodic disorders described; some of the Ss had problems in multiple domains)

– Rate/Rhythm = 27

– Stress = 29

– Volume = 28

 

 

  1. Was membership in groups maintained throughout the study?

                                                                                                             

  • Did the group maintain at least 80% of it original members? Yes, but 20% of the Ss were removed from data analysis because of absenteeism, SLP errors, or technical difficulties.

                                                               

  • Were data from outliers removed from the study? No

 

 

6 Were the groups controlled acceptably? NA, there was only one group.

 

 

  1. Were the outcomes measure appropriate and meaningful? Yes

                                                                                                             

– OUTCOMES

 

PROSODIC PERFORMANCE OUTCOMES (3 point scale; 0 = typical prosody; 1 = mildly atypical prosody, 2 = clearly atypical prosody; p 323; lower mean score signifies improvement)

 

  • OUTCOME #1: SLP Ranking of the quality of speaking rate in a 5 minute speech sample
  • OUTCOME #2: SLP Ranking of the quality of lexical stress in a 5 minute speech sample
  • OUTCOME #3: SLP Ranking of the quality of sentence stress in a 5 minute speech sample
  • OUTCOME #4: SLP Ranking of the quality of intensity in a 5 minute speech sample
  • OUTCOME #5: SLP Ranking of global intonation in a 5 speech minute sample

 

SOFTWARE USE OUTCOMES

  • OUTCOME #6: Number of times the SLP used the software
  • OUTCOME #7: Length of treatment sessions

 

STUDENT ENGAGEMENT OUTCOMES (ranked on a 5 point scale 1 = strong agreement/engagement, 5 = strong disagreement/no engagement)

 

  • OUTCOME #8: SLP’s ranking of student’s enjoyment
  • OUTCOME #9: SLP’s ranking of student’s attention maintenance
  • OUTCOME #10: SLP’s ranking of student’s consistency of responses
  • OUTCOME #11: SLP’s ranking of the student’s lack maladaptive behavior during treatment

 

SLP OUTCOMES

  • OUTCOME #12: SLP ranking of whether the software was enjoyable
  • OUTCOME #13: SLP ranking of ease of use of the software
  • OUTCOME #14: SLP ranking of function of the software
  • OUTCOME #15: SLP ranking of positive changes

 

All outcome measures that were subjective.

 

– None of the outcome measures were objective.

                                         

 

  1. Were reliability measures provided?

                                                                                                            

– Interobserver for analyzers? Yes, for some of the Prosodic Performance Outcomes:

 

  • OUTCOMES #1 through #4 (SLP Ranking of the quality of speaking rate, lexical stress, sentence stress, and in intensity in a 5 minute sample)—Cohen’s Kappa coefficient = 0.68.

 

  • Intraobserver for analyzers? No

 

  • Treatment fidelity for clinicians? No. Although no data were provided, SLPs were involved in a 20 minutes training session in which they were instructed about the treatment program (SpeechPrompts.)

 

 

  1. What were the results of the inferential, correlational, and descriptive analyses ?

 

— Summary Of Important Results

  •  What level of significance was required to claim statistical significance? p = 0.05

 

PRE AND POST TREATMENT ONLY ANALYSES:

descriptive data, correlational data, inferential data

 

 

PROSODIC PERFORMANCE OUTCOMES (3 point scale; 0 = typical prosody; 1 = mildly atypical prosody, 2 = clearly atypical prosody; p 323; lower mean score signifies improvement)

 

  • OUTCOME #1: (SLP Ranking of the quality of speaking rate in a 5 minute sample)

     – All Ss: no significant difference between pre- and post- intervention rankings

     – ASD subgroup: no significant difference between pre- and post- intervention rankings

 

  • OUTCOME #2: (SLP Ranking of the quality of lexical stress in a 5-minute sample)

All Ss: significantly lower post intervention scores

     – ASD subgroup: no significant difference between pre- and post- intervention rankings

 

  • OUTCOME #3: (SLP Ranking of the quality of sentence stress in a 5-minute sample)

All Ss: significantly lower post intervention scores

ASD subgroup: significantly lower post intervention scores

 

  • OUTCOME #4: (SLP Ranking of the quality of intensity in a 5 minute sample)

     – All Ss: significantly lower post intervention scores

ASD subgroup: significantly lower post intervention scores

 

  • OUTCOME #5: (SLP Ranking of global intonation in a 5-minute sample)

     – All Ss: significantly lower post intervention scores; no significant correlation between global intonation and the number of treatment minutes

ASD subgroup: significantly lower post intervention scores

 

SOFTWARE USE OUTCOMES

 

  • OUTCOME #6: (Number of times the SLP used the software)

     – All Ss: range of use 1 to 12 sessions; mean = 4.7 sessions

     – ASD subgroup: range of use 2 to 10 session; mean = 5.83 sessions

 

  • OUTCOME #7: (Length of treatment sessions)

   – All Ss: range of length sessions 5 to 90 minutes; mean = 21.25 minutes; VoiceMatch was used 52.9% of the time, VoiceChart was used 47.1% of the time; no significant correlation between the number of minutes of treatment received by the S and the length of time his/her SLP had been employed at the facility

     – ASD subgroup: range of length of session 10 to 30 minutes; mean 25.99 minutes

 

STUDENT ENGAGEMENT OUTCOMES (ranked on a 5 point scale

1 = strong agreement/engagement, 5 = strong disagreement/no engagement)

 

  • OUTCOME #8: (SLP’s ranking of student’s enjoyment)

     – All Ss: ratings were low and stable for 1st and last sessions– 92.5% of SLPs ranked this as ≤3; mean 1.66

     – ASD subgroup: ratings were low and stable for 1st and last sessions

 

  • OUTCOME #9: (SLP’s ranking of student’s attention maintenance)

– All Ss: ratings were stable for 1st and last sessions; 87.5% of SLPs ranked this as ≤3; mean 1.74

     – ASD subgroup: ratings were low and stable for 1st and last sessions

 

  • OUTCOME #10: (SLP’s ranking of student’s consistency of responses maintenance)

     – All Ss: ratings were stable for 1st and last sessions; 87.5% of SLPs ranked this as ≤3; mean 1.78

     – ASD subgroup: ratings were low and stable for 1st and last sessions

  • OUTCOME #11: (SLP’s ranking of the student’s lack maladaptive behavior during treatment maintenance)

     – All Ss: ratings were stable for 1st and last sessions 85% of SLPs ranked this as ≤3; mean 1.79

     – ASD subgroup: ratings were low and stable for 1st session and decreased for the final session.

 

SLP OUTCOMES

 

  • OUTCOME #12: (SLP ranking of whether the software was enjoyable)–≤80% of SLPs reported the software was enjoyable

 

  • OUTCOME #13: (SLP ranking of ease of use of the software) —≤80% of SLPs reported the software was easy to use

 

  • OUTCOME #14: (SLP ranking of function of the software)–≤80% of SLPs reported the software was functional

 

  • OUTCOME #15: (SLP ranking of positive changes0–≤80% of SLPs reported they observed positive changes in the Ss.

 

– What was the statistical test used to determine significance? Place xxx after any statistical test that was used to determine significance. t-test

 

– Were confidence interval (CI) provided? No

 

 

  1. What is the clinical significance

 

–  The EBP measure provided by the investigators was Standardized Mean Difference.

 

– Results of EBP testing and the interpretation:

 

  • OUTCOME #2: (SLP Ranking of the quality of lexical stress in a 5 minute sample)

All Ss: lower post intervention scores; d = 0.48 (small effect)

 

  • OUTCOME #3: (SLP Ranking of the quality of sentence stress in a 5 minute sample)

All Ss: lower post intervention scores; d = 0.77 (moderate effect)

ASD subgroup: lower post intervention scores; d = 0.80 (large effect)

 

  • OUTCOME #4: (SLP Ranking of the quality of intensity in a 5 minute sample)

     – All Ss: lower post intervention scores; d = 0.77 (moderate effect)

ASD subgroup: lower post intervention scores; d = 0.90 (large effect)

 

  • OUTCOME #5: (SLP Ranking of global intonation in a 5 minute sample)

     – All Ss: lower post intervention scores; d = 0.71 (moderate effect)

ASD subgroup: lower post intervention scores; d = 0.81 (large effect)

 

 

  1. Were maintenance data reported? No

 

 

  1. Were generalization data reported? Not clear

 

 

  1. A brief description of the experimental of the investigation:

 

  • This was a preliminary investigation to determine the potential of the software (SpeechPrompts) for treating prosody impairment. The investigators analyzed data from 2 groups of participants: SLPs (N = 10) and students (S) who were enrolled in speech-language therapy and had been identified as having prosodic problems.

 

  • Most (27), but not all, of Ss (N = 40) had been diagnosed with ASD. Accordingly, the investigators identified and analyzed separately data from a subgroup of Ss of with the diagnosis of ASD.

 

  • The outcomes associated with the SLPs involved

– their ranking their own perceptions of S engagement, attention, improvement , and behavior.

– their ranking their own perceptions regarding the function, ease of use, enjoyment, and student improvement associated with SpeechPrompts

– measures of number of uses and length of time of use of SpeechPrompts collected by the software.

 

  • S outcomes were derived from SLP rankings of perceptions of the acceptability of certain aspects of Ss’ prosody.

 

  • The investigators briefly trained the SLPs to use SpeechPrompts and requested that they use it at least one time a week over an 8 week period. (NOTE: Outcome data indicated that SLPs the average use was between 4 and 6 times over the 8 weeks.)

 

  • For the most part, the data were analyzed descriptively although paired t-tests and standardized mean difference were calculated for one set of analyses (i.e., perceived S performance outcomes.)

 

ASSIGNED OVERALL GRADE FOR QUALITY OF EXTERNAL EVIDENCE: C

 

 

 

SUMMARY OF INTERVENTION

 

PURPOSE: To investigate whether the mobile application SpeechPrompts has potential for treating prosodic impairments in children and adolescents.

 

POPULATION: ASD, Speech and Language Impairment; Children, Adolescents

 

MODALITY TARGETED: production

 

ELEMENTS/FUNCTIONS OF PROSODY TARGETED: speaking rate, stress (lexical, sentence), intensity, intonation

 

DOSAGE: 1 time a week (this was requested but SLPs average use was 4-6 times in 8 weeks)

 

ADMINISTRATOR: SLP

 

MAJOR COMPONENTS:

 

  • The investigators used SpeechPrompts which was delivered via iPads.

 

  • SpeechPrompts provides visual representations of acoustic measures of prosody. Two features of SpeechPrompts were used in this investigation:

– VoiceMatch: provides visual representation of short segments of speech as a waveform for 2 speakers, here the Clinician (C) and the S.

– When using VoiceMatch, C modeled a sentence and then S attempted to replicate the rate and stress patterns of the C.

– VoiceChart: was used to provide feedback to the S regarding loudness level.

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: