Yenkimaleki (2017)

November 26, 2018

 EBP THERAPY ANALYSIS

Treatment Groups 

Note: Scroll about two-thirds of the way down the page to read the summary of the procedures.

Key:

C = Clinician

EBP = evidence-based practice

NA = not applicable

P = Patient or Participant

pmh =  Patricia  Hargrove, blog developer

SLP = speech–language pathologist

 

 

SOURCE: Yenkimaleki, M. (2017). Developing listening comprehension skills by interpreter trainees through prosody teaching: Does methodology make a difference.  Educational Research Review, 3 (2), 26-42DOI: 10.20319/pijss.2017.32.2642

 

REVIEWER(S): pmh

 

DATE: October 25, 2018

 

ASSIGNED GRADE FOR OVERALL QUALITY:  B The highest possible grade based on this this design was  A. The Assigned Grade for Overall Quality is not a judgment about the worth of the intervention; it merely rates the quality of the evidence supporting the intervention.

 

TAKE AWAY: The results of this investigation that compared a control intervention and two experimental interventions (implicit and explicit instruction in prosody awareness) revealed that the explicit intervention yielded significantly higher scores on tests of the comprehension of English than either the implicit or control interventions for students in Iran who were training to be Farsi-English interpreters.

 

 

  1. What type of evidence was identified? Prospective, Randomized Group Design with Controls

                                                                                                          

  • What was the level of support associated with the type of evidence?

Level = A

 

                                                                                                           

  1. Group membership determination:

                                                                                                           

  • Were participants (Ps) randomly assigned to groups? Yes. It was referred to as random by the investigator. However, there was matching/blocking for certain P characteristics.

 

 

  1. Was administration of intervention status concealed?

                                                                                                           

  • from participants? No
  • from clinicians?No
  • from analyzers?No

                                                                    

 

  1. Were the groups adequately described? Yes

 

–  How many  Ps were involved in the study?

  • total # of Ps: 18 
  • # of groups:3
  • List names of groups and the # of participants in each group:

     –  Control Group  (Foil intervention); n = 6

–  Implicit Group (Comparison intervention); n = 6

–  Explicit Group (Experimental intervention); n = 6

           

–  CONTROLLED P CHARACTERISTICS

  • age:age range of all ages 18-27 years
  • gender:each group 6m and 6f
  • native language: all Ps spoke Farsi
  • language learning to translate:English (for all Ps_
  • nationality: Iranian
  • Social Economic Status:
  • educational level of all groups of Ps:undergraduate students, majoring in translation and interpreting; in last year of studies at University of Applied Sciences in Iran

 

–  DESCRIBED P CHARACTERISTICS

 

  • Listening Component Score on TOEFL proficiency test:

     –  Control Group =  mean – 56.4

     –  Implicit Intervention =  mean –  56.5

     –  Explicit Intervention =  mean –  56.5

  • Writing Component Score on TOEFL proficiency test:

     –  Control Group =  mean – 56.7

     –  Implicit Intervention =  mean – 57.1

     –  Explicit Intervention =  mean – 56.2

  • Reading Component Score on TOEFL proficiency test:

     –  Control Group =  mean – 56.0

     –  Implicit Intervention =  mean – 54.8

     –  Explicit Intervention =  mean –  56.0

  • Overall Score on TOEFL proficiency test:

     –  Control Group =  mean –  563.5

     –  Implicit Intervention =  mean – 561.6

     –  Explicit Intervention =  mean – 562.7

 

–   Were the groups similar before intervention began? Yes

                                                         

–  Were the communication problems adequately described?  Not Applicable, (NA), the Ps did not have impairments. They were learning to be Farsi-English interpreters.

 

 

  1. Was membership in groups maintained throughout the study?

                                                                                                             

  • Did each of the groups maintain at least 80% of their original members?Yes
  • Were data from outliers removed from the study? No 

 

 

  1. Were the groups controlled acceptably?  Yes
  • Was there a no intervention group?No   
  • Was there a foil intervention group? Yes
  • Was there a comparison group?Yes
  • Was the time involved in the foil/comparison and the target groups constant? Yes

 

 

  1. Was the outcome measure appropriate and meaningful? Yes

 

  • OUTCOME: Overall quality of Listening Comprehension on Longman’s TOEFL comprehension module

 

–  The outcome measures was subjective.

–  The outcome measure was NOT objective.

                                         

 

  1. Were reliability measures provided?
  • Interobserver for analyzers?No 
  • Intraobserver for analyzers?No 
  • Treatment fidelity for teachers?No

 

 

  1. What were the results of the statistical (inferential) testing and/or the description of the results?

 

  • What level of significance was required to claim significance? p = 0.05

 

TREATMENT AND FOIL/COMPARISON GROUP ANALYSES

 

  • OUTCOME: Overall quality of Listening Comprehension on Longman’s TOEFL comprehension module

–  The gain in performance from preintervention to postintervention was

∞  NOT significant for control versus implicit intervention

∞  was significantly different for

  • explicit versus control
  • explicit versus implicit

 

  • What was the statistical test used to determine significance?ANOVA

 

  • Were confidence interval (CI) provided?No

 

 

  1. What is the clinical significanceNA

 

 

  1. Were maintenance data reported? No

 

  1. Were generalization data reported?No

 

 

  1. Describe briefly the experimental design of the investigation.

 

  • Eighteen Farsi speaking Iranian University students learning to interpret English were sorted into 3 groups (control, implicit intervention, explicit intervention.) The groups were controlled for test performance on measure of English, gender, and educational level.

 

  • The Ps were administered pre and post intervention tests of English listening comprehension before and after being exposed to the interventions which were administered at similar dosage rates.

 

 

ASSIGNED OVERALL GRADE FOR QUALITY OF EXTERNAL EVIDENCE:  B

 

 

SUMMARY OF INTERVENTION

 

PURPOSE: To compare the effectiveness of implicit versus explicit teaching of  prosody awareness on the listening comprehension of English Language Learners.

 

POPULATION: English Language Learners; Adults

 

MODALITY TARGETED: comprehension

 

ELEMENTS OF PROSODY USED AS INTERVENTION:  Not clear. This information may be available in references provided by the investigator.

 

OTHER ASPECTS OF LANGUAGE/COMMUNICATION TARGETED:  comprehension of spoken English

 

DOSAGE: Ps from all 3 groups received a total of 400 minutes of treatment. All Ps participated in all treatment sessions

 

MAJOR COMPONENTS:

 

  • There were 3 treatment groups:

– Control  (received a placebo intervention)

– Experimental: Implicit intervention

– Experimental:  Explicit intervention

 

  • The content of the intervention was awareness of prosody

 

CONTROL INTERVENTION

  • Dosage = 400 minutes

–  of “authentic audio tracks” (p. 30)  and

– completed listening comprehension exercises

  • Instructor explained procedures and provided feedback.

 

EXPLICIT INTERVENTION

  • Part 1: Dosage = 200 minutes

– of “authentic audio tracks” (p. 30) and

– completed listening comprehension exercises

  • Instructor explained procedures and provided feedback.

 

  • Part 2: Dosage = 200 minutes

–  Instructor provided explicit instruction (theoretical explanations regarding English prosody.)

– Ps completed exercises based on the instruction.

 

IMPLICIT INTERVENTION

  • Part 1:Dosage = 200 minutes

– of “authentic audio tracks” (p. 30) and

– completed listening comprehension exercises

  • Instructor explained procedures and provided feedback.

 

  • Part 2: Dosage = 200 minutes

– Ps were provided with implicit instruction via “authentic audios.”

– Ps completed exercises based on the authentic audios

– Recasts (rewording but maintaining meaning) were used (by the instructor?) immediately after the Ps errors

_______________________________________________________________

 

 

 

Advertisements

van Rees et al. (2012)

October 6, 2017

 

ANALYSIS

Comparison Research  

NOTE: A summary of the intervention used in this investigation can be found by scrolling approximately two-thirds of the way down the page. 

KEY: 

eta = partial eta squared

f = female

KP feedback = Knowledge of Performance feedback.

KR feedback = Knowledge of Results feedback

m = male

MLU = mean length of utterance

NA = Not Applicable

NT = No treatment or control (group)

P = participant or patient

PCC = Percent Consonants Correct

pmh = Patricia Hargrove, blog developer

SLP = speech-language pathologist

SW = Strong – Weak

T = Treatment (group)

TD = typically developing

WNL = within normal limits

WS =   Weak- Strong

 

 

SOURCE: van Rees, L. J., Ballard, K. J., McCabe, P., MacDonald-D’Silva, A. G., & Arciuli, J. (2012). Training production of lexical stress in typically developing children using orthographically biased stimuli and principles of motor learning. American Journal of Speech-Language Pathology, 21, 197-206.

 

REVIEWER: pmh

 

DATE: September 20, 2017

 

ASSIGNED GRADE FOR OVERALL QUALITY: No overall grade was assigned because this was a comparison study, not an intervention study.

 

TAKE AWAY: This investigation presented evidence supporting the claim that typically developing children can be taught to produce targeted lexical stress patterns. Further research is necessary to determine if this procedure is applicable to children with speech-language problems.

 

  1. What type of evidence was identified?

                                                                                                           

  • What was the type of design? Comparison Research: Prospective, Nonrandomized Group Design with Controls

 

  • What was the focus of the research? Clinically Related: The participants were typically developing (TD)

                                                                                                           

  • What was the level of support associated with the type of evidence? Level = B (This is not a rating of the worth of the intervention, rather it rates the quality of the design for supporting claims of intervention effectiveness.)

 

                                                                                                           

  1. Group membership determination:

                                                                                                           

  • If there were groups, were participants randomly assigned to groups? Yes and No; first the participants (Ps) were matched for age and sex and then they were randomly assigned to a treatment or control group.
  • If there were groups and Ps were not randomly assigned to groups, were members of groups carefully matched? Yes

                                                                    

 

  1. Were experimental conditions concealed?

                                                                                                           

  • from participants? No

                                                                    

  • from administrators of experimental conditions? No

                                                                    

  • from analyzers/judges? No; the reliability judge was blinded but the original data was scored by the administrator of the experimental conditions (i.e., the Clinician, C.)

                                                                    

 

  1. Were the groups adequately described? Yes

 

– How many participants were involved in the study?

 

  • total # of Ps: 14 Ps
  • # of groups:  2
  • List names of groups and the number of Ps in each group:

– Treatment (T) – n = 7

– No treatment (NT) – n = 7

  • Did all groups maintain membership throughout the investigation? Yes

 

 

CONTROLLED CHARACTERISTICS                                                     

  • age: 5-0 to 13-0 years
  • gender: : age appropriate
  • cognitive skills: typically developing (TD)
  • overall language skills: age appropriate
  • receptive language: age appropriate
  • language: native speaker of Australian English and it was the first language
  • speech-sound production: age appropriate
  • oral-motor structure and function: within normal limits (WNL)
  • developmental and genetic diagnoses: none
  • previous speech-language assessment or treatment: none
  • reading skills: age appropriate
  • Hearing: within normal limits (WNL)

 

DESCRIBED CHARACTERISTICS

  • age:

     – T = 5-8 to 12- 4 years; mean = 9-8

     – NT = 5-2 to 12-2 years; mean = 9-5

  • gender: 8f, 6m overall

   – T = 4f; 3m

   – NT = 4f; 3m

 

  • expressive language:

     – T = expressive language percentile rank for age 98%ile to 32%ile; mean = 71%ile

     – NT = expressive language percentile rank for age   95%ile to 45%il; mean = 71%ile

  • receptive language: Note: 1 P in the NT group scored slightly below the criterion score on the designed test. This P remained in the NT group because of scores WNL on receptive vocabulary, expressive language, and overall language measures.

     – T = receptive vocabulary percentile rank for age  98%ile to 75%ile; mean = 85%

     – NT = receptive vocabulary percentile rank for age   96%ile to 55%ile; mean 75%ile

     – T = receptive language percentile rank for age 88%ile to 42%ile; mean = 63%ile

     – NT = receptive language percentile rank for age   95%ile to 14%ile; mean = 63%ile

 

    

  • overall language skills:

     – T = core language percentile rank for age 95%ile to 42%ile; mean = 70%ile

     – NT = receptive language percentile rank for age   97%ile to 39%ile; mean = 69%ile

 

 

  • Percent Consonants Correct (PCC):

     – T = 100% to 92.6%; mean = 95.8%

     – NT = 98.1% to 87.4%; mean = 94.5%

 

  • reading:

     – T = word identification percentile rank for age 95%ile to 26%ile; mean = 57%ile

     – NT = word identification language percentile rank for age   99.9%ile to 43%ile; mean = 65%ile

 

     – T = word attack percentile rank for age 96%ile to 33%ile; mean = 60%ile

     – NT = word attack percentile rank for age   95%ile to 35%ile; mean = 69%ile

 

     – T = core language percentile rank for age 95%ile to 42%ile; mean = 70%ile

     – NT = receptive language percentile rank for age   97%ile to 39%ile; mean = 69%ile

 

     – T = basic skills cluster percentile rank for age 98%ile to 44%ile; mean = 9.90%ile

     – NT = receptive language percentile rank for age   97%ile to 40%ile; mean = 69%ile

 

  • Socio-economic Status: middle class

 

  • Race: Caucasian

 

 

– Were the groups similar? Yes, there were no significant differences in the speech, language, and reading measures.  

                                                         

–  Were the communication problems adequately described? Not Applicable (NA)—all Ps were typically developing (TD.)

 

 

  1. What were the different conditions for this research?

                                                                                                             

– Subject (Classification) Groups? No

 

                                                               

– Experimental Conditions? Yes

– 2 Intervention groups ( T, NT)

 

Criterion/Descriptive Conditions? No

 

 

  1. Were the groups controlled acceptably? Yes

 

 

  1. Was the dependent measure appropriate and meaningful? Yes

                                                                                                             

  • OUTCOME: The number of bisyllable pseudowords read aloud with accurate lexical stress and speech sound production (19 words were targets of the intervention for T group and 11 were generalization words for T group. None of the words were to the NT group serving as a control.)

 

  • The dependent measure/outcome was subjective.

 

  • The dependent measure/outcome was not objective.

 

 

  1. Were reliability measures provided? Yes

                                                                                                            

  • Interobserver for analyzers? Yes

 

OUTCOME: The number of bisyllable pseudowords read aloud with accurate lexical stress and speech sound production = 96.12%

 

  • Intraobserver for analyzers? Yes

 

OUTCOME: The number of bisyllable pseudowords read aloud with accurate lexical stress and speech sound production = 96.83%

 

 

  • Treatment or test administration fidelity for investigators? Yes

 

– 95.55% for application of the treatment protocol

 

 

  1. Description of design:

 

  • This investigation involved a comparison design that was prospective and quasi-randomized (Ps matched into pair by sex and age and then randomly assigned to a treatment group) with a small number of TD Ps.

 

  • There were 2 treatment groups: T and NT.

 

  • There were 3 assessment phases:

– As a baseline measure, Ps were tested 3 times over a 7 day period.

– As a measure of progress related to treatment:

∞ The T Ps were assessed within 7 days of completing treatment.

∞ The NT Ps were assessed 3 weeks after the 3rd baseline test.

– As a maintenance measure:

∞   The T Ps were assessed 4 weeks after completing treatment.

∞ The NT Ps were assessed 7 weeks after the 3rd baseline test.

 

  • The testing stimuli were 30 pictures of pseudowords accompanied by written representations of the pseudowords. Nineteen of the pseudowords were target words from the T group’s treatment protocol and, as a generalization measure, 11 pseudowords had not been treated.

 

  • Criterion for success was 80% correct during training for 3 consecutive sessions.

 

  • For the most part, analyses involved inferential statistics with the 3rd baseline serving as the preintervention data point. The following data points were of interest:

– Preintervention versus immediate post intervention (T group) or 3 weeks post final baseline (NT group)

– immediate post intervention (T group) or 3 weeks post final baseline (NT group) versus 4 weeks post intervention or follow up (T group) or 7 weeks post final baseline (NT group)

– T group versus NT group

– Performance on generalization pseudowords.

 

 

  1. What were the results of the statistical (inferential) testing?

 

 

NOTE: The investigators selected 0.01 as the alpha level because there were multiple comparisons.

 

  • OUTCOME: The number of bisyllable pseudowords read aloud with accurate lexical stress and speech sound production

 

– Preintervention versus immediate post intervention (T group) or 3 weeks post final baseline (NT group)

Performance of T group, but not NT group, improved significantly .

– Immediate post intervention (T group) or 3 weeks post final baseline (NT group) versus 4 weeks post intervention of follow up (T group) or 7 weeks post final baseline (NT group)

 

T groups’ scores did not change significantly from immediate postintervention to the follow up (4 week post intervention) indicating that the progress was maintained.

          

∞ NT groups’ scores did not change significantly from 3 weeks post final baseline to 7 weeks post indicating stability, and no improvement when they were serving as a control group.

 

–  Performance on generalization pseudowords.

∞ For the T group, performance on generalization words improved from baseline to immediate post intervention. The progress was maintained at the follow-up testing.

For the NT group, performance was low and stable throughout all the testing phases indicating there was no improvement.

 

  • What was the statistical test used to determine significance? ANOVA

 

  • Were effect sizes provided? Yes, The investigators used partial eta squared (ETA), but ETA were not reported for the specific comparisons of interest.

 

  • Were confidence interval (CI) provided? No

 

 

  1. Summary of correlational results:  NA

 

 

  1. Summary of descriptive results for Qualitative research only: NA

 

 

  1. Brief summary of clinically relevant results:

 

  • The effectiveness of this program for TD children was supported—TD children can learn to produce the targeted lexical stress patterns in bisyllable pseudowords. The TD children also were able to produce untreated pseudowords accurately and their progress was maintained for 4 weeks after the conclusion of instruction.

 

  • Additional research is necessary to determine if this treatment is effective for children with communication disorders.

 

ASSIGNED GRADE FOR QUALITY OF EXTERNAL EVIDENCE: Not graded because this was comparison investigation

 

 

SUMMARY OF THE INTERVENTION

 

 

Population:  Typically developing

 

Prosodic Targets:  Lexical Stress (Strong – Weak, SW or Weak- Strong, WS)

 

Nonprosodic Targets: Speech sounds

 

Description of Procedure (Lexical Stress Training Using Principles of Motor Learning)

 

  • The training program used Principles of Motor Learning (PML) as its framework.

 

  • The 19 training stimuli were bisyllable pseudowords with 10 SW or 9 WS stress patterns. The stimuli were presented on cards with a picture of a “cartoon alien” (p. 199) and its written representation in letters with the targeted stress patterns.

 

  • The dosage of the treatment was 50 minute sessions, four times a week.

 

  • Treatment was terminated

– when a P’s percentage correct during sessions was at least 80% correct for 3 consecutive sessions or

– after 12 sessions.

 

  • There were 2 phases of each session:

– Prepractice: about 10 minutes

– Practice: about 40 minutes

 

PREPRACTICE (about 10 minutes)

 

  • Stimuli involved 5 randomly selected training pseudowords from the 19 training stimuli.

 

  • The Clinician (C) described the procedures to the P

– P was directed to read aloud some pseudowords

– C would judge the words based on lexical stress and speech sound (phoneme) accuracy

 

  • C randomly selected a written/illustrated pseudoword and

– P tried to identify the locus of the stress

– then C and P discussed the difference between long (stressed, S) and short (weak, W)

– C provided corrective feedback as necessary

 

  • Ps then attempted individually to read aloud the pseudowords and C provided Knowledge of Performance (KP) feedback. That is, she described how the attempt did or did not meet criteria and provided cues to how P could correct any errors.

 

  • P produced 5 to 10 trials during Prepractice

 

PRACTICE (about 40 minutes)

 

  • All 19 training pseudowords were used although they were presented in random order until P had practiced 100 words (trials) per session.

 

  • C provided a break after each 20 trials which consisted of a brief game.

 

  • C explained to P that during the Practice phase she would not provide

– a model

– feedback that provided cues to facilitate accurate production

 

  • At the beginning, when C directed P to read aloud from the stimulus cards without modeling the target, she provided Knowledge of Results (KR) feedback (good/not good with a 3 to 4 second delay to encourage self-monitoring) on 50% of the trials with 100% KR feedback on the first 10 trials. This was faded to random 10% on the last 10 trials.

 

  • NOTE: Treatment procedures and baseline assessments were modified for 1 P who had difficulty reading the pseudowords. For that P, C modeled pseudowords until the 7th session when the P could read the words.

 

 

Evidence Supporting Procedure/Source #1——(Lexical Stress Training Using Principles of Motor Learning)

 

The results of this investigation indicated that TD children could learn to produce targeted stress patterns of bisyllable pseudowords as well as generalize to untreated words. In addition, the progress and was maintained for 4 weeks.

 

 

Evidence Contraindicating Procedure (Lexical Stress Training Using Principles of Motor Learning)

 

  • Research is necessary to determine if this treatment is effective for children with communication disorders.