Description of Assessments of Overall Project
Students were assessed using formal and informal measures prior to and following intervention. The students were administered three reading comprehension assessments (the Developmental Reading Assessment (DRA), the Informal Reading Inventory-7 (B/RIRI-7), and an informal oral retelling rubric, for fiction and nonfiction text, to document change in this area.
The DRA was administered as a pretest and posttest for fiction and nonfiction texts. Scores were reported for the comprehension strategies and skills subtests only.
The B/RIRI-7 for fiction and nonfiction text was administered prior to and following intervention. Percentages were obtained from the comprehension questions that were provided for each passage. Scores were reported only for reading comprehension.
The oral retelling rubrics were collected as a pretest and posttest as well as weekly for a total of six assessments per genre (fiction and nonfiction). Each rubric was ranked 1 to 4 with 1 being the lowest possible score and 4 being the highest possible score.
Triangulation of data was achieved by using the three reading comprehension measures to provide stronger support regarding the effectiveness of explicit instruction in metacognitive strategies on reading comprehension.
Three listening comprehension assessments (the Clinical Evaluation of Language Fundaments-4 (CELF-4), the Informal Reading Inventory-7 (B/RIRI-7), and the informal oral retelling rubrics were given to assess a change in listening comprehension skills, prior to and following intervention. The goal of assessing listening comprehension was to determine if a transfer effect occurred as a result of the explicit instruction in metacognitive strategies during guided reading activities. Hence, no ongoing measures were conducted throughout the study that assessed listening comprehension.
The scores obtained for the B/RIRI-7 and the oral retelling rubric were reported in the same manner as the reading comprehension scores. The CELF-4 provided a percentile rank for the Understanding Spoken Paragraphs subtest, which was the only measure reported from this standardized assessment.
The Metacomprehension Strategy Index (MSI) taken from Harp (2006) and the metacognitive awareness subtest for fiction and nonfiction texts on the DRA were used to assess awareness and application of strategies before, during, and after reading. Scores from each assessment were obtained prior to and following intervention and graphed for interpretation and visual analysis. Additionally, two samples were analyzed using the Metacognitive Strategies Checklist adapted from Harp to further assess knowledge and application of strategies before, during, and after reading.
All scores from each test or subtest were graphed for each student individually, to allow for visual representation of information. Additionally, due to the small number of students, scores were reported for each student to allow for individual analysis of the benefits of 8 weeks of metacognitive strategies instruction on reading and listening comprehension of fiction and nonfiction text. Lastly, information taken from each student was synthesized to discuss the overall benefit of instruction.
Students were labeled subject A and subject B for data analysis purposes and to maintain confidentiality. It is important to note that the posttest score was not collected for fiction text on the Developmental Reading Assessment (DRA) for subject A. This impacts the triangulation of data for fiction texts. Subject A's fiction test results should be viewed with caution and will not be considered as strongly as the outcome for the nonfiction assessments that did achieve triangulation.