Quick Navigation

Assessment

The First-Year Composition Program at NIU is actively engaged in various forms of assessment, with the goal of using our assessment results to improve the program. In 2002-2003, we developed our own program outcomes based on the Council of Writing Program Administrators' Outcomes Statement. To support our development of authentic, evidence-based program assessment, we became a member of the first cohort of the Inter/National Coalition for Electronic Portfolio research in 2004, and have been improving our electronic portfolio pedagogy and assessments ever since.

For more information about the history of eportfolios in NIU FYComp, see http://www.engl.niu.edu/mday/eport.html

Electronic Assessment of Eportfolios: Quantitative Results

Description:

We collected a student eportfolio randomly from each section of composition and in a series of half-day reading sessions, scored the eportfolios according to our newly revised programmatic rubric. Portfolios and readers were collected into groups to assure that teachers were not reading portfolios from their own classes, and each portfolio was scored by at least 2 different readers. Each scoring session began with a series of calibration sets where readers were asked to score portfolios and then results were discussed in the group. Group leaders had themselves read, scored and discussed the calibration sets previously. In addition, group leaders were given an administrative interface to the assessment engine that allowed them to see and display scores from their group, and in the case of the calibration sets, from other groups as well. It is our belief that this calibration is an essential portion of the assessment process both because it allows users to see and possibly adjust their scoring given a communal norm, and more importantly facilitates a discussion about why certain features of a text are rewarded or penalized.

Results

Do keep in mind that the above data represents what is effectively a single data point and thus should not be interpreted as connotating trends. Having said that, the data does seem to suggest that the instrument itself appears both sound and valid as it indicates a general progression of skill through the course sequence. We also found it curious that there appears to be a strong correlation between control of document level structure and the holistic score.

Assessment of Attitudes Regarding Eportfolios

Student Survey: Quantitative Results

After obtaining IRB approval, we created an electronic survey that collected demographic information from students enrolled in English 103. This survey also asked students to indicate the kinds of attitudes they had toward eportfolios and reflective writing. 962 students participated. A large majority of these students were neutral about eportfolios - many of them would not have had any exposure to eportfolios yet at that point. Many students were neutral about or disliked reflective writing.

Faculty Survey: Quantitative Results

After obtaining IRB approval, we administered an electronic survey to faculty (instructors, SPS, teaching assistants, and teaching interns) that collected quantitative information about their perceptions about students, attitudes toward eportfolios, and performance in teaching eportfolios. 58 faculty participated. These faculty perceived that students preferred creating eportfolios to engaging in reflective writing, and they believed that students performed adequately or well in creating eportfolios and eportfolio artifacts.

Faculty Survey: Qualitative Results

After obtaining IRB approval, we administered an electronic survey to faculty (instructors, SPS, teaching assistants, and teaching interns) that collected qualitative information about their experiences in teaching eportfolios and reflective writing to their students. 58 faculty participated. These respondents enjoyed the relative ease of the eportfolio platforms they chose, but also identified various difficulties their students encountered in constructing eportfolios, including student resistance to the work. Many respondents believed that reflective writing had a positive influence on their students' writing and attitudes toward writing, but many also saw no relationship between reflective writing and eportfolios.