2014 Program Assessment

All English-111X classes taught on Fairbanks West campus (12) participated in the  assessment by submitting Diagnostic essays, Research papers, and Reflective assignments as an electronic course portfolio through Google Drive. The University Writing Director then led a group of 6 teachers in applying the program-designed, AAC&U-based rubrics to student work in course portfolios. You can access our rubrics  and  the AAC&U VALUE rubrics, from which we built ours.

First, the group met to calibrate their scales for assessing the texts, then readers independently assessed randomly selected course portfolios. Readers determined whether students were meeting benchmark, milestone, or capstone levels of our outcomes. Four course portfolios were chosen for assessment, and two readers read each selected course portfolio. The Director and research assistants compiled the results, reading student writing a third time if there was a disagreement between the readers of more than one point.

Findings   »
Assessment & Curricular Changes   »

 

Findings:

Diagnostic Essays: Where are students when they enter our classrooms?

Control of Syntax and Mechanics in Diagnostic Essays

diagnostic essays

assessment-scale

Self-Assessment in Diagnostic Essays

self-assessment

Analysis:
  • Only a small percentage of these essays (5%) did not meet our benchmarks; writing placement is 95% accurate.
  • We scored essays (less than 10 %) that were close to reaching capstone achievement.

We learned that for more than 75% of our students we should expect work that communicates general meaning with clarity from our students when they arrive. We also learned that about 40% of our students are able to articulate or evaluate their previous learning; however, more than 50% of the diagnostics reflect that students may not have understood the question, or had difficulty with specific discussion of their previous learning. The results of this assessment will be shared with high school teachers.

Research Paper-Projects: What did we learn about students’ informational literacy?

Uses of Sources and Evidence in Research Writing

research-sources

assessment-scale

Intellectual Curiosity in Research-based Writing

reserach-curiosity

Analysis:
  • More than 65% of research-based writing reaches milestone achievement for intellectual curiosity, with <5% reaching capstone level.
  • More than 50% of our students are just at the benchmark for their use of Sources and Evidence to support ideas.

We learned that “intellectual curiosity’ is more than just interest in the subject; instead, it should reflect the level of depth a student presents in their writing. 50% of our students reached milestone levels for this habit of mind. Students in English 111x show varying experience in critical thinking about research-based writing,. Just 52.4% of our students are evidencing a benchmark for uses of sources and evidence.

Final Reflective Prompt: What did we learn about students’ ability to assess their choices in writing?

Assessing their choices in Reflective Writing

reflective

assessment-scale

Analysis:
  • Only 50% of students’ reflective writing reached milestone levels for assessing the choices they make. However, nearly10% of these were almost at the capstone level.
  • 10% of the samples could not be scored because the prompts did not elicit responses about assessing choice.

We learned that the prompt for this signature assignment needs a complete revision. Also, because so few of these writing samples drew on terms for the rhetorical situation, a concept that is central to the 111x curriculum, we plan to use these terms explicitly in the prompt, the Program guidebook, on Write Alaska, during orientation, and reinforce it again in English 685.

Assessment and Curricular Changes Resulting from Above Conclusions

  • We will use only one diagnostic prompt for all 111X sections in 2014-2015, including all UAF campuses and course formats.
    • We will include specific instructions to write “I’ in case some students come with a version of the rule that one should never use “I’ in an academic writing situation.
    • We will specify the diagnostic prompt so that it communicates what will be assessed with the rubric. For example, it will be stated that we are looking for critical self-assessment and reflection.
  • We will no longer collect research-based writing in our assessments.
    • The assessment process (collecting and reading) was worthwhile but time-intensive for faculty. Assessing just the diagnostic and reflective essays will save resources on the assessment.
    • As part of a General Education discussion on campus in Spring of 2014, the Dean of the Library asserted that faculty in the library will take responsibility in assessing information literacy for the new learning outcomes.
  • We will devote more curricular attention to how the writer’s ideas and purpose in the writing relates to the use of sources or evidence.
    • One teacher’s assignment required students to use sources from at least three different academic disciplines. We will increase our resources on Write.Alaska.edu for teaching research-based writing, rather than referring to it as “synthesis’ exclusively.
    • We will be spending more time on teacher training and the importance of framing the assessment assignments. All teachers should use the program’s reflective prompt as a final assignment for 111X in order to ensure systematic review of how well our curriculum supports transfer.
    • We have revised the English 685 curriculum (which supports beginning teachers in the Writing Program) so that participants create assignments that highlight aspects of the rhetorical situation to increase our student’s rhetorical knowledge about audience, purpose, and context for their writing.

Leave a comment

Your email address will not be published. Required fields are marked *