All posts in Classroom-based Assessment

Research-Based Practices to Assist Students with Exam Preparation

by Jay Rasmussen

[This post is the third installment in a series on effective exams. The previous posts address Understanding and Preventing Test Anxiety and Writing High-Quality Assessment Items.]

Exams, whether in the form of quizzes or tests, present a powerful opportunity for student learning (Brown, Roediger, & McDaniel, 2014). That learning is often brought about because emotion drives attention and attention drives learning (Wolfe, 2010; Sylwester 1994). For the majority of Bethel students, preparing for and taking exams is an emotion-laden experience that tends to grab attention and maximize cognitive processing. The following suggested practices have potential to help your students gain maximum learning from the exam experience: study guides, study groups, review sessions, reading and review strategies, student metacognition, and reduction of test anxiety.

Study Guides

While study guides are a staple test preparation tool, research has revealed several specific ways in which they can be made more beneficial for students. Instead of basic lists of objectives and terms to know, study guides are most effective if they include review questions that mirror those to be found on the exam. So as to give students the best idea of what to expect on the exam and what will be required of them, it is best if the review questions reflect accurately the variety of assessment items they will encounter. In addition to review questions, the most effective study guides have been shown to include expected learning outcomes; however, students usually prefer study guides centered on review questions (Nilson, 2010).

As a whole, study guides are most effective and most utilized by students if they are attractive and well-organized, interactive, user-friendly, and varied in content presentation (Harden et al., 1999). Sample tests are also very helpful for students, as they will inform them of important content areas and just how they will be asked to master the material. These sample tests need to specifically reflect learning outcomes in the language (e.g. “know this”, “be able to reproduce this”, etc.). For essay tests in particular, providing students with test verb definitions on the study guide is helpful in eliminating the common error of prompt misunderstanding (e.g. “create/devise,” “infer,” “illustrate,” “synthesize,” “propose,” “interpret,” “trace”) (Nilson, 2010). Finally, it is especially helpful to provide students with ways to self-assess on study guides so that they can keep track of their level of mastery of the material (Harden et al., 1999).

Study Groups

In addition to study guides, study groups are a research-supported exam preparation technique. These groups tend to be effective because they have been shown to both facilitate learning and they tend to improve critical thinking and self-esteem more than traditional learning methods (Stage et al., 1999; Cooper et al., 1993). This assessment preparation technique works best when groups meet regularly, when the groups are either formalized by the instructor or formed via sign-up sheet (both means prevent group attrition), and when the instructor distributes a list of student names and contact information (Nilson, 2010).

Review Sessions

Holding instructor-facilitated review sessions before an exam can be extremely helpful to students in their test preparation if run effectively. First, it is most beneficial for students if the discussion is led by student Q&A rather than the instructor posing and answering all questions and regurgitating lectures. Second, it is important be sure that students know to come prepared to ask questions and discuss material and not to expect a condensed lecture review from the instructor. Specifically, having students discuss review questions in pairs or small groups and then mock grading their answers in front of the class gives students an idea of how they will be assessed and also encourages student self-assessment. For essay tests in particular, have students generate essay questions in pairs or small groups, try to answer them, and evaluate responses. This activity gives students an opportunity to practice taking the exam, which reduces test anxiety (Nilson, 2010).

Angelo and Cross (1993) also note that having students generate test questions provides information that lets the instructor know if students have mistaken expectations for an upcoming exam, and giving feedback on these questions will then help students reorganize their studying.

Reading and Review Strategies

As it is highly likely that the majority of the material being studied in study guides, study groups, and review sessions is written, yet another useful research-based test preparation method has to do with helping students absorb written material. Eighty percent of research supports the notion that reviewing pertinent readings and materials prior to the exam improves test performance. However, utilizing the active recall technique provides students with the most effective way to study, retain, and recall written material. Also called the 3R technique (read, recite, review), students read a section of material, put it down, try to recite/summarize aloud as much as they can remember, and then reread the section. This manner of self-testing gets directly at the skills needed for good exam performance (Nilson, 2010).

Student Metacognition

Along the same lines as the 3R technique, research also points to student metacognition as a key variable in ideal exam preparation and performance. Metacognition plays a key role in student learning capacity. When students have learned to actively reflect on their own thinking and learning, they develop the ability to take ownership of their learning, to self-regulate, which paves the way for effective learning (National Research Council, 2000). Research has discovered a positive correlation between improved metacognition and academic success (Adey & Shayer, 1993; Kuhn & Pearsall, 1998) as well as a positive correlation between poor metacognitive skills and poor academic performance (Kruger, 1999; Dunning et al., 2003)

Some key question students can ask themselves to engage in helpful metacognitive exam-related practices are as follows (Tanner, 2012):

TestPrepTable

Reduction of Test Anxiety

Finally, research provides instructors with ways in which they can reduce the negative effects of test anxiety. The nature of test anxiety is two-fold in that it concerns both cognitive and physiological reactions to stress, although the cognitive component is the most detrimental in terms of exam performance. Test anxiety causes the student’s thought process to be severely impeded by focus on peer comparison, consequences of bad grades, and equating self-worth with test performance. While mild levels of test anxiety actually encourage learning and improve test performance, high levels of anxiety are an overall hindrance. Also, although high levels of test anxiety improve rote recall, they stifle the cognitive flexibility and creativity required for short answer and essay questions (Rasmussen & Chamley, 2012).

There are several steps an instructor can take to reduce student test anxiety. An important first step is when the instructor reviews exam material prior to the exam, is explicit about the degree of difficulty of a given exam, and does not monitor exams-in-progress (e.g. walking around the room and looking over shoulders) or interrupt/make clarifications after the exam has been distributed. In addition to these precautions, it is beneficial when the course syllabus contains an exam schedule, course expectations, and a concrete grading system and make-up policies. It may also behoove the instructor to consider giving exams with more frequency (so as to prevent overwhelming material coverage and harsh consequences) and to be sure that exams are written with the time frame in mind (Nilson, 2010).

 

—————————————-

References

Adey, P., & Shayer, M. (1993). An exploration of long-term far-transfer effects following an extended intervention program in the high school science curriculum. Cognition and Instruction, 11(1), 1-29.

Angelo, T.A., & Cross, P. K. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco, CA: Jossey-Bass.

Brown, P., Roediger H., & McDaniel, M. (2014). Make it stick: The science of successful learning. Cambridge, MA.

Cooper, J. L., Robinson, P., & McKinney, M. (1993). Cooperative learning in the classroom.

Changing College Classrooms (pp. 74-92). San Francisco, CA: Jossey-Bass.

Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83-87.

Harden, R.M., Laidlaw, J.M., & Hesketh, E.A. (1999). AMEE Medical Education Guide No 16: Study guides – their use and preparation. Medical Teacher, 21(3).

Khogali, S.E.O., Laidlaw, J.M., & Harden, R.M. (2006). Study guides: A study of different formats. Medical Teacher, 28(4), 375-377.

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121-1134.

Kuhn, D., Pearsall, S. (1998). Relations between metastrategic knowledge and strategic performance. Cognitive Development, 13(2), 227-247.

National Research Council (NRC) (1996). National Science Education Standards, Washington, DC: National Academies Press.

Nilson, L. B. (2010). Teaching at its best: A research-based resource for college instructors. San Francisco, CA: Jossey-Bass.

Rasmussen, J.B., & Chamley, R. (2012). Expert agreement related to test anxiety. Unpublished manuscript, Bethel University.

Stage, F.K., Kinzie, J., Muller, P., & Simmons, A. (1999). Creating learning centered classrooms: What does learning theory have to say? Washington, DC: ERIC Clearinghouse on Higher Education.

Sylwester, R. (1994). How emotions affect learning. Educational Leadership, 52(2), 60-65.

Tanner, K.D. (2012). Promoting student metacognition. Life Sciences Education, 11, 113-120.

Wolfe, P. (2010). Brain Matters: Translating research into classroom practice. Alexandria, VA: ASCD.

Understanding & Preventing Text Anxiety

by Jay Rasmussen

When students have excessively high levels of anxiety it is very likely that the instructor and student will receive invalid and unreliable information about content area learning. We can mediate this concern by understanding

  • components of test anxiety
  • indicators of test anxiety
  • behavioral manifestations of test anxiety
  • social and beyond-the-classroom consequences of test anxiety
  • testing stakes/increase of anxiety among American children
  •  anxiety among minorities and disabled students
  • anxiety and student environment
  • strategies for preventing test anxiety

Components of Text Anxiety

Test anxiety is comprised of two components – cognitive (worry) and physiological (body’s physical reaction). Physiological anxiety displays symptoms such as increased heart and breathing rate, increased body temperature (sweating), nausea, muscle cramping, dry mouth, and fainting. The cognitive components have a significantly more negative impact. Three types of thoughts often occur:

  1. Comparison with other students (e.g., “I’m not as smart as everyone else.”)
  2. Consequences of a poor performance on the test (e.g., “I’m going to flunk this test and I won’t ever graduate from college.”
  3. Effects of failure on their self-concept and identity (e.g., “I’m so dumb that I’m not worth anything to anyone.”

The physiological components of test anxiety are the result of cognitive aspects. One student “I felt I was ready for the test, but when it started my mind just went blank. Before the test started I felt sick. I just wanted to get out of there. I kept thinking what would happen to myself if I did poorly on this test, I just knew it would be awful because I was going to fail again. I thought I did just fine, but when the grade came back I got a ‘D’, I don’t know what happened. I am always feeling under pressure, my life is too hectic.” (Ross, 1992)

Indicators of Text Anxiety

The ways in which test anxiety exhibits itself may include, but are not limited to, social anxiety, test anxiety, or performance anxiety. One common indicator of test anxiety is a discrepancy between actual performance on an assessment and predicted performance based on classroom observations and performance on practice assessments (Huberty, 2009). Symptoms of test anxiety debilitate students’ ability to function in all areas of life not only during a testing situation, but possibly in the days or weeks prior to a test depending on the scope and importance of the test as well.

In an effort to control and minimize anxiety, students will often withdraw from evaluation situations, causing them to appear unmotivated, lazy, or incompetent when compared to their peers. Children and adolescents rarely seek help for feelings of anxiety, fearing the public attention or scorn that may coincide with treatment. If anxiety is not addressed and treated during childhood, it can continue to manifest itself students’ adult lives, consequently decreasing quality of life and opportunities for career advancement (Larson, ElRamahi, Conn, Estes, & Ghibellini, 2010).

Behavioral Manifestation of Test Anxiety

Research carried out by Plass and Hill (1986) indicates that there are four ways through which anxiety will manifest itself in student behavior:

  1. learned helplessness
  2. slow, off-task test taking
  3. slow, cautious, accurate test taking
  4. fast, inaccurate test taking

Emotional feelings of test anxiety can be further categorized into four distinct groups, each with its own set of consequences. Positive activating emotions, such as pride, can lessen feelings of anxiety and motivate students to try harder on assessments. Positive deactivating emotions, such as contentment, will help keep feelings of anxiety at bay following an assessment. Negative activating emotions, such as shame, may increase motivation to avoid future failures. Negative deactivating emotions, such as boredom or hopelessness, can debilitate students’ motivation to try completely. Students displaying all types of anxiety will experience reduced working memory capabilities, which lead to decreased performance on more complex or abstract tasks that draw on this level of functioning. Negative, anxious feelings tend to trigger the use of simple rehearsal and fact reliance procedures in place of problem-solving skills, often leading to decreased

Social and Beyond-the-Classroom Consequences of Test Anxiety

In addition to the academic consequences of test anxiety, students suffer from emotional and social consequences, such as decreased social functioning and impaired social development. According to Turner, Beide, and Turner (1993), there is a positive correlation between anxiety and each of the following: submissive behavior, failure to accept behavioral responsibility, expressions of masculinity, increased negative self evaluations, poor self-esteem, increased generalized anxiety, and fewer positive relationships with peers. Findings from Pekrun, Goetz, Titz, and Perry (2002) suggest that students who experience negative feelings of anxiety will rely more on external guidance from other peers and adults, rather than self-guidance of emotions and learning. Relying on external control then leads to increased feelings of anger, anxiety, and boredom, whereas depending on self-regulation can lead to increased positive feelings regarding learning and achievement.

Feelings of test-anxiety are often exhibited in ways that are socially unacceptable, such as auto-manipulation (rocking, playing with hair), object manipulation (pencil tapping, nail biting), and distracted or paranoid behaviors (looking around the room) which draws further negative attention from peers (Wren & Benson, 2004). It is clear that the effects of extreme test anxiety exist far beyond the walls of the classroom and the school; anxiety is a chronic and debilitating condition. Due to similar symptoms, there is a coexistence of test anxiety and depression – anxiety often serves as a precursor to depression in that it causes significant emotional problems and damage (Huberty,2009).

Interestingly enough, mild degrees of anxiety usually facilitate learning, but high anxiety levels usually hinder learning. Anxiety, on the whole, however, does not lower a student’s motivation to learn. Unfortunately yet understandably, the less able student has a higher level of test anxiety than does the more capable one. During testing, highly anxious students do better than less anxious ones at rote recall; they perform less well, however, in situations where flexibility of thought is required. Inadequate preparation is often the reason for anxiety displays during testing situations, the inadequate preparation being the result of the student’s uncertainty as to how to successfully prepare.

Testing Stakes/Increase of Anxiety Among American Youth

Typically, as the stakes of test increase, so does the intensity of the feeling of anxiety. Anxiety has been coined an “invisible disability” in that it is difficult to identify and its duration often includes a students’ entire educational career – it is a stable personality trait (Cheek, Bradley, Reynolds, & Coy, 2002). Studies conducted as recently as 2004 have shown that more than 33% of school age children experience some type or degree of test anxiety, this rate being higher than was found in previous studies of American school children. Although there is no known reason for the increase, one theory attributes it to the increase in standardized testing and in testing stakes for teachers, administrators, and schools (Sena, Lowe, & Lee, 2007). A second theory attributes it to the increase in testing at younger ages in schools (McDonald, 2001).

As high-stakes testing becomes more of a norm in education, increased pressure is placed on administrators, teachers, and students to perform well; this increased pressure can have negative effects on adults as well as students (Sena, Lowe, & Lee, 2007). Under the No Child Left Behind (NCLB) legislation, a minimum of 95% of all American school children in grades 3-8 will be tested annually and students in grades 10-12 will be tested at least once. Following this agenda, students will have taken 23 standardized tests by the time they finish eighth grade, equating to approximately 90 school days’ worth.

Anxiety Among Student Minorities and Students Who Are Disabled

Minority students, as well as those with learning disabilities, have been shown to experience heightened levels of test anxiety. Special attention must also be given to students with disabilities who experience test anxiety. Studies report that students with Learning Disabilities (LD) report more stress, nervousness, frustration, helplessness, concentration difficulties, and doubt than their peers without LD.

The most significant predictor of test anxiety reported by LD students is a greater prevalence of task irrelevant thoughts that interfere with demonstration of knowledge. Increased feelings of worry also lead to an inability to solve complex problems due to concerns about outside performance factors, and these feelings are often compounded by previous failure on academic evaluations caused by their disability. Similarly, students with Emotional Behavioral Disabilities (EBD) report more feelings of worry and nervousness than students without EBD (Sena, Lowe, & Lee, 2007). English Language Learners (ELL) who do not speak English as their native language also experience increased levels of test anxiety (McDonald, 2001). Special provisions should be arranged ahead of time to accommodate for the sensory needs of students receiving special education services, such as frequent breaks, shorter test segments, or dictation of the test (Syncamore & Corey, 1990).

Anxiety and Student Environment

One major factor contributing to test anxiety is the classroom environment and the tone set by teachers regarding testing (Larson, et al., 2010). Research carried out by Zatz and Chassin (1985) found a direct connection between increased feelings of test anxiety in classrooms perceived by students to have high threat evaluation settings, identified by feelings of competition and teacher control.

30 Ways You Can Address Text Anxiety

  1. To the person experiencing it, test-anxiety is real. Take it seriously.
  2. Recognize symptoms (flushed face, sweat, fear in the eyes, unable to focus, or sick feeling in the stomach)
  3. Be supportive and understanding. Students who identify a high frequency of positive academic feedback report higher self-concepts of ability levels; in contrast, students in classrooms where a high rate of criticism is identified hold lower self efficacy beliefs (Burnett, 1999).
  4. Be a friend to a person experiencing test-anxiety.
  5. Read and research the topic; learn as much as you can about it so you can help others.
  6. Find alternative measurement and assessment methods to evaluate a student.
  7. “An ounce of prevention is worth a pound of cure.” Be aware of situations in the classroom andprevent the first occurrence.
  8. Remember — test-anxiety can happen to anyone.
  9. Make grading decisions on the basis of multiple assessments.
  10. Let assessment become a familiar experience in your classroom.
  11. Be predictable in your expectations of students.
  12. Avoid timed tests if at all possible.
  13. Don’t spring tests on your students without warning.
  14. Consider providing practice assessment items and/or a study guide.
  15. Discuss the item type(s) found on a test.
  16. After an assessment, give positive and productive feedback to students in a timely manner to decrease their post-assessment anxiety and to show students the purpose of the assessment
  17. Help students with study skills (time management, goal setting, note taking, immediate review,
  18. Help students with test taking skills. These include pacing responses according to number of questions and amount of time given to complete the assessment, giving priority to questions you feel confident about, being aware of absolute qualifiers such as “always” and “never” as indicators of an incorrect response, taking notes in the margin, highlighting important information (Syncamore & Corey, 1990), and starting with the easiest questions to best use energy towards achievable tasks.
  19.  Have a good balance between standards-based or testing-based lessons/activities and authentic or project-based learning experiences.
  20. When creating an assessment, teachers should make sure that each assessment is comprised of a mix of easy and hard test items.
  21. Teachers should continually use a variety of assessment practices (such as observations, writing tasks, performance tasks, short quizzes, completion tasks, and multiple choice tasks) throughout the duration of a unit rather than only utilizing summative assessment at the end of a unit that has more weight toward the final grade.
  22. After an assessment, teachers should give students positive and productive feedback in a timely manner to decrease their post-assessment anxiety and to show students that the purpose of the assessment was to inform their learning.
  23. Students put forth more effort when they feel involved with the results of an assessment.
  24. Laughter decreases the hormone that causes stress in the body. Humor can be effective only if the instruction included the use of humor, the test has either no time limit or a very generous one, the humor is positive and constructive, the humor is appropriate for the test content and the test developer feels comfortable using humor.
  25. Relaxation exercises before a test can be helpful. Two well-documented techniques to reduce test anxiety and increase ability to focus are deep breathing and muscle relaxation; these two techniques have also been found to control physiological hyper-arousal including heart rate, respiration, and body
  26. Helping a student with negative self talk (cognitive restructuring) or other off task behavior can be helpful. A school councilor may be of assistance in this process.
  27. Positive self-talk is a well-documented technique that combats feelings of test anxiety. Self- talk is a metacognitive strategy aimed at increasing students’ ability to regulate their own thoughts and behaviors regarding academic assessment and achievement (Manning, 1990). Self-talk can be divided into two categories: positive and negative. Positive self-talk includes statements of optimism, encouragement, praise, or hope aimed at increasing self-esteem and self-perception, such as “Just stay calm,” “Everything will work out okay,” and “I can do well on this task.” Positive statements from important adults such as parents, teachers, and administrators are associated with increased positive self-talk in students, resulting in higher self-learner esteem in students (Burnett, 1999).
  28. The most effective treatment of test anxiety is one which combines study skills, relaxation techniques, and cognitive restructuring.
  29. The most effective strategies for preventing anxiety are multimodal in nature- i.e. they address the cognitive and physiological factors.
  30. Watch out for ways that anxiety can manifest in student behavior: learned helplessness; slow, off-task test taking; slow, cautious, accurate test taking; and fast, inaccurate test taking.

—————————————-

References

Berk, R. A. (2000). Does humor in course tests reduce anxiety and improve performance? College

Burnett, P.C. (1999). Children’s self-talk and academic self‑concepts. Educational Psychology in Practice,

Cheek, J.R., Bradley, L.J., Reynolds, J., & Coy, D. (2002). An intervention for helping elementary students reduce test anxiety. Professional School Counseling, 6(2), 162-164.

Dendato, K. M., & Diener, D. (1986). Effectiveness of cognitive/relaxation therapy and study skills training in reducing self-reported anxiety and improving the academic performance of text‑anxious students. Journal of Counseling Psychology, 22, 131-135.

Fisher, B. L., Allen, R. & Kose, G. (1996). The relationship between anxiety and problem solving skills in children with and without learning disabilities. Journal of Learning Disabilities, 29, 439-446.

Foster, S.K., Paulk, A., Riederer, B. (1999). Can we really teach test taking skills? New Horizons in Adult

Hancock, D. R. (2001). Effects of test anxiety and evaluative threat on students’ achievement and motivation. Journal of Educational Research, 94(5), 284‑290.

Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational

Huberty, T.J. (2009). Test and performance anxiety: Good communication and support can help minimize its negative impact. Principal Leadership, 10, 12-16.

Larson, H. A., El Ramahi, M. K., Conn, S. R., Estes, L. A., & Ghibellini, A. B. (2010). Reducing test anxiety among third grade students through the implementation of realization techniques. Journal of School

Manning, B.H. (1990). A categorical analysis of children’s self‑talk during independent school assignments. Journal of Instructional Psychology, 17(4), 208.

McDonald, A.S. (2001). The prevalence and effects of test anxiety in school children. Educational Psychology, 21(1), 89-101.

Pekrun, R., Goetz, T., Titz, W., & Perry, R. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist,

Plass, J. A., & Hill, K. T. (1986). Children’s achievement strategies and test performance: The role of time pressure, evaluation anxiety, and sex. Developmental Psychology, 27(1), 31-36.

Ross, D. B. (1990). Controlling school anxiety: A practical guide for counselors and teachers. Grayslake,

IL: Lake County College. (ERIC Document Reproduction Service No. ED03500)

Sena, J.D.W., Lowe, P.A., & Lee, S.W. (2007). Significant predictors of test anxiety among students with and without learning disabilities. Journal of Learning Disabilities, 40(4), 360‑377.

Speilberger, C.D.(2011). Test anxiety inventory. Retrieved from http://www.mindgarden.com/docs/TAISample.pdf

Stallworth‑Clark, R., Cochran, J., & Scott, J. S. (1998, November). Text Anxiety and effect of anxiety‑reduction training on student’s performance on the Georgia regent’s reading exam. Paper presented at the meeting of the Georgia Educational research Association, Atlanta, GA.

Supon, V. (2004). Implementing strategies to assist test‐‑anxious students. Journal of Instructional Psychology, 31(4), 292-296.

Syncamore, J.E., & Corey, A.L. (1990). Reducing test anxiety. Elementary School Guidance & Counseling, 24(3), 231-233.

Turner, B.G., Beidel, D.C., & Turner, M.W. (1993). Text anxiety in African American school children. School Psychology Quarterly, 8(2), 140‑152.

Vasey, M. W., El‑Hag, N., & Daleiden, E. L. (1996). Anxiety and the processing of emotionally threatening stimuli: Distinctive patterns of selective attention among high and low – test anxious children. Child Development, 67, 1173‐1185.

Williams, J. E. (1996). Gender‑related worry and emotionally test anxiety for high-achieving students. Psychology in the Schools, 33, 159‑162.

Wren, D.G., & Benson, J. (2004). Measuring test anxiety in children: Scale development and internal construct validation. Anxiety, Stress, and Coping, 17(3), 227-240.

Zatz, S., & Chassin, L. (1985). Cognitions of test‑anxious children under naturalistic test‑taking conditions. Journal of Consulting and Clinical Psychology, 53(3), 393-401.

Writing High-Quality Traditional Assessment Items

By Jay Rasumssen

One of the most challenging tasks for professors is providing high quality assessment that properly aligns with the learning outcomes of a given course. Some course outcomes are best assessed with performance measures and at times traditional assessment measures are the most appropriate. Creating both forms of assessment presents unique challenges. This particular piece focuses on the creation of high quality traditional assessments and it will help you avoid common problems associated with teacher-made traditional test items: shallow questioning, ambiguous questioning, excessive wording, extremes in difficulty, bias, wrong item format, inadequate planning, and validity and reliability.

One would think commercially prepared assessments would be higher quality than teacher-made assessments. Unfortunately, this is not always the case because not all publishing companies use fully trained test designers. There is a specific body of knowledge about test design and item creation that all test designers, professional or otherwise, should be aware of.  This post will assist you in understanding that body of knowledge and in preparing high quality of test/quiz items once you’ve determined that a particular learning outcome is best assessed by traditional methods. Specifically, we will focus on the work of Miller, Linn & Gronlund (2013) to address:

  • general guidelines for item writing,
  • advantages and disadvantages of each item type, with
    • writing guideline considerations for each item type, and
    • sample item(s) for each item type.

General Guidelines for Item Writing

  1. Relate items to instructional objectives.
  2. Write items with academic and non-academic vocabulary at the appropriate level.
  3. Write items as clearly as possible. Use a sentence structure that is clear and simple – keep it brief.
  4. Avoid using words that are ambiguous. (e.g. hot, cold, few,  many, large, small, high, low)
  5. Don’t lift items word for word from the text. Because the item is being taken out of context, the meaning may be lost.
  6. Don’t include items that are interrelated – i.e. students must be able to correctly answer one item before another item may be answered correctly.
  7. Write questions with one best answer. If it’s an opinion question, state whose opinion you are looking for.

For example:

Poor – A major cause of heart attacks is cholesterol.

Better – According to Dr. White, a major cause of heart attacks is cholesterol.

  1. Avoid negative questions when possible. Reason – the student needs to change normal thought processes and it will take more time and produce more errors.

For example:

Poor – T/F: A United States congressman is not elected for a 2-year term.

Better – T/F: A United States congressman is elected for a 2-year term.

  1. Don’t give the answer away, either in that question or a previous one.
  2. Avoid racial, ethnic, or gender bias.

 

Advantages and Disadvantages of Question Types

Short Answer Items

Advantages

  1. Require the student to produce rather than recognize an answer.
  2. Are easy to administer.
  3. Provide diagnostic information.

Disadvantages

  1. Require more time to score than many other item types.
  2. May not be effective at measuring higher level thinking skills.

Writing Considerations

  1. Can the items be answered with a number, symbol, word, or a brief phrase?
  2. Has textbook language been avoided?
  3. Have the items been stated so that only one response is correct?
  4. Are the answer blanks equal in length?
  5. Are the answer blanks at the end of the items?
  6. Are the items free of clues (such as a or an)?
  7. Has the degree of precision been indicated for numerical answers?
  8. Have the units been indicated when numerical answers are expressed in units?
  9. Have the items been phrased so as to minimize spelling errors?
  10. If revised, are the items still relevant to the intended learning outcomes?

Sample Items

(Completion Variety)

Lines on a weather map that join points of the same barometric pressure are called __________

(Direct Variety)

If the temperature of a gas is held constant while the pressure applied to it is increased, what will happen to its volume?  __________

True-False Items

Advantages

  1. Can cover a large amount of subject matter in a short-period of time. (students can answer three T-F items for every two multiple choice items)
  2. Can be scored quickly.
  3. Can be written easily from a salvage pool of multiple choice items.

Disadvantages

  1. Are most affected by guessing.
  2. Are most susceptible to ambiguity.
  3. Make it easier to cheat.
  4. Allow students to enter a pattern of responding without thinking.

Writing Considerations

  1. Can each statement be clearly judged true or false?
  2. Have specific determiners (e.g., usually, always) been avoided?
  3. Have trivial statements been avoided?
  4. Have negative statements (especially double negatives) been avoided?
  5. Have the items been stated in simple, clear language?
  6. Are opinion statements attributed to some source?
  7. Are the true and false items approximately equal in length?
  8. Is there an approximately equal number of true and false items?
  9. Has a detectable pattern of answers (e.g. T, F, T, F) been avoided?

Sample Items

(True-False Variety)

T    F    A virus is the smallest known organism.

(Cluster Variety)

Mary Ann wanted her rose bush to grow faster, so she applied twice as much chemical fertilizer as was recommended and watered the bush every morning. About a month later she noticed that the rose bush was dying.

T   F    The following principles are necessary in explaining why the rose bush is dying.

  1. A chemical compound is changed into other compounds by taking up the elements of water. (F)
  2. Semipermeable membranes permit the passage of fluid. (T)
  3. Water condenses when cooled. (F)
  4. When two solutions of different concentration are separated by a porous partition, their concentration tends to equalize. (T)

(Correction Variety)

If False Option is Selected, Students Must Replace the Underlined Word with Word to Make the Statement True

T    F    The green coloring material in a plant leaf is called chlorophyll.

Matching Items

Advantages

  1. Are especially effective at measuring learning of items, definitions, dates, locations, and events.
  2. Require a limited amount reading so many questions may be asked.

Disadvantages

  1. Require the student to produce rather than recognize an answer.
  2. May overemphasize memorization.
  3. Are difficult to write because items must be clustered.

Writing Consideration                                                                    

  1. Is the material in the two lists homogeneous?
  2. Is the list of responses longer or shorter than the list of premises?
  3. Are the responses brief and on the right-hand side?
  4. Have the responses been placed in alphabetical or numerical order?
  5. Do the directions indicate the basis for matching?
  6. Do the directions indicate that each response may be used more than once?
  7. Is all of each matching item on the same page?
  8. If revised, are the items still relevant to the intended learning outcomes?

Sample Items

Column A (Premise) Column B (Response)

  1. Air pressure A. Anemometer
  2. Air temperature B. Barometer
  3. Humidity C. Hygrometer
  4. Wind velocity D. Rain gauge  
  1. Thermometer
  2. Wind vane

Multiple-Choice Items

Advantages

  1. Effectively measure various levels of thinking.
  2. Avoid ambiguity because the thought is completed by the options.
  3. Can be used to provide diagnostic information.
  4. Are easily scored.
  5. Are the most popular item type among students.
  6. Can be controlled for difficulty by making the options more or less homogenous.
  7. Can be used effectively with drawings, maps, graphs, and visuals.

Disadvantages

  1. Are very time consuming to construct.
  2. Require more student time to complete than some other item types.

Writing Considerations

  1. Does each item stem present a meaningful problem?
  2. Are the item stems free of irrelevant material?
  3. Are the item stems stated in positive terms (if possible)?
  4. If used, has the negative wording been given special emphasis (e.g. capitalized)?
  5. Are the alternatives grammatically consistent with the item stem?
  6. Are the alternative answers brief and free of unnecessary words?
  7. Are the alternatives similar in length and form?
  8. Is there only one correct or clearly best answer?
  9. Are the distractors plausible to low achievers?
  10. Are the items free of verbal clues to the answer?
  11. Are verbal alternatives in alphabetical order?
  12. Are numerical alternatives in numerical order?
  13. Have none of the above and all of the above been avoided (or used sparingly and appropriately)?
  14. If revised, are the items still relevant to the intended learning outcomes?

Sample Items

(Direct Question Variety)

What is the function of the kidneys?

  1. Eliminate waste products.
  2. Improve the circulation of blood.
  3. Maintain respiration.
  4. Stimulate digestion.

(Incomplete Statement Variety)

Alternating electric current is changed to direct current by means of a

  1. condenser.
  2. generator.
  3. rectifier.
  4. transformer.

(Best-Answer Variety)

Which one of the following best illustrates the principle of capillarity?

  1. Fluid is carried through the stems of plants.
  2. Food is manufactured in the leaves of plants.
  3. The leaves of deciduous plants lose their green color in winter.
  4. Plants give off moisture through their stomata.

Interpretive Items

Advantages

  1. Measures ability to interpret various forms of written information (charts, graphs, maps, etc.)
  2. Possible to measure more complex learning outcomes than a single assessment item
  3. Greater depth and breadth in measurement of intellectual skills when multiple items are based on a single set of data
  4. Measurement of complex learning outcomes is not likely to be influenced by irrelevant information
  5. More structured than performance assessment tasks
  6. Ability to measure separate aspects of problem-solving

Disadvantages

  1. Difficult to construct (need to find new yet relevant and fair information; need to create questions that measure desired learning outcomes)
  2. Highly dependent on reading skills – poor readers are at a serious disadvantage
  3. Provides diagnostic rather than holistic information about problem-solving abilities
  4. Limited to measuring recognition-level learning outcomes

Writing Considerations                                                 

  1. Is the material to be interpreted relevant to the intended learning outcomes?
  2. Is the material to be interpreted appropriate to the students’ curricular experience and reading level?
  3. Have pictorial materials been used whenever appropriate?
  4. Does the material to be interpreted contain some novelty (to require interpretation)?
  5. Is the material to be interpreted brief, clear, and meaningful?
  6. Are the test items based directly on the introductory material (cannot be answered without it) and do they call for interpretation (not just recall or simple reading skills)?
  7. Has a reasonable number of test items been used in each interpretive exercise?
  8. Do the test items meet the relevant criteria of effective item writing?
  9. When key-type items are used, are the categories homogeneous and mutually exclusive?
  10. If revised, are the interpretive exercises still relevant to the intended learning outcomes?

Sample Item

  1. The following scatterplot shows the relationship between scores on an anxiety scale and an achievement test for science. Choose the best interpretation of the relationship between anxiety level and science achievement based on the scatterplot (Developed by the Web ARTIST Project https://app.gen.umn.edu/artist/).

 A. This graph shows a strong negative linear relationship between anxiety and achievement in science.

 B.  This graph shows a moderate linear relationship between anxiety and achievement in science.

 C. This graph shows very little, if any, linear relationship between anxiety and achievement in science.

 

Essay Items

Advantages

  1. Effectively measure the ability to organize ideas.
  2. Are easily constructed.

Disadvantages

  1. Require an expert to grade.
  2. Have low content validity because sampling is limited.
  3. Are often scored unfairly because of writing ability, length of paper, sequence of reading, and the “Halo Effect”.

Writing Considerations

  1. Are the questions designed to measure higher-level learning outcomes?
  2. Are the questions relevant to the intended learning outcomes?
  3. Does each question clearly indicate the response expected?
  4. Are the students told the bases on which their answers will be evaluated?
  5. Are generous time limits provided for responding to the questions?
  6. Are students told the time limits and/or point values for each question?
  7. Are all students required to respond to the same questions?
  8. If revised, are the questions still relevant to the intended learning outcomes?

Scoring Guidelines

  1. Prepare a short outline of the correct answers for each item.
  2. Before any grading is done, read through a sample of papers to get an idea of student responses.
  3. Grade without looking at names.
  4. Grade the same question on all papers before moving to the next question.

Sample Item

Restricted Response – 20 minute time limit, must demonstrate: thorough understanding of relationship (5pts.) and writing clarity/grammar (5pts.).

  1. The percentage of the workforce that is unemployed increased following an increase in interest rates. Briefly explain the relationship.

 

———————————————–

References

Miller, M. D., Linn, R. L., & Gronlund, N. E (2013). Measurement and assessment in teaching. Upper Saddle River, NJ: Pearson.

 

Summer Reading & Viewing: Recommendations from the faculty development team

When grading subsides and parts of your brain regain normal functioning, consider engaging faculty development resources you may have missed in the craze of the academic year. The faculty development team suggests these highlights:

Keith Brooks recommends:

  • Leah Fulton and Sam Mulberry’s Interviews with Students of Color (Short and Full Versions) These exit interviews provide great insight to some of the easily-overlooked issues relating to human dynamics. This is relevant in regard to changing demographics nationally and locally, and addresses our ongoing task of being a place that communicates effectively across cultures.

Kathy Nevins recommends:

Jay Rasmussen recommends:

  • Making Thinking Visible & Selected Harvard Thinking Routines Over the last several years a number of faculty have been using selected Harvard Thinking Routines to make thinking visible and to create a culture of thinking. When learners speak, write, or draw their ideas, they deepen their cognition. Thinking routines jump-start and make thinking visible so students can direct and improve their thoughts. This narrated PowerPoint and the related routines provide a good starting point.
  • Grouping for SuccessStrengthening Our Use of Groups & Group Contract Form Group work within the college classroom has the potential of being a powerful learning experience for students. Unfortunately, group work often turns into a less-than-desirable experience for students and professors alike. This video shares how groups can be successfully structured and used within a variety of instructional settings.
  • Using Flow Theory to Promote Engaged Learning Flow Theory, as developed by Mihaly Cskszentmihalyi, explains why humans fully engage in certain activities to the point of being fully absorbed, completely focused, and unaware of passing time. This narrated PowerPoint explores the origins of Mihaly’s research, conditions typically present when flow occurs, and how the theory can be applied to an instructional setting.
  • Power of Formative Assessment & Formative Assessment Strategies The use of formative assessment is one of the most exciting developments in the field of education in recent years. Used effectively, formative assessment has the power to double the speed of student learning. These resources provide an in-depth look at why to use formative assessment, guidelines for providing formative assessment, and specific formative assessment strategies.

April Schmidt recommends:

  • Schedule Like Beethoven: How to organize your workday like a genius One of the gifts of the academic calendar is an opportunity to re-approach our work each year with fresh strategies. This DIY Innovation Studio session is chock-a-block with wisdom from geniuses of history and tips and resources from Bethel profs. for making the most of your work day. Take a strategic, proactive approach to next year’s workdays and equip yourself to maximize efficiency and prevent burnout.

Sara Wyse recommends:

  • Top 21 1/2 Teaching Tips If you are short on time and want a great overview and synthesis of the research literature and what we’ve learned through Learner Perspectives on Instruction, check out the 21 1/2 teaching tips.  From course design to interacting with students, these tips will help energize you when you begin to transition back to thinking about teaching come August.