No./Title: COM-G06/ Rekey Guidelines for Preclinical Exam Questions	Resp. Office: Medical Education Approval Body: CUME	Effective Date: 01/27/25
Category: COM/UME	Last Review: N/A	Next Review: 01/27/28
Contact: Michael Whitt, Ph.D. Assoc. Dean for Medical Education	 901-448-4634	 mwhitt@uthsc.edu
Related Policy: N/A

Rationale: Testing student knowledge and their ability to integrate multiple concepts presented in-class and in self-studies during the preclinical curriculum is key to evaluating whether a student has gained sufficient understanding of these materials to progress in the curriculum and their preparation for the USMLE Step 1 exam. The following rekey guidelines were developed to ensure fair assessment of student understanding of materials presented and were decided upon after several meetings and much discussion involving faculty and student representatives of the CMDCS and approved by the CUME.

Procedures:

1. Simple errors in which correct answers were originally mis-keyed are subsequently corrected and changes are made in ExamSoft.
2. Questions that statistically perform inadequately are reviewed. The table below is used as a guide for possible rekeys by the course director or the lecturer/content expert.

Total % Correct	Disc Index and/or Pt Biserial < -0.10	Disc Index and/or Pt Biserial -0.10- 0.0	Disc Index and/or Pt Biserial 0.01- 0.099	Disc Index and/or Pt Biserial 0.10-0.20	Disc Index and/or Pt Biserial 0.21 -0.30	Disc Index and/or Pt Biserial ≧0.30
0.00-29.99%	Re-Key*	Re-Key*	Re-Key*	Re-Key*	Re-Key*	Re-Key*
30.00-39.99%	Re-Key*	Re-Key*	Re-Key*	Re-Key*	Re-Key*	No re-key
40.00-49.99%	Re-Key*	Re-Key*	Re-Key*	Re-Key*	No re-key*	No re-key
50.00-59.99%	Re-Key*	Re-Key*	Re-Key*	Re-Key*	No re-key*	No re-key
60.00-69.99%	Re-Key*	Re-Key*	Re-Key*	No re-key*	No re-key	No re-key
70.00-79.99%	Re-Key*	Re-Key*	Re-Key*	No re-key	No re-key	No re-key
80.00-89.99%	Re-Key*	Re-Key*	No re-key*	No re-key	No re-key	No re-key
90.00-94.99%	Re-Key*	No re-key*	No re-key*	No re-key	No re-key	No re-key
95.00-100.00%	No re-key*‡	No re-key*‡	No re-key*‡	No re-key*‡	No re-key*‡	No re-key*‡

* Do not use on future exams without editing ‡ Consider changes to increase difficulty

• Rekey any question if less than 30% of the class answers correctly, regardless of the point biserial statistic of the item.
  Rationale: if less than 30% of the class got the question right, it is possible that the tested concept/information was not taught sufficiently well.
• Rekey if 30-39.99% of class answer correctly if the point biserial is below 0.30
  Rationale: questions in this range should be very highly discriminating
• Rekey if 40-49.99% of class (or less) answer correctly if the point biserial is below 0.20
  Rationale: questions in this range should be highly discriminating
• Rekey if 50-59.99% of class (or less) answer correctly if the point biserial is below 0.20
  Rationale: questions in this range should be highly discriminating
• Rekey if 60-69.99% of class (or less) answer correctly if the point biserial is below 0.1
  Rationale: questions in this difficulty range should be discriminating; should be on the upper end of this discrimination range or higher
• Rekey if 70-79.99% (or less) of class answer correctly if the point biserial is below 0.1
  Rationale: questions in the difficulty range do not have to be highly discriminating, but should have a positive point biserial
• Rekey if 80-89.99% (or less) of class answer correctly if the point biserial is below 0.00
  Rationale: questions in this difficulty range are not very discriminating, but negative point biserial is indicative of a question that might not be fair
• Rekey if 90-94.99% (or less) of class answer correctly if the point biserial is below -0.10
  Rationale: such a strongly negative point biserial indicates that the question was flawed and unfair
• Rekey if 95-100.00% no rekey is necessary
  Rationale: questions this easy do not discriminate as well; statistics are not very meaningful

To ensure fairness and consistency across the curriculum, course/module directors should consider the broader context of overall student performance when finalizing rekey decisions. This includes factors such as the assessment average for that course, the overall course average, and any other extenuating factors that may have significantly impacted student performance.

Decision on Rekey types:

No question should be removed from an exam, numerator AND denominator, because some students would lose credit for questions they may have gotten right because they knew the material. Rekeys should fall into one of the following categories:

All-key: if point biserial is less than 0.15, all students should receive credit for getting the question correct

Rationale: To simply remove a question from an exam (reducing both the numerator and denominator for grade calculation) punishes the students for a mistake made by the faculty (writing a poor exam question). If a question is too hard, poorly written, confusing, etc., many students will spend an inordinate amount of time attempting to answer the question. This likely impacts their performance to some degree on the remainder of the exam. Giving bonus credit to those that get one of these questions right while removing it from the other student’s exam calculation was illogical because it would typically be rewarding some students for getting a question right for the wrong reasons) while punishing those that did not get the question right. Finally, giving all students credit for a bad question has very little impact on a student’s overall exam grade (most exams have 110 or more items, so it is less than one percentage point, but it has a very positive impact on student morale. Moreover, it might encourage faculty to put forth better effort writing their exam questions to avoid giving away points to the students.

Double-key: if there are two answer choices that are both correct, students that gave either of the correct answers should receive credit for getting the question correct

Rationale: this is obvious and needs no explanation

Remove with Bonus: if point biserial was 0.15 or higher and the question had only one correct answer, the question can be removed from the test (denominator) with the students that answered the question correctly retaining credit for getting that question correct

Rationale: if the question has a strong point biserial, most of the students that got the answer right likely did so because they knew the material. This method gives those students credit for knowing the material and is fair to those that got the question wrong because the question will not count against them.

APPROVAL HISTORY

Effective: 01/27/25

Revised: N/A