MEDICAL EDUCATION MULTIPLE CHOICE QUESTIONS- PART II (CLASSIFICATION, ITEM PREPARATION, ANALYSIS AND BANKING) Chaudhary N, Bhatia BD, Mahato SK, Agrawal KK MULTIPLE CHOICE QUESTIONS- PART II (CLASSIFICATION, ITEM PREPARATION, ANALYSIS AND BANKING) Chaudhary N1, Bhatia BD2, Mahato SK1, Agrawal KK3 In the previous issue, we discussed single best response MCQs. In this section, we will discuss other forms of MCQs, item preparation, pre-validation process, item analysis and banking of MCQs. TYPES OF MCQS IN OBJECTIVE TESTING 1. 2. 3. 4. 5. 6. 7. One best response type (most widely used and already discussed in previous issue) K-type MCQs (next most widely used MCQ) Matching type Relationship analysis type Case history type (Patient management problem) Pictorial type Multiple independent true-false selection type 1. Assistant Professor, Department of Pediatrics, Universal College of Medical Sciences & Teaching Hospital, Bhairahawa, Nepal 2. Professor, Department of Pediatrics, Universal College of Medical Sciences & Teaching Hospital, Bhairahawa, Nepal 3. Assistant Professor, Department of Medicine, Universal College of Medical Sciences & Teaching Hospital, Bhairahawa, Nepal For Correspondence Dr. Nagendra Chaudhary Assistant Professor, Department of Paediatrics Universal college of medical Sciences Bhairahawa, Nepal E-mail: enagendra@hotmail.com Journal of Universal College of Medical Sciences (2014) Vol.02 No.03 Issue 07 54 MEDICAL EDUCATION MULTIPLE CHOICE QUESTIONS- PART II (CLASSIFICATION, ITEM PREPARATION, ANALYSIS AND BANKING) Chaudhary N, Bhatia BD, Mahato SK, Agrawal KK K TYPE MULTIPLE CHOICE QUESTIONS (MCQS) Is it effective than standard single option MCQs? are similar to single response MCQs, as discussed in previous issue of the journal. K-type multiple choice questions are also called combined response multiple choice questions or multiple-multiple choice questions. Disadvantages: Anatomy of k- type MCQs K type MCQs have similarities with single response MCQs. It shares most of the advantages and disadvantages of a multiple choice questions (as discussed in previous communication). In items of the combined-response variety, one or more of the alternatives are correct answers; the remaining alternatives serve as distractors. The student is directed to identify the correct answer or answers by selecting one of a set of letters, each of which represent a combination of alternatives. The parts of K-type MCQs item are i) Stem ii) 4 alternatives (One or more are correct) iii) Incorrect responses are called as distractors which could be none or more There cannot be all incorrect responses in these type of questions. At least one option should be correct response. Key pattern for correct answer is: “e” in the given MCQ example a) b) c) d) e) When 1, 2, and 3 are correct When 1 and 3 are correct When 2 and 4 are correct When 4 is correct All of the above (1, 2, 3, 4) are correct It has a disadvantage of providing clues that help students with even partial knowledge of subject to guess the correct combination of alternatives. In the example above, a student can identify combination “e” as the correct response simply by knowing that alternatives 1 and 4 are both correct. MATCHING TYPE MCQs Directions: Each group of questions below consists of five lettered headings followed by a list of numbered words or phrases. For each numbered word or phrases, select ONE lettered heading that is most closely related to it and blacken the corresponding space in the answer sheet. Each lettered heading may be selected once, more than once, or not at all. Example: A. Increased metabolic activity B. Hyper insulinism C. Lack of storage of glycogen in liver D. Storage of abnormal glycogen in liver E. Decreased secretion by pituitary or adrenal gland 1. Adenoma of islets of Langerhans 2. Violent exercise 3. Hyperthyroidism 4. Simmond`s disease 5. Von Gierke`s disease 6. Epidemic hepatitis Key: A-2, 3 ; B-1; C-6; D-5; E-4 RELATIONSHIP ANALYSIS TYPE MCQs: Example: This type tests reasoning and understanding of the basic principles, or the cause and effect relationship. Edema is seen in: 1. Chronic liver disease 2. Nephrotic syndrome 3. Congestive cardiac failure 4. Kwashiorkor Purpose of K-type MCQs The k-type MCQs can be used to analyse, synthesize, and evaluate in order to select an answer. It can also be used to assess integration of knowledge and decision making. Guidelines for Constructing K-type Multiple-Choice Items: Direction: Each item below consists of two statements numbered (1) and (2) respectively. Please mark your response according to the instructions given below on the appropriate space in the answer sheet by blackening the space number. A- If (1) is correct and (2) is correct and (1) and (2) are causally related. B- If (1) is correct and (2) is correct and (1) and (2) are not causally related. C- If (1) is correct and (2) is not correct D- If (1) is not correct and (2) is correct E- If (1) is not correct and (2) is not correct The guidelines for constructing item (stem/distractors/key) Journal of Universal College of Medical Sciences (2014) Vol.02 No.03 Issue 07 55 MEDICAL EDUCATION MULTIPLE CHOICE QUESTIONS- PART II (CLASSIFICATION, ITEM PREPARATION, ANALYSIS AND BANKING) Chaudhary N, Bhatia BD, Mahato SK, Agrawal KK Example: Item I: (1) Staphylococcus aureus can cause food poisoning Because (2) Staphylococcus aureus produces an exotoxin called enterotoxin Key: A CASE HISTORY TYPE (PATIENT MANAGEMENT PROBLEM) MCQs Example: A 5 year child presented with upper GI bleed. Past history of PRBC transfusion twice. No significant family history. On examination, he had anemia, massive splenomegaly with normal liver span. No evidence of ascites. a) 1. 2. 3. 4. What is the most common likely diagnosis? Wilson disease Extra hepatic portal vein obstruction Galactosemia Tyrosinemia b) 1. 2. 3. 4. Which is the drug used to treat acute upper GI bleed? Somatostatin Vitamin K IV antibiotics Lactulose PICTORIAL TYPE MCQs Diagrams, charts, photomicrographs, ECGs, may be used to elicit the knowledge of structure, function, the course of clinical situation and interpretation of data Directions: The question below consists of a numbered list of descriptive words accompanied with a sectional diagram of rectum and anal canal with certain parts indicated with letters. For each of the numbered word(s) select the matching lettered part on the diagram and blacken the circle under the corresponding alphabet on the answer sheet: 1. 2. 3. 4. 5. Pecten Hilton`s white line Houstan`s valves Pectinate line Anal columns M U LT I P L E I N D E P E N D E N T T R U E - FA L S E SELECTION TYPE MCQs Candidate has to answer separately to each of the four or five choices and is instructed to respond to each choice by “yes” or “No”/ “True” or “False”. Example: Direction: To each of the alternatives for the item below you are to respond “Yes” or “No”. 1. Greatly increased level of serum alkaline phosphatase is commonly found in: a) Osteoporosis (No) b) Obstructive jaundice (Yes) c) Prostatic carcinoma with bony metastasis (No) d) Paget`s disease of the bone (Yes) 2. The consequences of breathing 5% CO2 in air include True False a) Increased in mixed venous PCO2 ( ) ( ) b) Increase in PH of alveolar blood ( ) ( ) c) Increase in alveolar ventilation ( ) ( ) d) Decrease in cerebral blood flow ( ) ( ) e) Shift of the O2 dissociation curve to right ( ) ( ) ITEMS CAN BE PREPARED BY a) An individual: the quality of item is poor b) Group of teachers: It is brainstorming and produces better quality of question. c) a +b, i.e., itemsproduced by an individual teacher are usually incomplete and these can then be open for group discussion to remove ambiguity, providing good quality distractors and improve language so ultimately a good quality item is framed. Time for preparing an item While preparing for lesson or immediately after taking a lecture on a particular topic. Best way to derive distractors: Derive distractors from Journal of Universal College of Medical Sciences (2014) Vol.02 No.03 Issue 07 56 MEDICAL EDUCATION MULTIPLE CHOICE QUESTIONS- PART II (CLASSIFICATION, ITEM PREPARATION, ANALYSIS AND BANKING) Chaudhary N, Bhatia BD, Mahato SK, Agrawal KK student`s incorrect responses and also from misconceptions evident in their answers. This is done while a teacher is evaluating answer sheet. Questions with item difficulty of 50-60% are considered good, 30-70% are acceptable and those with indices above 70% or below 30% may require modification. Further improvement of an item by trying on group of students followed by: Questions with discrimination index of 0.35 and above are considered as excellent, 0.25-0.34 good, 0.2-0.25 acceptable and those below 0.2 require modification. a) Discussion after they have been answered by students. b) Advantages outweigh the disadvantages, for eg, ambiguities, errors in content, errors in construction, instances of contradictory teaching, triviality of content. The improved item can be used in subsequent tests. Other way to improve an item is through an item analysis. ITEM ANALYSIS - Identifies deficiencies in questions Questions are looked for importance of content. If not, revise questions to achieve desirable level of difficulty. If weaker students answer correctly in greater number, then check the keying error, ambiguous in statement or misleading distractors. Pre validation process INTERPRETATION OF ITEM ANALYSIS After an MCQ test has been administered, the students` responses are evaluated. Item analysis includes: 1) Difficulty index or facility value 2) Discrimination index (the discrimination power of an item) 3) Distractor functionality or effectiveness This will tell us whether MCQ, -Is too easy or too difficult -How well it discriminates between high and low scorers on test -Whether all alternatives functional as intended We should generally choose items at the 50% level of difficulty and those with highest discriminating power. It is good practice to record item analysis data on a card and as well maintain in computer for banking and further improvement as well. Item analysis procedure We shall use 60 test papers to illustrate the steps 1. Arrange the 60 test papers results in rank order from the highest to lowest scores. 2. Select one third of the papers with highest scores (i.e., the first 20) and we call them as the upper group. 3. Select another one third papers with low scores (i.e., the last 20) and we call them as the lower group. 4. Set aside the middle thirdi.e., middle 20 values. 5. For each item, prepare a frequency table by counting the number of students in the upper group who selected each alterative; similarly count for lower group also. Example: Item no No of responses in No. of responses in Alternatives* Upper group Lower group A B C D E No response Total response (T) *Encircle the correct answer for each item 6. Calculate the item difficulty, (P) by using the formula given below: H + L x 100 P = -------------T Where, H= number of correct responses in the upper group L= number of correct responses in the lower group T= Total number of responses in both groups Journal of Universal College of Medical Sciences (2014) Vol.02 No.03 Issue 07 57 MEDICAL EDUCATION MULTIPLE CHOICE QUESTIONS- PART II (CLASSIFICATION, ITEM PREPARATION, ANALYSIS AND BANKING) Chaudhary N, Bhatia BD, Mahato SK, Agrawal KK Note: P is usually expressed as a percentage. The lower the percentage figure, the more difficult is the item. 7. Calculate the index of discrimination (d) of item by using the formula, 2 x (H - L) d = ---------------------T “d” is reported as a decimal fraction. Maximum positive discrimination power is indicated by 1.00 and minimum is 1.00 (negative discriminant). Interpretation of Analysis data 1. Difficulty index: 30% IDEAL 40% 50% 60% 1. Discrimination index: Revise Acceptable 0.15 Too easy Acceptable Too difficult or defective 0.20 70% Good 0.25 Excellent 0.35 HOW TO ORGANIZE A MCQ BANK?  Creation of good questions is not an easy task  To construct a fair MCQ paper is still more difficult The practice of creating new questions each time and discarding them after examination is over, is WASTE of effort and INEFFICIENT use of an academician`s time. Banks of questions increase efficiency in the use of MCQs and improvement of their quality. B) Assign a number to each question and code to identify nature of question Fig: 1 2 3 4 Type of question Body Assigned in series Computerized banking (Database) or store cards can be used. This task should not be taken lightly. Cooperation of all but commitment of one is required in proper storage of question banks. REFERENCES: 1. Cox KR, Ewan CE. The medical Teacher. London, Churchill Livingston, 1982. Chapter 37. Writing multiple choice questions. Bandarnaike RC, Cox KR. Chapter 38. How to construct a fair multiple Choice question paper? Cox KR. Chapter 39. What can teachers and students get out of multiple choice questions. Cox KR 2. Cox KR. How did you guess? Med. J. August 1976; 1: 884-86. 3. Ebel RL. Essentials of educational management: Englewood Cliffs; New Jersey, prentice-Hall Inc. 1972 4. Anderson J. For Multiple choice questions. Medical teacher 1979;1: 37-42. http://dx.doi.org/10.3109/01421597909014340 http://dx.doi.org/10.3109/01421597909010580 PMid: 24483175 5. Picketting G. Against multiple choice Questions. Medical teacher 1979; 1: 84-86. http://dx.doi.org/10.3109/ 01421597909019397 PMid:24479932 6. Hubbart JP, Clemans WV. Multiple Choice Examinations in Medicine. Philadelphia: Lea and Febiqer; 1961. HOW CAN QUESTIONS BE STORED? 7. Hubbart JP. Measuring medical education. Philadelphia: Lea and Febiqer;1981. A) Several classifications could be used: Answers to the previous issue MCQs 1. Subdivision of subject: Eg, in Pediatrics- Neonatology, Nutrition, Growth and development, haematology etc. In Anatomy: Gross anatomy, histology, embryology etc. 2. Body systems: Cardiovascular, Respiratory, Central nervous systems, etc. 3. Level of knowledge to be tested: Recall, Application, Understanding 4. Type of item: Single response, multiple response, true/false, reason/assertion, matching etc. 1. It includes double negatives which should be avoided. Item could be as: Which of the following is a loop diuretics? 2. It gives additional clues like pneumothorax and hydrothorax.Item could be as: An intercostal tube may be placed in a patient with flail chest to… 3. It includes repetition of similar words in each distractors. Stem could be written as: Protein requirement of Indian Journal of Universal College of Medical Sciences (2014) Vol.02 No.03 Issue 07 58 MEDICAL EDUCATION MULTIPLE CHOICE QUESTIONS- PART II (CLASSIFICATION, ITEM PREPARATION, ANALYSIS AND BANKING) Chaudhary N, Bhatia BD, Mahato SK, Agrawal KK reference man in gm/kg/day is: a) 1 b) 2 c) 3 d) 4 4. It includes repetition of the word congenital in Distractors (a) and (b) which should be avoided. 5. The stem is too long. 6. It includes EXCEPT and none of the above, which should always be avoided. 7. The stem gives a clue (an), so the students can guess as an antidepressant. 8. The distractors are not of same length in this item. 9. The numerical values in distractors should be similar, eg., 1.3, 2.3, 3.3, 4.3 10. In this item posterior pituitary and pars nervosa are similar, which should not be used. ADVERTISEMENT CHARGES S.NO PAGE RATE (NC) 1. FULL BACK COVER 15000 2. FULL INSIDE OF FRONT COVER 12000 3. FULL INSIDE 10000 4. HALF SIDE 8000 5. QUARTER 4000 Journal of Universal College of Medical Sciences (2014) Vol.02 No.03 Issue 07 59