DOI: 10.1148/rg.275065208
RadioGraphics 2007;27:1511-1517
© RSNA, 2007
Education Techniques for Lifelong Learning
Designing Learning Experiences1
Jannette Collins, MD, MEd
1 From the Department of Radiology, University of Wisconsin Hospital and Clinics, E3/311 Clinical Science Center, 600 Highland Ave, Madison, WI 53792-3252. Received December 21, 2006; revision requested March 26, 2007, and received April 2; accepted April 4. The author is editor for Seminars in Roentgenology but has no other financial relationships to disclose.
Address correspondence to the author (e-mail: jcollins{at}uwhealth.org).
 |
Abstract
|
|---|
Creation of significant learning experiences follows basic steps of instructional design related to situational factors, goals and objectives, feedback and evaluation methods, teaching and learning activities, alignment of the preceding elements, and course evaluation. Goals should reflect what students will learn at the end of the course and what will still be with them several years later. Objectives should focus on learner performance, not teacher performance, and on behavior, not subject matter; there should be only one learning outcome per objective. Students learn more and retain their knowledge longer if they acquire it in an active rather than a passive manner. The situational factors, goals and objectives, feedback and evaluation, and teaching and learning activities should all reflect and support each other. The act of course evaluation closes the educational loop of design, implement, evaluate, and modify.
© RSNA, 2007
|
Introduction
|
|---|
Radiologists participate in many types of educational activities as both learners and teachers. As teachers, radiologists plan learning experiences for large and small groups of medical students, residents, colleagues, and other health care professionals.
The likelihood that a learning experience will be effective is increased when the experience follows a carefully prepared plan. Although flexibility is important for the optimal acquisition of knowledge, so are carefully designed learning experiences. Such experiences are founded on several key components: (a) situational factors; (b) goals and objectives; (c) feedback and evaluation; (d) teaching and learning activities; (e) integration of the first four elements, a through d; and (f) course evaluation (1).
The article describes six steps that radiologists can follow in designing an effective learning experience. It concludes with a table outlining each step—a template that can be used by radiologists to apply the steps in actual practice.
|
Situational Factors
|
|---|
Situational or environmental factors refer to the context of the learning experience, the subject matter, the learners (or students), and the teachers. The purpose for developing a learning experience and the expectations placed on that activity by an institution, profession, or society relate to the context of the learning experience. For example, a radiology review course targeted to residents preparing for a certifying examination has a distinct purpose. Such a course generally provides a concise but thorough overview of each subspecialty area in radiology, and typically it is strategically planned to occur before the time of the examination. The students are highly motivated and expect the course to be focused on content relevant to the certifying examination; they also anticipate that the learning activities will be dynamic and delivered at a pace compatible with covering a large amount of content in a short time. Deviation from this model or inclusion of controversial or theoretical content would be incongruous with learner expectations and would likely detract from the perceived effectiveness of the course. The learners constitute a mostly homogeneous group of individuals who have passed requisite written examinations, have completed a similar length of radiology training, and share a common goal of passing the certifying examination. Knowing this information about the students is very helpful for planning the learning experience. Unfortunately, in many teaching situations this type of information about learners is difficult to obtain.
An instructor’s attitudes, beliefs, and values about teaching and learning; level of knowledge of the subject matter; experience with teaching and working with specific student populations; interest in teaching; and effectiveness as a teacher all influence the learning experience. In the example posed above, a teacher who understands the learners’ anxieties about the upcoming examination and their expectations for the course is a better match for teaching the material than is an instructor who does not share this understanding. If the students expect the course to provide clear answers and the instructor’s teaching style is to present open-ended problems with undefined answers, both parties will likely be dissatisfied with the learning experience. One way to avoid this conflict is to specifically inform the learners at the outset how the course will be structured. Students then have the option to alter their expectations or, in the case of voluntary activities, decline to participate in the course.
|
Goals and Objectives
|
|---|
The terms goals and objectives are often used synonymously. However, a goal differs substantially from an objective. In the field of education, a goal describes a global learning outcome and may comprise several, specific, subordinate learning objectives (2). A learning objective is a statement of one specific performance of what learners will be able to do when they have completed the educational activity. Achievement of learning objectives contributes to attainment of a goal. For example, the goal of this article is for readers to understand the steps involved in designing a variety of effective learning or educational experiences. The three objectives are for the student to be able to (a) describe six components of an effective learning experience, (b) describe the importance of integrating these components, and (c) design a 1-hour radiology course that incorporates the components of an effective educational experience.
Learning objectives guide the selection of content, development of an instructional strategy, development and selection of instructional materials, and construction of tests and other instruments for assessing and evaluating student learning outcomes. Objectives should focus on learner performance, not teacher performance, and on behavior, not subject matter. Good objectives identify only one learning outcome, are consistent with course goals, and are precise.
Clark (3) has described the "ABCs" of learning objectives: Audience, Behavior, and Conditions/criteria. The instructor or designer of learning experiences must craft each objective by considering the questions directed toward these essential elements: Audience, who is the target of this objective and what are the learner’s characteristics; Behavior, what behavior is expected from the student to show that he or she has learned the material; and Conditions/criteria, under what conditions will the learner be expected to demonstrate his or her knowledge (3). For example, will the student be given clinical information, radiologic images, and reference material, or must he or she perform from memory? In addition, the objective should specify how well a learner must perform for his or her behavior to be judged adequate. These expectations can be given in a statement that indicates a degree of accuracy or a quantity or proportion of correct responses. Behavior should be defined with specific action words, such as "identify," "describe," "detect," "recognize," "demonstrate," "state," "define," "compare and contrast," and "list." Words such as "learn," "understand," "grasp," "appreciate," and "know" are vague and make it difficult to determine whether the student has learned the material. For example, to demonstrate achievement of the learning objectives for this article, the reader must describe (an action word) six (precise number) components of an effective learning experience and design (action word) a 1-hour (precise number) radiology course.
In 1956, Bloom (4) published a taxonomy of cognitive learning as a hierarchy of knowledge, comprehension, application, analysis, synthesis, and evaluation. Educators have adopted Bloom’s taxonomy for test development (5,6) and simplified and collapsed it into three general levels (7). The three levels include (a) knowledge (recall or recognition of specific information), (b) combined comprehension and application (understanding or being able to explain in one’s own words previously learned information and using new information, rules, methods, concepts, principles, laws, and theories), and (c) problem solving (transferring existing knowledge and skills to new situations).
There is a fine line between choosing objectives that reflect an important and meaningful outcome of instruction, those that trivialize information into isolated facts, and objectives that are extremely vague. A good objective should give different people the same understanding of the desired educational outcome. Many instructors take a content-centered approach to developing objectives ("I want students to learn about topic X, topic Y, and topic Z"). This approach generally overemphasizes "understand and remember" kinds of learning, which are important, but when teachers take a learning-centered approach, they usually identify several additional skills, such as critical thinking, how to creatively use knowledge from the course, how to solve real-world problems, changing the way students think about themselves and others, and recognizing the importance of lifelong learning (1). The desired outcome of many educational programs requires that participants do more than recall facts. Programs should be designed to enable learners to apply knowledge or skills. Therefore, the objectives and test questions of the program should reflect different levels of learning. Thoughtfully written objectives are critical to the construction of appropriate test questions and in ensuring adequate assessment of intended learner competence. Test questions written to assess acquisition of knowledge (lower-level learning) would not be appropriate to test competent performance of objectives reflecting comprehension (higher-level learning). For example, a test question asking the learner to recognize benign dermal calcifications on a mammogram does not test his or her problem-solving ability. A question that provides specific patient information and imaging data (patient vignette) and asks the learner to choose the most appropriate management is one that tests problem-solving skills. Such patient vignettes offer several benefits in addition to assessing application of knowledge. Because patient vignettes require problem solving, they increase the validity of the examination. Such test questions are more likely to focus on important information rather than trivia. Lastly, they help identify examinees who have memorized facts but are unable to use the information effectively.
|
Feedback and Evaluation Methods
|
|---|
The terms feedback and evaluation are often used interchangeably (8), which leads to confusion about these concepts. Feedback has been defined as presenting information, not judgment, and is formative. It allows the learner to remain on course in reaching a goal. Evaluation has been defined as coming after the fact and as presenting judgment about how well or poorly the learner met a given goal; evaluation is summative. However, in the case of medical education, judgment is almost always assigned to feedback information. Positive feedback sounds good and negative feedback sounds bad. Often, feedback elicits an emotional reaction. When handled poorly, feedback may inhibit giving or receiving feedback in the future.
Ende (8) offers guidelines for giving feedback: (a) feedback should be undertaken with the teacher and learner working as allies, (b) it should be expected, (c) it should be based on first-hand data, (d) it should be regulated in quantity and limited to behaviors that are remediable, (e) it should be phrased in descriptive nonevaluative language, (f) it should deal with specific performance not generalizations, (g) it should offer subjective data, and (h) it should deal with decisions and actions rather than assumed intentions or interpretations. Fink (1) describes high-quality feedback as having the characteristics of "FIDeLity": (a) Frequent (daily, weekly, or as frequently as possible), (b) Immediate (given as soon as possible), (c) Discriminating (with the differences between poor, acceptable, and exceptional work clearly explained), and (d) Loving (delivered empathetically).
Formative feedback is important in medical education when it serves to help a learner reach a goal. Reaching a goal helps the student feel good about his or her performance and builds learner self-confidence. When effective feedback is received, the student identifies the giver of feedback as someone who is concerned with the learner’s success. This perception leads to a trusting, mutually respectful relationship. Such a relationship enhances the learning environment on many levels for both student and teacher.
Summative evaluation is often a necessary component of a learning experience. Formal examinations consisting of multiple-choice questions are commonly used to measure a learner’s mastery of educational objectives. Such examinations are often used to assign a grade or promote a student to a higher level. The higher the stakes are, the more important it is that such examinations be valid, reliable, and fair. Much has been written about development of effective multiple-choice questions (9). Multiple-choice questions are better for assessing lower levels of learning and, when well written, can accomplish this task quite well. However, they are generally not as effective for measuring higher levels of learning. They are better for assessing whether students learned the presented material and not very good for determining whether the students are able to apply that material to future activities. For example, a multiple-choice examination given after completion of a course on quality in radiology practice might have items that address issues of radiation dose, magnetic resonance imaging safety, institutional and federal regulatory requirements, and components of systems analysis. To understand how well a learner can apply the course content, the teacher must ask "In what kind of situation do I expect students to need or be able to use this knowledge" and then create a question or problem that replicates this real-life context as closely as possible (1). The instructor in the example above might ask learners to imagine that they will be charged with developing a quality assurance program for their radiology practice. The teacher might then pose a series of questions. What policies and procedures must be established for the practice to be in compliance with institutional and federal regulations? What type of training will be required of faculty and staff? Who will monitor equipment applications that will allow the most effective and safest patient care? How can the current systems be evaluated for their effectiveness, and how can systems problems be resolved? What resources will be needed to analyze systems problems?
Teachers must share with students the criteria and standards used to assess learners. The criteria refer to the traits or characteristics of high-quality work. Standards refer to how good the work must be to be judged acceptable, good, or exceptionally good. Sharing these expectations with students ensures that the instructor has thought about the criteria and standards that will be applied.
Self-assessment has become recognized as an important component of learning evaluation. Lifelong learning and self-assessment constitute one of four components of the American Board of Radiology’s Maintenance of Certification Program (10). Early in their training, radiologists should learn how to develop appropriate criteria for evaluating and assessing their own work to be prepared to meet the requirements for self-assessment that they will face later in their professional careers. Self-assessment can be performed in groups or individually. Portfolios, an emerging tool conducive to self-assessment, are now a required part of the assessment of radiology residents in Accreditation Council for Graduate Medical Education (ACGME)–accredited programs (11). Portfolios are similar to diaries or journals and represent a collection of statements and other materials that document a learner’s process of self-assessment. In the "1-minute paper," the teacher poses a short, but well-focused question for students to answer. Examples are "What is the most important thing you learned today?" and "What part of the course is confusing to you?" This exercise helps both the student, who can recognize his or her knowledge gaps, and the teacher, who can appropriately modify the learning activity.
|
Teaching and Learning Activities
|
|---|
The majority of learning activities in a medical education course are lectures. However, lectures often lack interactivity, and the learner is often a passive recipient of knowledge. An emerging concept in medical education, active learning, is slowly creeping into radiology education. Students learn more and retain their knowledge longer if they acquire it in an active rather than a passive manner. Examples of interactive learning strategies include simulations, small-group problem solving, case studies, games, and role playing. An audience response system can be effectively used to allow active learning in large lecture settings. Other techniques for making large group learning more interactive include (a) using pre-tests and posttests (with either slides or paper to present the materials to be covered), (b) asking questions (the audience is given questions about the content of the presentation before it begins and told to write down their answers), (c) using voting cards (with either true/false cards, numbered cards, or colored paper strips), (d) reading (the audience receives a passage to read and is asked for comments), (e) implementing buzz groups (the audience is asked to discuss a problem with two to four neighbors for 3–10 minutes, and then some groups volunteer or one group is selected to present its consensus to the entire audience), (f) using a mini-quiz (the audience members self-test their knowledge either before, after, or during the presentation), (g) posing a progressive problem (a patient scenario is presented in a progressive sequence; parts of the history are followed by questions about what is important or missing), (h) using discussion groups (panel members present mini-lectures or vignettes and solicit commentary from other panel members and the audience at large), (i) holding debates (expert facilitators present opposing viewpoints and ask the audience to vote or discuss issues), (j) acting out a patient-related intervention (a patient relates the history of his or her disorder, which is often accompanied by role playing), (k) presenting video vignettes (questions about the video trigger discussion with a panel or the audience at large), (l) planting "standardized" patients in the group (standardized patients give the appropriate responses on cues from the audience; eg, history, effects of therapy), (m) using a Phil Donahue style (a talk show host technique in which the lecture setting is in a U-shape, and the facilitator or teacher is allowed to move within the center for closer contact with the audience and more interactivity), and (n) distributing question cards (cards are passed to the audience and attendees are asked to jot down any questions that arise during the presentation; at given pauses, the cards are collected and given to a moderator who establishes which will be of most value and asks the questions of the speaker) (12).
The above-mentioned activities generally take more time to conduct than do traditional lectures. Typically, less content can be covered when interactive learning techniques are used. Because lectures have traditionally been used to introduce course content, the use of interactive techniques will require alternative means of delivering content to learners. Students could be instructed to read specific material from handouts, textbooks, articles, compact disks, or a course-specific Web site before attending the course. Students can be directed to go to selected Web sites that contain course-relevant content. The content of the assigned reading should be aligned with the goals and objectives of the course. Requiring students to read an entire textbook may be both impractical (depending on the length of the textbook) and unfocused. If a student would need 60 hours to read a specific textbook, it would be impractical to assign that book as required reading 1 week before a course begins. Teachers may need to direct students to specific chapters of a textbook, specific pages in one or more chapters, parts of journal articles, or areas on a Web site. The closer the content is to the goals and objectives of the course, the more motivated students will be to read the material and the better the material will prepare the student for his or her participation in the course’s active learning activities.
|
Integration of Goals, Evaluation, and Teaching and Learning Activities
|
|---|
Situational factors, goals and objectives, feedback and evaluation methods, and teaching and learning activities should be aligned. Misalignment or nonintegration of components can derail the learning experience. For example, an audience of 500 (situational factor) would not integrate well with instructing students how to perform stereotactic breast biopsy (teaching and learning activity). If the goal is for students to be capable of performing stereotactic breast biopsy (according to recommended standards and guidelines), a multiple-choice question test (evaluation method) would not accurately assess whether the student can adequately execute the procedure. Adequate evaluation requires observing the student doing the procedure; therefore, the assessment must involve observation of student performance. With multiple-choice questions, instructors can determine whether a student understands certain procedural and management issues, but they cannot assess the student’s ability to actually perform the procedure. The need to observe performance of a medical skill is analogous to the requirement for a driving test to obtain an initial driver’s license. Adequate completion of a paper test will only partially determine whether a person is competent to drive a car. A course goal of students being able to think critically about ethical issues faced in radiology practice would be misaligned with the teaching and learning activities if the course were conducted solely with lectures. Compared with debates, role playing, and small group discussion, lectures are limited in their ability to teach students how to think critically.
In planning a course, the teacher should ask key questions about the components of the learning experience. Are there any misalignments between the student characteristics or the course content and the way the course will be conducted? How well do the assessment methods address the course goals and objectives? Do the goals include helping the student learn how to assess his or her own performance? Do the learning activities effectively support all goals and objectives? Are there extraneous learning activities that do not serve any of the goals or objectives? Is feedback incorporated into the course that will help students understand the criteria and standards that will be used to assess their performance? How well do the learning activities and associated feedback opportunities prepare students for the evaluation activities?
|
Course Evaluation
|
|---|
The final step in course design, the step that completes the educational "loop," is to plan and conduct an evaluation of the course and one’s teaching. This step is critical and is the only way to obtain information needed to improve the course and teaching. Course effectiveness can be measured in terms of whether the course goals were achieved, the effectiveness of particular learning activities, and the teacher’s ability to interact effectively with students.
Feedback can be solicited throughout the course as well as at the end. Feedback can be obtained through video or audio taping of learning activities, student ratings of instruction and instructor, student interviews or questionnaires, assessments by outside observers, and test results. When information from such feedback is used to change a course, and results in course improvement, the educational loop of design, implement, evaluate, and modify is complete.
|
Summary
|
|---|
Creation of "significant learning experiences" follows basic steps of instructional design related to situational factors, goals and objectives, feedback and evaluation, teaching and learning activities, alignment of the preceding elements, and course evaluation. Goals should reflect what students will learn at the end of the course and what will still be with them several years later. Objectives should focus on learner performance, not teacher performance, and on behavior, not subject matter; there should be only one learning outcome per objective. Students learn more and retain their knowledge longer if they acquire it in an active rather than a passive manner. The situational factors, goals and objectives, feedback and evaluation, and teaching and learning activities should all reflect and support each other. The act of course evaluation closes the educational loop of design, implement, evaluate, and modify.
The Table outlines the steps involved in designing an effective learning experience and provides a template that can be used in designing such activities.
|
References
|
|---|
- Fink LD. Creating significant learning experiences: an integrated approach to designing college courses. San Francisco, Calif: Jossey-Bass, 2003.
- Arreola RA, Aleamoni LM. Assessing student learning outcomes: a workshop resource document. University of Tennessee, Memphis; published 1998. Available at: http://webster.utmem.edu/grad/MISCELLANEOUS/Learning_Objectives.pdf. Accessed December 12, 2006.
- Clark D. More on learning objectives. Updated May 2000. Available at: http://www.sos.net/~donclark/hrd/objectives.htm. Accessed December 12, 2006.
- Bloom BS, ed. Taxonomy of educational objectives. Vol I: cognitive domain. New York, NY: McKay, 1956.
- Fuhrmann BS, Grasha AF. A practical handbook for college teachers. Boston, Mass: Little, Brown, 1983; 170.
- Schultheis NM. Writing cognitive educational objectives and multiple-choice test questions. Am J Health Syst Pharm 1998;55:2397–2401.[Abstract/Free Full Text]
- Crooks TJ. The impact of classroom evaluation practices on students. Rev Educ Res 1988;58: 438–481.[CrossRef]
- Ende J. Feedback in clinical medical education. JAMA 1983;250:777–781.[Abstract]
- Collins J. Writing multiple-choice questions for continuing medical education activities and self-assessment modules. RadioGraphics 2006;26: 543–551.[Abstract/Free Full Text]
- American Board of Radiology. MOC requirements. Available at: http://www.theabr.org/DR_MOC_Req.htm. Accessed December 13, 2006.
- Accreditation Council for Graduate Medical Education. Competencies: definitions and practice performance measurements. Available at: http://www.acgme.org/acWebsite/RRC_420/420_compDefsPerfMeas.pdf. Accessed December 13, 2006.
- University of Toronto Department of Medicine. Techniques to make large group learning more active. Available at: http://www.deptmedicine.utoronto.ca/PageFactory.aspx?PageID=273. Accessed December 13, 2006.
Top
Abstract
Introduction
