TITLE: Learning Event Development Process

NUMBER: TRA120

EFFECTIVE: 20 MAY 2016

REVIEWED:

PURPOSE: To provide employees of IPS a process for the a systematic, educationally sound process for the development of events including courses, seminars, and online training.

PROCEDURES:

Whether you are developing a new classroom training course, creating an online training course, or coordinating an informational conference, a systematic, educationally sound process for development is important for the learning process. The IPS Training Course Development Process is based on a proven development process used throughout the adult education field. It ensures high quality skill based training by incorporating principles of adult learning. Adhering to this process for course development helps create consistency in quality training methods across IPS agencies.

The IPS Training Course Development Process requires─

1. Needs assessment and development of learning outcomes
2. Course agenda and lesson plan
3. Assessment methods that test achievement of learning outcomes.
4. Documentation of course materials

Needs Assessment and Learning Outcomes

A needs assessment is performed to determine if there is an audience for a course and, if so, what content should be included in the course. What may trigger the need for a course? There may be an ongoing need as a result of a government regulation or an established training certificate program. Different customers may request the training course or it may be a new course you are interested in offering. A course may be needed because of current or future problems or because of a specialized need. Changes in government regulations and social, economic and political concerns may also create the need for a course.

Information collected in a needs analysis includes─

1. How does the course fulfill the organization’s needs?
2. What should people know after the course?
3. What skills do people need to learn in order to change their behavior?

Information sources for a needs analysis include─

1. Targeted organizations and professional associations
2. Advisory committees
3. Customers
4. Subject matter experts
5. Learners
6. Previous instructors
7. Existing organization data

Methods that may be used to gather information for a needs assessment include─

1. Interviews
2. Job Analysis
3. Observation
4. Focus groups
5. Survey
6. Previous learning event evaluation data
7. Examination of existing organization data
8. Direct request from funding agency

Gathered data provides input into course development, most especially the learning outcomes.

Learning outcomes are statements that describe what the learner will be able to do upon completion of the learning event. They help us decide not only what a learner should learn but also how we will assess achievement of the learning outcome. It is important to have learning activities and assessment methods aligned to learning outcomes. Learning outcomes should emerge from the needs assessment. While conducting the needs assessment you should determine what behavior needs to be changed, what skill is needed, or what knowledge is lacking.

Learning outcomes should identify specifically what will be learned and what learners will be able to do when the course is completed. They are planned outcomes that establish the foundation of your training. Your course should be built around the learning outcomes you select.

Learning outcomes are depicted with action verbs so the leaner’s behavior is measurable. Learning outcomes serve as guidelines for the level of skill to be taught, selection of content, development of learning activities and evaluation and measurement.

Bloom’s Taxonomy is a widely used method of creating learning outcomes. IPS learning event development will focus on the cognitive domain. The cognitive domain focuses on content and intellectual knowledge; what do the learners think or care about? The six levels of the cognitive domain are knowledge, comprehension, application, analysis, synthesis, and evaluation.

It is important to create learning outcomes for the level that you expect learners to perform. You must be able to assess the learning outcomes. The following table, Bloom’s Taxonomy Action Verbs, defines each level in the cognitive domain and lists verbs for each level. Use this tool when writing learning outcomes to ensure the learning outcome measures the appropriate level.

Examples of learning outcomes include the following:

1. Define internal controls.
2. Explain the Chart of Accounts.
3. Demonstrate how to don a Self‐Contained Breathing Apparatus.

The first one measures the knowledge level, the second one measures the comprehension level, and the third one measures the application level.

Because you must assess whether or not each learning outcome is achieved, it is best to limit the number of learning outcomes.

1. Develop learning outcomes for the most important information in each module. One or two learning outcomes per module should be sufficient.
2. Do not repeat yourself. If you are going to require the learner to “Make an omelet” do they really need to “Describe how to make an omelet”?
3. Make sure you can measure whether or not learning occurred. Avoid verbs such as know, understand, learn, appreciate, become aware of, and become familiar with because they can’t be measured.

Agenda and Lesson Plan

The agenda and lesson plan explain how the course should be conducted. When creating an agenda and a lesson plan, you must select the content and course activities.

Content Selection

The content included in your course should be based on the learning outcomes. Include what is considered “need to know” information and not “nice to know” information. Things to consider when selecting content include─

1. The learners’ familiarity with the subject matter,
2. The length of the course,
3. What you want to achieve through the learning outcomes,
4. The instructional methods that will be used, and
5. How assessment of the learning outcomes will be conducted.

Start your lesson plan by developing an outline for the course with each of your learning outcomes as a major topic or module. Then list the key information under each learning outcome and prioritize the information as “need to know” or “nice to know.”

The order in which the content is presented is important. Content should be sequenced in a

logical manner with simple concepts presented first and building to more complex theories. Organize your course so that there is an introduction, body, and conclusion or summary. Learning Activities

Learning activities, including posttests, are used to assess the achievement of learning outcomes.

Learning activities include:

1. Lecture: A presentation of content by facilitator; question and answer sessions are included.
2. Discussion: A planned conversation with planned questions where learners make comments; facilitator follows up on learners’ comments.
3. Structured Exercise or Lab: Learners work in a group to discover a new concept or apply a recently-learned concept.
4. Post Test: Structured quiz; learners answer questions individually.
5. Case Study: A complex structured exercise utilizing a problem scenario that learners analyze and resolve.
6. Role Play: A complex structured exercise utilizing an interpersonal problem situation that learners enact according to certain criteria and resolve.

It is important that learning activities “match” the learning outcomes. As stated earlier, learning outcomes help us decide not only what a learner should learn but also how we will assess achievement of the learning outcome. Use Bloom’s Taxonomy Staircase (below) to ensure your learning activities and instructional methods assess achievement of learning outcomes at the appropriate level.

1. Find the cognitive level based on the verb used in the learning outcome. Verbs are listed below each step.
2. Suggested activities for the levels are listed above each step.

If the learning outcome requires the learner to “describe”, this is the comprehension level and a learning activity could be a test or assessment. If the learning outcome requires the learner to “demonstrate”, this is the application level and the learning activity could be a demonstration. Remember that learners learn from hearing, seeing, and doing so it is best to utilize different learning strategies and employ all three methods.

Add selected learning activities and necessary instructions for conducting the activities to the appropriate topics/modules on your lesson plan. If the only learning activity is a post test, add it to the end of your lesson plan or to the Course Summary module. Be sure to estimate the amount of time required to conduct each learning activity.

Agenda

Now you are ready to create your course agenda. Course agendas should include:

1. Learning Outcomes
2. Each activity/module with approximate allotted times.
3. Any materials required for the activities/modules.

You should be able to find all of this information in the lesson plan for the course. Agenda formats can vary. You may develop an agenda for the course or, for long courses, you may want to develop an agenda for each module. Appendix A is a sample agenda.

Lesson Plan

A lesson plan is not a PowerPoint presentation. The lesson plan includes a full description of each activity, including the learning outcome that the activity fulfills and detailed directions for the instructor on what questions/comments to expect, how to set up activities, how to accommodate different learning styles, media to be used, etc.

Lesson plans should include─

1. Learning points-The information that should be presented.
2. Materials needed.
3. Process-What the facilitator will do in the activity.
4. Instructions for learners.
5. Debrief questions.

Lesson plan formats can vary. Some lesson plans are very specific while other lesson plans are more general.

The agenda is to be distributed to learners so they will know what to expect in the course. The lesson plan is used by the instructor to conduct the course.

Remember-The content of the course and the learning activities should support the learning outcomes.

Appendix B is a sample lesson plan.

PowerPoint

Do not depend on PowerPoint to present the content in the course. The presenter explains the content and PowerPoint is a tool to help the presenter. Appropriate use of PowerPoint supports the content being presented.

A good PowerPoint presentation will─

1. Have a simple and consistent design,
2. Limit one idea per slide,
3. Simplify and limit the number of words on the screen,
4. Use appropriate fonts and sizes,
   1. Sans Serif fonts for titles, bullets and body text.
   2. When giving a presentation in front of people, make sure font is big enough for the people in the back of the room to read.
5. Use good quality images that reinforce your message, and
6. Use appropriate graphs when necessary rather than charts or words.

Assessment Methods

You must be able to assess your learning outcomes. Donald Kirkpatrick created a four level model for training evaluation that is utilized by training professionals. Each IPS learning event must be evaluated at Level 2 (attainment of learning outcomes). The Level 2 assessment(s) will measure attainment of the learning event’s stated learning outcomes by the learners. Lesson plans should explain how and where each learning outcome is being assessed. There should be a clear correlation between the learning outcomes and the assessment process. Use Bloom’s Taxonomy Staircase (presented earlier) to ensure your learning activities and instructional methods assess achievement of learning outcomes at the appropriate level.

Documentation of Course Materials

IPS has a process for approving training courses for CEUs. This process is explained in the IPS Training Development Policies and Procedures. If you are not submitting your training course for CEU approval, you should still upload the course materials to the IPS Training Idea Exchange SharePoint site. Along with the learning outcomes, agenda, lesson plan, and assessment materials, you should upload instructor credentials, participant materials, program evaluation, and promotional materials. These materials should be uploaded before the course is delivered. An explanation of these requirements is included in the IPS Training Development Policies and Procedures.

Appendix A: Sample Agenda

Succeeding Under Stress Agenda

Learning Outcomes

1. Define Stress
2. Identify Sources of Stress
3. Analyze Participants sources of stress
4. Describe ways to prevent stress

Course Agenda

TOPIC

ACTIVITY

Introduction

Review: Logistics, Objectives

Conduct the pre‐test

Time: 10 minutes

Defining Stress (LO1)

Exercise: What stresses have you

encountered today?

Time: 45 minutes

Identifying sources of

stress (LO2)

Exercise: Stress level

inventory

Exercise: Top Ten Stressors?

Time: 55 minutes

Analyze participant’s

sources of stress (LO3)

Time: 55 minutes

Describe ways to prevent

stress (LO4)

Time: 70 minutes

Summary

Complete stress management match

game

Review course objectives

Complete class evaluation

Time: 15 minutes

Appendix B: Sample Lesson Plan

Internal Controls Workshop

Lesson Plan

Prepared 1/29/16

Instructional Methods: This course utilizes lecture to review the material. There is a pretest and posttest and a small group activity used to evaluate attainment of learning outcomes. Upon completion of the posttest, learners complete a course evaluation. Different learning styles will be accommodated through lecture, PowerPoint presentation, answering questions and a small group activity.

Activity	Instructions
Introduction (approximately 20 minutes) PowerPoint slides 7 ‐ 36	1. 1.	Introduce yourself and the course. Explain that a pretest and posttest will be administered to evaluate attainment of learning outcomes. The same test is used for both the pretest and posttest.
	2.	Cover breaks and the location of restroom facilities.
	3.	Explain the purpose of the course and the learning outcomes. Upon completion of this course, learners should be able to─
	4.	Define internal controls
	5.	Describe the five components of the internal controls framework (theory) including the principals of each component.
	6.	Explain how to comply with the amendment to T.C.A. § 9‐18‐102(a) and federal grant requirements for internal controls.
Module 1: What are Internal Controls? (LO1) (approximately 45 minutes) PowerPoint slides 37 ‐ 67	1. 2. 3.	Discuss the USGAO green book definition for internal controls. Describe an internal controls system. Discuss the Committee of Sponsoring Organizations (COSO)
	4.	Explain T.C.A. § 9‐18‐102(a); an amendment to State law requiring all county governments establish and maintain internal controls. Benefits Audit requirements Non‐compliance
	5.	Case Study – Dixon, Illinois
Break (15 minutes)
Module 2: Who is Responsible for Establishing Internal Controls? (LO2 and LO3) (approximately 45 minutes) PowerPoint slides 68 ‐ 95	1. 2. 3. 4. 5.	Elected and appointed officials are responsible for design, implementation and maintenance Auditor’s role Fraud detection including fraud triangle Examples of fraud in Tennessee Why good internal controls are needed/common problems
	6.	Explain the five components of internal controls including the principles of each component
	7.	Discuss the first two components in detail, Control Environment and Risk Assessment
Break (15 minutes)
Continuation of Module 2 (LO2 and LO3) (approximately 40 minutes) PowerPoint slides 96 ‐ 1117	1. 2. 3.	Discuss the remaining three components in detail, Control Activities, Information and Communication, and Monitoring Third party service organizations County legislative body responsibility for internal controls
	4.	What steps should county officials take to establish internal controls
Break 10 minutes)
Module 3: Internal Controls Exercise (LO2 and LO3) (approximately 30 minutes) PowerPoint slides 118 ‐ 122	Break class into small groups and assign each group a role. Have groups read the narrative and audit findings. Each group will take about 15 minutes to determine what specific internal controls to recommend. Have each group share, via a spokesperson, the group’s recommendations for new or improved internal controls.
Summary (approximately 20 minutes) PowerPoint slides 123 ‐ 149	Answer any questions. Administer the posttest and have participants complete the program evaluations.

RESOURCES:

CONTACT:

Mary Ann Moon maryann.moon@tennessee.edu