|
Simulation Evaluation System |
||||||||||||||||||||||||||||||||||||
|
There a number of concerns or criteria involved in assessing the value of a particular simulation for a public administration course. These include cost, ease of administration, difficulty of operation, support of learning objectives, ease of debriefing, and student assessment. These concerns or criteria were used to develop a simulation classification matrix. Each of the criteria was then given a level of “desirability” from the viewpoint of the instructor. From the instructor’s point of view, the ideal simulation would meet the criteria established at Level 4 – free, no facilitation after set-up, immediate operation by the student, integrates course learning objectives, provides the means to reinforce the learning during the debriefing and the ability to assess individual student learning. To assess the value of the various simulations, a value was assigned to each of the criteria based on level. For each Level 4 criteria four points are assigned. Therefore the ideal simulation would have a value of 24 points. A detailed analysis of the criteria is provided in Attachment 1 to this paper.
Application of the criteria Cost - Getting what you pay for does not appear to be the “rule of thumb” when it comes to simulations. The majority of the simulations evaluated in this study were either free or available at less than $25 per student. Administration – Ease of administration of the simulation by the instructor is critical in the hybrid learning environment. Ideally the use of the simulation is done outside of the class, either on-line or as a standalone program on the students computer. Using this criteria, the basic concept of the simulation can be included in the syllabus along with expectations. After a brief overview in class, the student should be able to operate the simulation without assistance or involvement of the instructor. Operation – The degree of difficulty in learning how to effectively use the simulation is critical. Considering most graduate students are working full time and have family and other responsibilities, the time spent learning how to operate the simulation, can be viewed as being non-productive time. Learning Objectives – Perhaps one of the most important criteria is how the simulation goals fit into the overreaching goals of the course, specific course learning objectives or a particular lesson. When setting up the lesson, it is critical to establish this relationship for the students. The simulations should be evaluated on their ability to achieve and integrate course learning objectives. \ Debriefing - In order for the simulation to have value as a learning experience, it must be conducive for the students to discuss how the experience contributed to the overall learning goals or objectives. The instructor should write the vital questions down as part of the lesson design. The questions should do the following: · Describe the important aspects of the interaction, the easy, fun and difficult parts. · Relate real life experiences and course concepts to the simulation · Relate what they saw as successful strategies, successful decisions, and poor strategies or decision. · Relate the level of goal achievement. The instructor should be able to reinforce the learning experience by · Provide students with real-life scenarios and ask about comparisons. · .Writing down the predictions, comparisons, and the similarities between the real topic and the game can be powerful methods to strengthen students’ understanding.
Assessment – Most simulations are designed to achieve specific goals. A good simulation will provide the participant feedback on their mastery of the simulation strategies or decisions. The ease of this feedback to be translated by the instructor into student assessment or grading is an important criteria. |