Although simulations are often used in teaching political science, they are rarely used as forms of assessment. Over the last few years, however, Peter Shearman and Roni Linser have offered simulations as an alternative assessment of learning to University of Melbourne students in a second- and third-year course entitled "World Politics in Transition." This decision arose from a need to demonstrate to students the strengths and weaknesses of international relations theories that were the focus of the course, and to enliven what might otherwise seem dry theory. The decision was predicated on pedagogy that long-term retention and use of learning are better achieved through experiential learning.

Before 1997, the course relied on a traditional approach: lectures, tutorials, and individual research by students. Students were assessed on the basis of a 2,000 word essay (50%) and a 2-hour exam (50%) at the end of the course. In 1999, when students were given the alternative of doing a goal-based role-play simulation in lieu of the final examination, 86 of the 120 students in the course chose the simulation. This article describes the simulation used to assess student learning as well as students' thoughts about the alternative.

The course syllabus describes the simulation as follows: "By playing roles of different world leaders, organizations, and media outlets, students will experience some of the dilemmas and issues involved in World Politics and will be able to test... the applicability of different theoretical positions involved in... international relations." The simulation is based on the following four essential ingredients (Linser et al, 1999):

• goal-based learning, a strong motivator of learning. Typically, goal-based learning comprises a scenario or context that includes a "trigger," or precipitating event. Our goals are dynamic in the sense that students set the goals themselves during the "role profile" write-up.
• role-play: playing with possibilities and alternative worlds, and playing to have fun. By playing a role (in all these senses), students start to identify with the role, which gives them a personal stake in the proceeding and enhances their motivation. Of course, in order to play roles effectively, students must research, which they can do via links on the role-play Web site that are selected by the simulation creator. In addition to using the Web site, students must also do traditional library research and attend lectures and tutorials.
• the Web, which provides the virtual space for the role-play and enables communication and collaboration among students and lecturers. It also gives students access to up-to-date news from electronic newspapers and Web sites from all over the world.
• traditional teaching: face-to-face lectures and tutorials. Many experienced online educators (Hedberg & Harper, 1998; Brown, 1998; Durham, 1998) have emphasized the importance of coupling face-to-face interaction with computer-mediated communication and online teaching. Incorporating traditional teaching into the learning architecture not only helps present facts, cases, and theories, but also stimulates discussion about actions and strategies that develop in the comparison of real-world events to simulated ones.

Pedagogical Comparison to Other Web-based Role-Play Simulations

Apart from the World Politics in Transition simulation at the University of Melbourne, there are two other long-standing Web-based role-play simulations: the ICONS (International Communication and Negotiation Simulations) Project at the University of Maryland, developed by Jonathan Wilkenfeld and Richard Brecht, and Politics and International Relations in the Modern Middle East at Macquarie University in Australia, developed by Andrew Vincent and John Shepard (who works at the University of New South Wales). All three simulations begin with a scenario outlining the state of the world based on present-day facts. The World Politics and Middle East Politics simulations then add potential perturbations. (The ICONS Project does not.) These perturbations aim to engage students with "what if" situations, forcing them to think of the implications of different permutations of real-world facts. All three simulations put individual roles into larger groupings: for example, a larger grouping like the United States may include the roles of President, national security advisor, secretary of state, and secretary of defense—each played by teams of two to four students.

While the role-playing aspects of all three simulations are similar, their pedagogical objectives differ. The ICONS Project aims to capture the complexity and subtlety of international political issues through detailed scenarios focusing on real or plausible policy. The Macquarie Middle East simulation aims to introduce students to the facts of Middle East politics (the major leaders, groups, movements, and relationships in the region), give them experience with the complexities of negotiation and decision-making in "real" political systems, and improve their use of computer technology and the Internet as tools for the workplace. Like the Macquarie simulation, the World Politics in Transition simulation enables students to gain knowledge of facts and cases, to develop negotiating and decision-making skills, and to improve their computer and Internet research skills. But in contrast to both the Macquarie simulation and the ICONS Project, the World Politics in Transition simulation has as its main purpose enabling students to gain an understanding of the theories of international relations. The simulation achieves this goal by asking students to reflect and present papers on the theories behind their actions.

From a pedagogical perspective, the ICONS Project focuses on policy alternatives with regard to a specific and limited number of issues. Both the Macquarie and Melbourne simulations capture a larger number of issues as an integrated whole. In other words, the advantage of these simulations is that they present international politics issues in relation to other national and international issues, so that students must think of how their priorities and decisions will affect the larger whole.

Finally, the World Politics in Transition simulation's focus on the relationship between theory and practice enables students to gain a deeper understanding of international relations theory, so they can apply this understanding to other contexts and issues.

Software for Simulation Generation

The World Politics in Transition simulation is powered by a role-play simulation generator called Fablusi (TM). The software design is based on the idea that human interactions are communicative events requiring information exchange (Ip & Linser, 1999). Using a form-based interface (Figure 1), Fablusi provides a skeleton to:

• determine the roles in the simulation;
• write up a scenario by providing basic information to all roles and additional, specific information to some roles in order to create an information gap;
• design specific forums (modeled by conferences) for some of the roles to simulate real world organizations;
• set up the social structure for the roles by setting the rights in the conferences; and
• optionally create tasks to scaffold the roles into the simulation.

Each student takes on a role specified during the simulation creation process. Fablusi supports communication of roles using Sim-mail and Sim-conference. Sim-mail is similar to a Web-based e-mail system but is totally isolated from the real world; roles can send mail only to other roles. Sim-mail also allows the lecturers to view all mail in order to monitor the progress of the simulation and provide support to roles. Sim-conference is similar to most other generic conferences, but the participation rights of roles can be individually set to reflect real-life relationships. Sim-conference also supports multiple document types in order to simulate different positions within an organization. For example, the program may simulate a news agency with the positions of journalists, editors, and readers, or an international forum with participants and a chairperson.

Each role is played by a team of two to four students who research that role together before starting the simulation. The students then write a role profile. Fablusi's formal writing task allows students to publish role profiles for the other roles automatically. (To see the profiles, please click here and log in using username "guest" and password "demonstration." Click the Profiles button.) Each team must become familiar with the objectives and strategies of each role; each role profile then functions as a case study for evaluating theories studied in the course. By acting in the role, students are able to assess the effectiveness of different theories in explaining a particular role's actions.

The object of the simulation is for students to reach the objectives of their role profiles and enhance the positions of their roles in the simulation. They achieve these goals by contacting relevant players through the mail system and Sim-conferences and using diplomacy, threat, coercion, etc., as warranted by the specific conditions of their role. The simulation begins with a scenario to which students in different roles react. This scenario, written by the lecturers, is developed in accordance with real-world events but includes plausible "what if" events. Students attempt to confront these developments from the perspective of their role by creating new developments, the viability of which they assess through their research.

In 1999, the simulation was set around the Kosovo crisis and highlighted the role of the UN. This was a fairly large simulation consisting of 37 team roles. The simulation was played out for three weeks (set at three weeks ahead of real time, or, in game parlance, "simulation time"). In these three weeks, the students sent out more than 3,000 messages with an average of seven recipients per message. Figure 2 shows the read mail screen with a Sim-mail from the role of Bill Clinton.

Comparison of Software Architecture to Other Web-based Role-Play Simulations

Fablusi, the software driving the simulations described in this case study, runs on Microsoft NT servers (Microsoft Internet Information Server and SQL server). A PIET engine provides the underlying authoring capability, with the role-play simulation implemented as a combination of ASP pages and COM objects. The ICONS Project is based on ICONSnet, which is written in Oracle PL/SQL and runs on Oracle Database and Oracle Application Server. ICONSnet supports both real time and asynchronous message exchange, team anonymity, and the possibility of language translation. The technology used for simulations at Macquarie University is based on the Unix operating system, largely a collection of appropriate Unix tools and their customization.

Student Assessment of Simulation Effectiveness

Students in the 1999 World Politics class found the simulation effective. Thirty-eight of the 86 students who participated in the simulation responded to an end-of-term questionnaire. As recorded in Naidu et al. (2000), these students generally reported that the simulation enabled them to achieve the specified learning outcomes of the course. Table 1 shows the questions asked and the average responses on a scale of 1 to 5, with 1 signifying that the exercise was "very useful" and 5 signifying that the exercise was "useless." The average response to questions relating to understanding and identifying facts, issues, problems, and factors was between 1.34 and 1.92, which is in the "very useful" to "useful" range. The question of whether the simulation added "active and dynamic dimensions to classroom learning processes" generated an average response of 1.08, an overwhelming indication that the simulation created a new, innovative learning environment.

Table 2 shows how the simulation was perceived by the students as an instrument for learning. On a scale of 1 to 4 (this time, 1 signifies "strongly disagree" and 4 signifies "strongly agree"), students agreed that the simulation was useful for developing understanding, identifying issues and problems, gaining knowledge, and evaluating strategies learned in the theory class. Students also wrote a position paper in preparation for a face-to-face conference to round out the simulation experience. When asked whether the position paper had contributed to their learning, students responded positively, with an average score of 2.95.

The students spent a lot of time on the simulation, as Table 3 shows. There was a significant increase in time spent among those with home Internet access (item 36) versus those without.

Evaluation of Using Role Simulations in Instruction

The utility of the simulation as a whole can be gleaned from student responses to free-text questions.

Question: Would the use of a Web-based simulation such as this influence your decision to take a course? If so, why? If not, why?

Students responded positively to this question. They enjoyed the effectiveness of the learning experience, together with the element of "fun" and "getting to know and interact with a wider range of students within the subject." Students acknowledged that this exercise helped them understand and tackle complicated issues and achieve deeper learning. However, some students had reservations due to the large amount of time that they spent on the simulation. The data from optional items indicates that some students spent more than 15 hours a week on the simulation during the three weeks that it ran (see Figure 3).

Question: The mail system in the simulator is NOT a real e-mail system. Please state the advantage(s), if any.

Unlike generic mail systems used in some simulations, the mail subsystem in the role-play simulation generator is fully integrated into the simulation and completely separate from real-world mail systems. Thus, all mails are sent in the name of the role. We believe that it is important to distinguish between a simulated world and the real world. One student recognized this, writing that the role-play mail system "Kept it [simulation e-mail] separate from my other e-mail so when I entered the world of the simulation, it was a distinct and separate world" (see Figure 4).

Question: We would like to get your comments on any aspect of the simulation that may help us in making it a better experience for students.

This question solicited feedback on students' overall experiences. We received mixed responses to the system. Some felt that the site was fast; others found it intolerably slow (see Figure 5).

Conclusion

The role play simulation described in this article is a proven learning architecture with a long history of use in political science and other subjects. Students report that the recent conversion of this learning environment to a Web-mediated environment has been a positive innovation; few had trouble with the computer technology or the Fablusi software. Yet more work is needed to find out whether a real learning-outcomes benefit has been achieved. It is hard to evaluate the effectiveness of simulations as an assessment tool versus traditional means of assessment by essays and exams, because the Political Science Department at the University of Melbourne uses a standardized criterion for student assessment results. Student scores are made to fit a bell curve distribution, making it difficult to evaluate whether a group of students would perform better with the simulation or with essays and exams.

However, three significant anecdotal results may be highlighted, from Shearman and Linser's teaching over a five-year period using both means of assessment. First, weaker students who participate in the simulation tend to understand the material better than weaker students who do not participate. Second, since simulations are a process rather than a finished object (as an essay or an exam), fewer students freeze up, and their mistakes are corrected during the process. The pressure of an exam, or the mistakes to be found in essays, have less of an impact on students' results. Finally, since the simulation takes place over time, examiners may intervene to pose specific questions that clarify whether a student understands the material. In short, using simulations for assessment gives examiners and students greater scope in the assessment process: it allows students to demonstrate their abilities and examiners to evaluate the level of student understanding and knowledge. It may be worthwhile to point out that students vote with their feet: approximately 72% of students given the choice of simulation or essays and exams for assessment purposes chose the simulation. Even if further research shows that no significant improvement in learning is achieved with the simulation, we believe that the increase of fun and playfulness, which enhance student motivation, justifies the effort.

The quality of the simulation and its effectiveness as a learning vehicle depend on the creativity of the simulation designer. The role of the generator is to provide a skeleton that reduces the amount of work done by the lecturer. However, in the exercise reported, the effort by the role-play coordinator was only somewhat reduced. This outcome was mostly a result of the novelty of the technology and, consequently, the necessity of testing the system and fixing bugs. The role structure has recently been enhanced to allow better monitoring of the simulation activities by lecturers and tutors. A private, real-time chat room has also been added. During two other simulations in April and May 2000, the performance of the system was demonstrably improved with the use of better hardware. We expect that future simulation creations will take significantly less time as the bugs in the systems are removed and we gain additional experience in creating a more effective delivery system. We hope to see more academics taking up such an approach. Readers are welcome to contact Albert Ip for guidance on how to implement Fablusi ^(TM) or Roni Linser for questions on simulation design.

References

Brown, M. (1998, September). New teacher for a new age: The myths and realities of the global classroom. Paper presented at Flexible Learnings: The Apple University Consortium Conference at the University of Melbourne, Australia.

Durham, M. (1998, September). Working at virtual records—a simulated workplace. Paper presented at Flexible Learnings: The Apple University Consortium Conference at the University of Melbourne, Australia.

Hedberg, J., & Harper, B. (1998). Supporting flexible thinking with interactive multimedia. Paper presented at Flexible Learnings: The Apple University Consortium Conference at the University of Melbourne, Australia.

Ip, A., & Linser, R. (1999). Web-based simulation generator: Empowering teaching and learning media in political science. Paper presented at the Australasian Society for Computers in Learning in Tertiary Education conference, Brisbane, Queensland, Australia. Retrieved 25 May 2000 from the World Wide Web: http://www.roleplaysim.org/papers/rpsg.htm

Linser, R., Naidu, S., & Ip, A. (1999). Pedagogical foundations of Web-based simulations in political science. In J. Winn (ed.), Proceedings of the Australian Society for Computers in Learning in Tertiary Education (pp. 191-198). Retrieved 23 October 2000 from the World Wide Web: http://www.ascilite.org.au/conferences/ brisbane99/papers/linsernaidum.pdf

Naidu, S., Ip, A., & Linser, R. (2000). Dynamic goal-based role-play simulation on the Web: A case study. Manuscript submitted for publication. Retrieved 25 May 2000 from the World Wide Web: http://www.roleplaysim.org/papers/Naidu_etal.html

downloadable pc gamesdownloadable gamesplatform gamesmatch 3 gameshidden object gamespc game downloadsaction gameskids gamespc gamesshooter games