A systematic literature review on serious games evaluation: An application to software project managementby Alejandro Calderón, Mercedes Ruiz

Computers & Education

About

Year
2015
DOI
10.1016/j.compedu.2015.07.011
Subject
Computer Science (all) / Education

Similar

A systematic literature review on the barriers faced by newcomers to open source software projects

Authors:
Igor Steinmacher, Marco Aurelio Graciotto Silva, Marco Aurelio Gerosa, David F. Redmiles
2015

Management of traumatic thoracolumbar fractures: a systematic review of the literature

Authors:
Nicole van der Roer, Elly S. M. de Lange, Fred C. Bakker, Henrica C. W. de Vet, Maurits W. van Tulder
2005

Knowledge-based approaches in software documentation: A systematic literature review

Authors:
Wei Ding, Peng Liang, Antony Tang, Hans van Vliet
2014

Humanitarian supply chain performance management: a systematic literature review

Authors:
Hella Abidi, Sander de Leeuw, Matthias Klumpp
2014

Text

Accepted Manuscript

A systematic literature review on serious games evaluation: An application to software project management

Alejandro Calderón, Mercedes Ruiz

PII: S0360-1315(15)30016-6

DOI: 10.1016/j.compedu.2015.07.011

Reference: CAE 2879

To appear in: Computers & Education

Received Date: 7 May 2015

Revised Date: 20 July 2015

Accepted Date: 21 July 2015

Please cite this article as: Calderón A. & Ruiz M., A systematic literature review on serious games evaluation: An application to software project management, Computers & Education (2015), doi: 10.1016/j.compedu.2015.07.011.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

M

AN

US

CR

IP

T

AC

CE

PT

ED

ACCEPTED MANUSCRIPT

A SYSTEMATIC LITERATURE REVIEW ON SERIOUS GAMES

EVALUATION: AN APPLICATION TO SOFTWARE PROJECT

MANAGEMENT

Alejandro Calderóna, b, Mercedes Ruiza, c a Department of Computer Science and Engineering, University of Cádiz, Avenida de la Universidad de Cádiz, 10, 11519,

Puerto Real, Cádiz, Spain b E-mail: alejandro.calderon@uca.es; Phone number: +34 956483248 (corresponding author) c E-mail: mercedes.ruiz@uca.es; Phone number: +34 956483440

M

AN

US

CR

IP

T

AC

CE

PT

ED

ACCEPTED MANUSCRIPT

Abstract

Training that future practitioners receive in software project management is a topic of great importance. The objective of this systematic literature review is to summarize the current state of the art of the different methods and procedures used to assess serious games. The review follows a predefined procedure that involves automatically searching well-known digital databases. 1199 papers were found by the automatic searches in the digital databases and 102 papers were selected as primary studies. The process was complemented with manual searches using author and backward snowballing techniques. Our systematic literature review identified the main methods followed to assess serious games, the application domains in which the assessments took place, the categories of serious games assessed, the main features considered to assess the educational effectiveness of serious games, the procedures followed for the assessments and the size of the population that participated in the assessments. The results are useful to researchers and practitioners willing to assess serious games in different fields, but specially to those interested in assessing serious games in the area of software project management.

Keywords

Serious game; Evaluation; Systematic literature review; Software project management 1. Introduction

Software project management is a knowledge area of great importance to keep in mind by experts on the road to success in the development of software projects. The relevance of teaching software project management for Information Technology (IT) learners has always been supported by organizations such as the Association for Computing Machinery (ACM) and IEEE-Computer Society in their joint task force curricula (ACM/IEEE-CS Joint Task Force on Computing Curricula, 2013), in which they recommend that learning about the methods and principles of project management should be practical rather than theoretical.

However, bringing professional practice into the classroom is a difficult task. Frequently, future computing professionals lack practical training with real-life scenarios. In addition, a highly theoretical learning environment, as in current software project management syllabi, is quite uninteresting for the future professionals (Ibrahim, 2011). Consequently, novel professionals develop their experience working in real projects, where the effects of a wrong plan or decision-making can lead to a failed project or the loss of benefit for the companies they work for.

Given the importance that knowledge gained from practical and action-based learning experiences with real consequences have in software project management, the use of games and simulation-based experiences help us to achieve these objectives by teaching practically within a risk-free environment. Such games, designed with a different purpose than only entertainment, are called serious games (Abt, 2002) (Zyda, 2005).

However, despite the potential benefits of using serious games in software project management, there is a lack of serious games to teach and assess learners in this area. The available games are frequently not flexible, have a very specific scope and do not allow assessing the learner’s new skills automatically. The weaknesses observed in the available serious games led the authors of this work to develop a new simulation-based serious game that is able to bring real-life practice in software project management as well as performing an automatic assessment of the skills developed by the learners while playing the game (--------)1

The next step, in our research work, is to perform the assessment of both the effectiveness of our game and its level of compliance with the objectives for which it was created. In order to design the assessment process of our game, we conducted this study aimed at finding the current state of the art of the different methods and procedures used to assess serious games, disregarding the application area of the serious game. In 1 In order to not show anything identifying the author in this manuscript, we do not incorporate this reference that cites our previous work, and we refer to “our game” instead of saying its name.

M

AN

US

CR

IP

T

AC

CE

PT

ED

ACCEPTED MANUSCRIPT addition, the works found in the specific area of software project management were also identified and explicitly analyzed. The study was conducted as a systematic literature review (SLR) following the guidelines proposed by Kitchenham (MacDonell, Shepperd, Kitchenham, & Mendes, 2010) (Kitchenham & Charters, 2007).