Previous stages
As those of you who follow us at GECON.es website or at our Linkedin may already know, MEGASKILLs began in February 2023 and continues with its agenda to fulfill the dream of creating an educational methodology for training and evaluating soft skills through the use of video games.
The first executive stage of the project (WP 2), led by prof. Jaroslava Kubatová and David Kosina (Univ. UPOL) and in close collaboration with prof. Chris Brown and Ruth Luzmore (Univ. Southampton) consists of carrying out an in-depth analysis of the terminology and taxonomy of soft skills for a standardization of these competencies within the European space as well as the identification of the most key and necessary potential skills in a contemporary industrial-labor context. Among others, some results of this very extensive research (still to be completed) have already materialized in an article titled “CREA – the essential soft skills for the post-2022 world” (Kubátová et al., 2023). Ideally, all this research and its results will be incorporated into the knowledge base of the European Union agencies that manage everything related to skills, hard and soft, their certifications and the market demand for them.
Skills list (Selection Criteria)
As a second step to the executive task of analysis, taxonomy and definition of the most key soft skills in a Western framework of demand in the labor sector, it was necessary to carry out a selection of the skills that would form the core of the project and, therefore, from the subsequent experiment. In this way, a first selection of competencies was carried out through a survey in two phases: qualitative (interviews with different experts in the field of business and education) and quantitative (a form answered by some 99 organizations from various sectors). The objective of this action was to obtain a first screening of the complete list of skills provided by WP 2.
From the initial list provided in the survey, the following skills were selected:
- Adaptability/Flexibility
- Communication
- Creativity
- Critical thinking
- Empathy
- Leadership
- Motivation
- Problem solving
- Self-awareness
- Self-regulation
- Social skill
- Teamwork/Collaboration
- Time management
The next step was to develop a selection criterion of between three and four skills, so that we could limit the successive phases of the project to a manageable dimension based on time and resource requirements. The following parameters were established as Selection Criteria:
- Relevance based on Academic Arguments. The arguments extracted from the research carried out by the WP2 team were used.
- Relevance based on Business/Industrial Arguments. The arguments extracted from the two-level survey carried out with business and educational actors were used.
- Potential Measurement Feasibility. That is, the capacity that we intuit based on our previous analysis and experience, these skills can be measured using commercial video games in an online experience.
- Potential Training Viability. That is, the capacity that we intuit based on our previous analysis and experience, these skills can be trained using commercial video games in an online experience.
Of course points 3 and 4 are part of a kind of tautological dead-end loop, that is, the reason for this experiment is precisely to measure the potential of various soft skills to be measured with video games and now, we are precisely using that argument, not yet validated, for its selection. However, it is true that although this argument has not yet been validated within MEGASKILLS, we can speak of a “pre-validation” based on previous experiments carried out in other projects (of lesser significance) and on our experience in pedagogically based projects and coaching using commercial video games. To give an example, we know that skills such as “Communication” can be measured qualitatively in a face-to-face cooperative or competitive multiplayer game experience, but that an entire semantic identification technology for language would have to be developed if we wanted to do that same measurement in an experience with the same characteristics but not in person (this is: online).
The final evaluation taking into account the previously mentioned Selection Criteria were:
- Adaptability/Flexibility
- Critical Thinking
- Problem Solving
Additionally, and at the request of one of the members of the consortium (European Entrepreneurs CEA-PME), Time Management was included because it was classified as vitally important in surveys of companies and educational organizations.
The final list, with a very brief definition of each skill, would be as follows:
- Adaptability/Flexibility. Adaptability is a soft skill that empowers individuals to navigate and adjust to new circumstances, challenges, and changing environments.
- Critical Thinking. Critical thinking is the ability to objectively and logically analyse and evaluate information in order to form well-reasoned judgments and make informed decisions.
- Problem Solving. Problem-solving is the aptitude to identify, analyse, and effectively resolve issues by employing a range of strategies and techniques.
- Time Management. Time management is the ability to effectively plan, organize, and prioritize tasks and activities within a given timeframe. It involves allocating appropriate time to different responsibilities, setting realistic goals, and meeting deadlines.
Once we had obtained the limited list of soft skills to measure and train, we were ready to continue with the next phase of the project, that is, Work Package 3, consisting of finding evidence on the relationship between playing video games and development. of the selected soft skills.
Conceptual map for the “Critical Thinking” skill of D2.2 within the MEGASKILLS WP2.
Beginning of soft skills tasks – video games
Although GECON.es is integrated as a collaborating party in the rest of the Work Packages (WP) of the project, it is in WP3 in which it is designated as a leader. This WP3 is titled “Psycho-pedagogical methodology for detection and training of soft skills with commercial video games.” The objectives set for this work package are the following:
- To Select and review current soft skills measurement and training methodologies. This includes, above all, a selection of the most appropriate tests to measure these competencies.
- To Evidence the links between the use of video games and their ability to train soft skills.
- To digitally parameterize the metrics created to measure the video game’s ability to train soft skills.
- Finally, to configure an accessible methodology to incorporate all these results in a classroom context in the easiest and most intuitive way possible both in educational and work contexts, thus facilitating upskilling and reskilling tasks.
In our review of current methodologies and evidence of links between video games and soft skills training capacity, we have come across at least twenty different papers that justify, either through experiments or through a systematic literature review, that there is a relationship between various skills and the use of video games. Of course, these studies contain certain particularities that make a literal translation from their experimental models to ours barely impossible due to their differences: the skills selected by this project are not exactly the same and neither are the video games that we will use to measure them. On the other hand, these references do help us to configure an experiment based on a more standardized system of measurement and experimentation. That is to say, in some of these documented experiments we find, in the best of cases, soft skills similar to our definitions (remember that there is still no consensus on this matter), video games with similar characteristics and tests, to a certain extent, analogous to those selected by our team. Even in these cases, the variables of the experiment cannot be adapted directly or linearly but rather serve as inspiration and reference at a structural level, that is, for the design of the structure of the experiment, that is: finding the analogies between the indicators of the tests and the mechanics of the video games, then carry out a pre-test with several groups among which there will be a control group, then have them play the selected games and, finally, do the post-test to see if they are significant relationships between the use of the game and skill training.
The References
Among the increasingly numerous academic articles that attempt to establish significant relationships between video games and soft skills, we find the experiment by Adachi & Willoughby (2013) in which almost 2,000 adolescents are investigated based on the frequency with which they play certain video games cataloged as role and strategy indicate a positive prevalence of self-reported tests in problem solving, however, we cannot directly connect this research with ours because the type of games being used and the criteria for cataloging them that way is not specified.
In Barr’s (2017) study, 100 students play video games to test “generic skills” such as problem solving, communication, initiative or adaptability. In this case, the video games used by the students are identified, among which are some present on popular PC or console platforms during the execution of the experiment, such as Papers Please, Portal 2 or TF2. The result at the end of the research project was that there was a positive effect on skills such as communication, adaptability and initiative by playing these games.
The studies analyzed present experimental diversity both in the number of subjects involved, the number of hours played, the typology of the games proposed and the diversity of tests used. All but one study are unanimous regarding the positive relationships between the use of video games and the development of certain competencies or soft skills. The study by Unsworth et al. (2015) infers that using larger samples of subjects in an experiment of this type could demonstrate that studies of the relationships between video games and certain skills could be spurious when the groups of participants are smaller given that in larger groups the relationships are diluted and cannot be taken as significant.
In another methodological magnitude we would find studies that do not use tests but rather MRI-type technologies that attempt to link structural changes in the central nervous system to justify that video games have a positive impact on the acquisition of certain skills but that they can also generate a certain atrophy in others due to the hyperstimulation of the former. In the case of Gong (2015), through the comparison between the morphological particularities of professional and non-professional players, he found that expert players of action games had greater functional connectivity between insular subregions and also in anterior-posterior integration and left lateralization which would mean changes in functions related to attention and sensorimotor, among others. The study carried out by researcher Simona Kühn (2014) has the video game Super Mario 64 DS as its protagonist and found significant increases in the volume of gray matter in certain areas of the right hippocampus, right dorsolateral prefrontal cortex and bilateral cerebellum, areas crucial for spatial navigation, strategic planning, working memory and motor performance, along with evidence of behavioral changes in navigation strategy.
The Experiment
To connect what happens in our mind while we play video games, it is necessary to carry out an experiment that tests our hypotheses. In this experiment we have formulated about thirty hypotheses and sub-hypotheses, however the main one remains the same: “Do video games have a positive influence on the development of soft skills compared to the control group?” Taking into account that the control group will take the tests but will not play the proposed video games.
To give a summarized example of this link between the tests and their rubrics, we can say that one of the tests selected to measure the Problem Solving skill of an individual goes through “CPS2. Learn from previous problems and improve your performance and efficiency when solving new problems.” Subsequently, a matrix is created listing the video games that contain mechanics in which the player, in one way or another, is evaluated based on their ability to improve their performance and efficiency as they solve increasingly complex situations but with relationships to their previous challenges. We parameterize these parallel rubrics in video games thanks to the use of Steam achievements. After comparing a list of potentially compatible video games, those with a greater compatibility rate between their mechanics and the rubrics of the standard tests were selected, resulting in the following list:
- Complex Problem Solving – Train Valley 2*
- *Special thanks to the FLAZM team for providing us with the necessary licenses to carry out this experiment.
- Critical Thinking – Lightseekers
- Flexibility/Adaptability – Relic Hunters Zero: Remix
- Time Management – Minion Masters
Smart assessment and digital certification
One of the novelties that we include in our research trajectory in the relationships between soft skills and video games comes from the hand of the project leader. TECNALIA will incorporate its Artificial Intelligence team with the aim of establishing relationships between gamers’ behaviors within the game and the results of the tests through the identification of specific patterns. The advantage of using advanced artificial intelligence algorithms is that an immense amount of information can be processed from interactions with games and test data. It will also be possible to generate comparisons between participating users so that we can find patterns not only of improvement but also of projection and potential, as well as improve the detection model and even, in the future, be able to automate the cataloging procedure of games based on their training capabilities for these skills.
The tasks of this part of the project involve training the AI to be able to identify the rubric system of the tests that we use and find their equivalences in the game mechanics, detecting whether the relationships between them are significant enough to corroborate that soft skills learning really happens and how can be measured.
The most technological part of the project culminates with a digital and standard accreditation system for the skills acquired through the MEGASKILLS platform. Our purpose is for these certifications to be protocolized with the European Europass curriculum vitae system standards.
A platform adapted to a pedagogical implementation model
In the MEGASKILLS project we clearly differentiate what is the development of a technological innovation in the educational sector from its implementation in a pedagogical context. The creation of a technology does not mean that it will be naturally and intuitively accepted in a classroom and in a job training context. That is why it is necessary to propose a methodology to solve potential challenges in the use of the platform for measuring soft skills with video games, among which are: teachers and/or students with deficiencies in digital skills and gamer culture, deficient knowledge in soft skills (importance and definition), learning process perceived as a memorable and enjoyable experience, flexibility in adapting content and experience to the receiving sectors, among others.
Once the pedagogical design is completed, several pilot-tests will be executed in collaboration with our partner CEGOS to evaluate the performance of the platform and the satisfaction of the experience. Our intention is to collect enough information and suggestions to refine the model and turn it into the reference pedagogical system for the training and measurement of soft skills. Also generate the necessary materials for both students and teachers and thus be able to eliminate the gaps mentioned above.
References
Adachi, P. J. C., & Willoughby, T. (2013). More than just fun and games: the longitudinal relationships between strategic video games, self-reported problem solving skills, and academic grades. Journal of Youth and Adolescence, 42(7), 1041–1052. https://doi.org/10.1007/s10964-013-9913-9
Barr, M. (2017). Video games can develop graduate skills in higher education students: A randomised trial. Computers and Education, 113, 86–97. https://doi.org/10.1016/j.compedu.2017.05.016
Gong, D., He, H., Liu, D., Ma, W., Dong, L., Luo, Ch. & Yao, D. (2015). Enhanced functional connectivity and increased grey matter volume of insula related to action video game playing. Scientific Reports, 5(9763). https://doi.org/10.1038/srep09763
Kühn, S., Gleich, T., Lorenz, R. C., Lindenberger, U., & Gallinat, J. (2014). Playing Super Mario induces structural brain plasticity: gray matter changes resulting from training with a commercial video game. Molecular Psychiatry, 19(2), 265–271. https://doi.org/10.1038/mp.2013.120
Kubátová, J., Kosina, D., Kročil, O., Müller, M., & Slavíčková, P. (2023). CREA – the essential soft skills for the post-2022 world. In P. Slavíčková & M. Müller (Eds.), Knowledge on Economics and Management. Conference Proceedings (pp. 89-95). Palacký University Olomouc. ISBN 978-80-244-6390-2.
https://kems.upol.cz/files/FF/katedry/kae/knowcon/proceedings/KNOWCON_2023_final.pdf (p. 89)
Unsworth, N., Redick, T. S., McMillan, B. D., Hambrick, D. Z., Kane, M. J., & Engle, R. W. (2015). Is playing video games related to cognitive abilities? Psychological Science, 26(6), 759–774. https://doi.org/10.1177/0956797615570367