Evaluation of the Ontological Engineering Approach to Gamify Collaborative Learning Scenarios
Empirical Studies
Scripts in this folder - path: /
For the validation of the IMI and IMMS questionnaires, the R scripts should be executed as follows:
R script to extract information from the Moodle platform (00-processing-mysql.R)
R script to extract information from the MySQL database of the Moodle platform.
The results obtained by the execution of this R script are:
- List of all students as participants in the pilot empirical study (lines 15-70)
File: SignedUpParticipants.csv (more info …) - List of students with effective participation in the pilot empirical study (lines 72-80)
File: EffectiveParticipants.csv (more info …) - Experiment design for the pilot empirical study (lines 81-114)
File: pilot_design.csv (more info …) - Information from the IMI questionnaire - pilot empirical study (lines 120-205)
File (legend): SourceIMILegend.csv (more info …)
File (responses): SourceIMIWithCareless.csv (more info …) - List of all students as participants in the first empirical study (lines 210-272)
File: SignedUpParticipants.csv (more info …) - List of all students as participants in the second empirical study (lines 273-325)
File: SignedUpParticipants.csv (more info …) - List of all students as participants in the third empirical study (lines 326-395)
File: SignedUpParticipants.csv (more info …) - List of students with effective participation in the full-scale empirical studies (400-425)
File (first empirical study): EffectiveParticipants.csv (more info …)
File (second empirical sudy): EffectiveParticipants.csv (more info …)
File (third empirical study): EffectiveParticipants.csv (more info …) - Experiment designs for the full-scale empirical studies (lines 427-508)
File: exp_design.csv (more info …) - Information from the IMI and IMMS questionnaires - third empirical study (lines 510-620)
File (legend): SourceMotLegend.csv (more info…)
File (responses from the IMI questionnaire): SourceIMIWithCareless.csv (more info …)
File (responses from the IMMS questionnaire): SourceIMMSWithCareless.csv (more info …)
R script to extract information from AMC questionnaires (00-processing-amc.R).
R script to extract information from responses of AMC (Auto-Multiple-Choice) questionnaires. Such questionnaires were built to measure the skill/knowledge of students throughout the empirical studies, and they were built using the AMC software - https://www.auto-multiple-choice.net.
The results obtained by the execution of this R script are:
- Data with responses from the AMC questionnaire of conditional structures p1a. Responses obtained throughout the pretest phase of the first empirical study and scored using the GPCM-based rule.
File: PreAMC.csv (more info …) - Data with responses from the AMC questionnaire of conditional structures p1b. Responses obtained throughout the posttest phase of the first empirical study and scored using the GPCM-based rule.
File: PosAMC.csv (more info …)] - Data with responses from the AMC questionnaire of conditional structures p2a. Responses obtained throughout the pretest phase of the first empirical study and scored using the teacher’s rule.
File: PreAMCscr.csv (more info …) - Data with responses from the AMC questionnaire of conditional structures p2b. Responses obtained throughout the posttest phase of the first empirical study and scored using the teacher’s rule.
File: PosAMCscr.csv (more info …) - Data with responses from the AMC questionnaire of loop structures p2a. Responses obtained throughout the pretest phase of thr second empirical study and scored using the GPCM-based rule.
File: PreAMC.csv (more info …) - Data with responses from the AMC questionnaire of loop structures p2b. Responses obtained throughout the posttest phase of the second empirical study and scored using the GPCM-based rule.
File: PosAMC.csv (more info …) - Data with responses from the AMC questionnaire of loop structures p2a. Responses obtained throughout the pretest phase of the second empirical study and and scored using the teacher’s rule.
File: PreAMCscr.csv (more info …) - Data with responses from the AMC questionnaire of loop structures p2b. Responses obtained throughout the posttest phase of the second empirical study and scored using the teacher’s rule.
File: PosAMCscr.csv (more info …) - Data with responses from the AMC questionnaire of recursion p3a. Responses obtained throughout the pretest phase of the third empirical study and scored using the GPCM-based rule.
File: PreAMC.csv (more info …) - Data with responses from the AMC questionnaire of recursion p3c. Responses obtained throughout the posttest phase of the third empirical study and scored using the GPCM-based rule.
File: PosAMC.csv (more info …) - Data with responses from the AMC questionnaire of recursion p3a. Responses obtained throughout the pretest phase of the third empirical study and scored using the teacher’s rule.
File: PreAMCscr.csv (more info …) - Data with responses from the AMC questionnaire of recursion p3c. Responses obtained throughout the posttest phase of the third empirical study and scored using the teacher’s rule.
File: PosAMCscr.csv (more info …)
R script to extract information related to the VPL Moodle plugin (00-processing-vpl.R)
R script to extract information related to the adapted version of VPL (Virtual-Programming-Lab) Moodle plugin. This adapted VPL Moodle plugin with code recording log can be download from: https://github.com/geiser/moodle-mod_vpl.
The results obtained by the execution of this R script are:
- Information of the programming problem tasks solved throughout the pretest phase of the pilot empirical study, and scored with Guttman-based rules.
File: PreGuttmanVPL.csv (more info …) - Information of the programming problem tasks solved throughout the posttest phase of the pilot empirical study, and scored with Guttman-based rules.
File: PosGuttmanVPL.csv (more info …) - Information of the programming problem tasks solved throughout the pretest phase of the first empirical study, and scored with Guttman-based rules.
File: PreGuttmanVPL.csv (more info …) - Information of the programming problem tasks solved throughout the pretest phase of the first empirical study, and scored with Guttman-based rules.
File: PosGuttmanVPL.csv (more info …) - Information of the programming problem tasks solved throughout the pretest phase of the second empirical study, and scored with Guttman-based rules.
File: PreGuttmanVPL.csv (more info …) - Information of the programming problem tasks solved throughout the pretest phase of the second empirical study, and scored with Guttman-based rules.
File: PosGuttmanVPL.csv (more info …) - Information of the programming problem tasks solved throughout the pretest phase of the third empirical study, and scored with Guttman-based rules.
File: PreGuttmanVPL.csv (more info …) - Information of the programming problem tasks obtained throughout the pretest phase of the third empirical study, and scored with Guttman-based rules.
File: PosGuttmanVPL.csv (more info …)
R script to remove careless responses (01-removing-careless-motivation.R)
R script to remove careless responses on the data gathered by the IMI and IMMS questionnaires. A careless response is defined as a response in which the length of uninterrupted identical values for the items is greater than half of the items in the questionnaire.
The results obtained by the execution of this R script are:
- Careless responses identified on the data gathered throughout the pilot empirical study and using the IMI questionnaire.
File: careless-IMI-pilot-study.pdf (more info …) - Responses of the IMI questionnaire with data gathered throughout the pilot empirical study and without careless responses.
File: SourceIMI.csv (more info …) - Careless responses identified on the data gathered throughout the first empirical study and using the IMI questionnaire.
File: careless-IMI-study01.pdf (more info …) - Responses of the IMI questionnaire with data gathered throughout the first empirical study and without careless responses.
File: SourceIMI.csv (more info …) - Careless responses identified on the data gathered throughout the second empirical study and using the IMMS questionnaire.
File: careless-IMMS-study02.pdf (more info …) - Responses of the IMMS questionnaire with data gathered throughout the second empirical study and without careless responses.
File: SourceIMMS.csv (more info …) - Careless responses identified on the data gathered throughout the third empirical study and using the IMI questionnaire.
File: careless-IMI-study03.pdf (more info …) - Responses of the IMI questionnaire with data gathered throughout the third empirical study and without careless responses.
File: SourceIMI.csv (more info …) - Careless responses identified on the data gathered throughout the third empirical study and using the IMMS questionnaire.
File: careless-IMMS-study03.pdf (more info …) - Responses of the IMMS questionnaire with data gathered throughout the third empirical study and without careless responses.
File: SourceIMMS.csv (more info …)
R script to winsorize extreme responses (02-winsorizing-extreme-motivation.R)
R script to winsorize extreme responses from the data gathered through the IMI and IMMS questionnaires. Extreme responses correspond to answers given by participants who have tendency to indicate extreme lower and upper values in questionnaires. Such responses are outliers that affect the assumptions for parametric tests, but that can’t simply removed to satisfy these assumptions. To reduce the impact of extreme values, they should be shrunk to the border of the main part of the data through the winsorization method.
The results obtained by the execution of this R script are:
- Extreme responses identified on the IMI questionnaire.
File: winsorized-IMI.pdf (more info …) - Extreme responses identified on the IMMS questionnaire.
File: winsorized-IMMS.pdf (more info …) - Data with winsorized responses to validate the IMI questionnaire.
File: WinsorizedIMI.csv (more info …) - Data with winsorized responses to validate the IMMS questionnaire.
File: WinsorizedIMMS.csv (more info …)
R script to validate the IMI questionnaire (03-validating-IMI.R)
R script to validate the adapted Portuguese IMI questionnaire through the exploratory/confirmatory factorial analysis and the reliability test.
The results obtained by the execution of this R script are:
- Validation of univariate distribution for each items.
File: univariate-histogram.png (more info …) - Goodness of fit statistics for the factorial analysis on the responses of IMI questionnaire.
File: cfa-model-fit.pdf (more info …) - Summary of the factorial analysis and reliability test.
File: reliability-analysis.pdf (more info …) - Report for the measure sampling adequacy, factorial analysis, and reliability test.
File: RelAnalysis.xlsx (more info …) - Data with the responses of the validated IMI questionnaire.
File: IMI.csv (more info …)
R script to validate the IMMS questionnaire (03-validating-IMMS.R)
R script to validate the adapted Portuguese IMMS questionnaire through the exploratory/confirmatory factorial analysis and the reliability test.
The results obtained by the execution of this R script are:
- Validation of univariate distribution for each items.
File: univariate-histogram.png (more info …) - Goodness of fit statistics for the factorial analysis on the responses of IMI questionnaire.
File: cfa-model-fit.pdf (more info …) - Summary of the factorial analysis and reliability test.
File: reliability-analysis.pdf (more info …) - Report for the measure sampling adequacy, factorial analysis, and reliability test.
File: RelAnalysis.xlsx (more info …) - Data with the responses of the validated IMMS questionnaire.
File: IMMS.csv (more info …)
Install & Run the R Scripts
Requirements:
- Docker (>= 18.06.0)
- Docker Compose (>= 1.22.0)
Procedure to install requirements in Ubuntu 18.04:
sudo apt-get purge docker docker-engine docker.io docker-ce
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce
Procedure to update docker compose in Ubuntu 18.04
sudo curl -L "https://github.com/docker/compose/releases/download/1.23.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose --version
The output will look something like this: docker-compose version 1.23.1, build b02f1306
Setup the docker as service and run it at startup
sudo systemctl start docker
sudo systemctl enable docker
- Download source code:
git clone https://github.com/geiser/phd-thesis-evaluation.git - Configure and build imagens to run the project as container
cd phd-thesis-evaluation ./configure make
- Use
make shellto run the/bins/bashon the containergeiser/r-studio:latest - Use
make runto run the r-studio and execute the scripts through the http://127.0.0.1:8787/ - Use
make helpto view more options