View on GitHub

phd-thesis-evaluation

Empirical Study 03

Comparing ontology-based gamified CL sessions (ont-gamified) against CL sessions gamified without using the ontology OntoGaCLeS (w/o-gamified).

All materials used in the empirical study are available at (materials)

Research Questions (RQs)

  1. Do the ont-gamified CL sessions have positive impacts on the students’ motivation?
  2. Do the ont-gamified CL sessions affect on the students’ learning outcomes?
  3. Are the participants’ motivation and learning outcomes linked in those sessions?

Scripts to answer the RQ1

RQ1: Do the ont-gamified CL sessions have positive impacts on the students' motivation?

R script to validate the IMI questionnaire (00-reliability-analysis-IMI.R)

R script to validate data gathered in the empirical study through the IMI questionnaire.

The results obtained by the execution of this R script are:

This validation of the IMI questionnarie was carried out with all the data gathered throughout all empirical studies, and this procedure is detailed in ../report/validation-IMI/.

R script to validate the IMMS questionnaire (00-reliability-analysis-IMMS.R)

R script to validate data gathered in the empirical study through the IMMS questionnaire.

The results obtained by the execution of this R script are:

This validation of the IMMS questionnarie was carried out with all the data gathered throughout all empirical studies, and this procedure is detailed in ../report/validation-IMMS/.

R script to find significant differences on the scores of motivation for all students (01-scr-motivation-signedup-analysis.R)

R script to find significant differences on the motivation scores obtained through the IMI and IMMS questionnaire, and for all the students signed-up as participants in the CL sessions. These differences were firstly finding by using the nonparametric statistical tests as the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, by using the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found on the students’ motivation.

The results obtained by the execution of this R script are:

R script to find significant differences on the scores of motivation for students with effective participation (01-scr-motivation-effective-analysis.R)

R script to find significant differences on the motivation scores obtained through the IMI questionnaire, and for the students with effective participation in the CL sessions. These differences were firstly finding by using the nonparametric statistical tests as the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, by using the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found on the students’ motivation.

A student with effective participation is a student that, at least one time, interacted with other member of the CL group by following the necessary interactions indicated in the CSCL script

The results obtained by the execution of this R script are:

R script to calculate the IRT-based estimates of students’ motivation (00-rsm-motivation-measurement-building.R)

R script to calculate the IRT-based estimates of students’ motivations through the building of RMS-based measurement instruments. These estimates were calculated based on the Item Response Theory (IRT) and Rating Scale Model (RSM), and through the process detailed in the file: irt-instruments.pdf.

The results obtained by the execution of this R script are:

R script to find significant differences on the IRT-based estimates of motivation for all students (01-rsm-motivation-signedup-analysis.R)

R script to find significant differences on the IRT-based estimates of motivation obtained through the RSM-based measurment instrument of motivation built from the IMI questionnaire. These differences were find on the estimates of motivation for all the students signed-up as participants in the CL sessions, and they were firstly finding by using the nonparametric statistical tests as the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, by using the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found on the students’ motivation.

The results obtained by the execution of this R script are:

R script to find significant differences on the IRT-based estimates of motivation for students with effective participation (01-rsm-motivation-effective-analysis.R)

R script to find significant differences on the IRT-based estimates of motivation obtained through the RSM-based measurment instrument of motivation built from the IMI questionnaire. These differences were finding on the estimates of motivation for students with effectie participantion in the CL sessions, and they were firstly finding by using the nonparametric statistical tests as the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, by using the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found on the students’ motivation.

A student with effective participation is a student that, at least one time, interacted with other member of the CL group by following the necessary interactions indicated in the CSCL script

The results obtained by the execution of this R script are:

Scripts to answer the RQ2

RQ2: Do the ont-gamified CL sessions affect on the students' learning outcomes?

R script to find significant differences on the gain scores of academic performance tests for all students (02-scr-learning-outcomes-signedup-analysis.R)

R script to find statistical significant differences on the gain scores (posttest - pretest) obtained through the AMC questionnaires and solving-programming problem tasks. This statistical analysis was carried out with data gathered from the responses of all the students signed-up as participants in the CL sessions. Firstly, a nonparametric analysis was carried out by using the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, through the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found by the nonparametric analysis.

The results obtained by the execution of this R script are:

R script to find significant differences on the gain scores of academic performance tests for students with effective participation (02-scr-learning-outcomes-effective-analysis.R)

R script to find statistical significant differences on the gain scores (posttest - pretest) obtained through the AMC questionnaires and solving-programming problem tasks. This statistical analysis was carried out with data gathered from the responses of students with effective participation in the CL sessions. Firstly, a nonparametric analysis was carried out by using the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, through the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found by the nonparametric analysis.

The results obtained by the execution of this R script are:

R script to calculate the IRT-based estimates of learning outcomes (00-gpcm-learning-outcomes-measurement-building.R)

R script to calculate the IRT-based estimates of the students’ gains in skill/knowledge. These estimates were calculated using the stacking process based on the General-Partial Credit Model (GPCM), and this process is detailed in the file: irt-instruments.pdf.

The results obtained by the execution of this R script are:

R script to find significant differences on the estimates of skill/knowledge gains for all students (02-gpcm-learning-outcomes-signedup-analysis.R)

R script to find statistical significant differences on the estimates of skill/knowledge gains (posttest - pretest) obtained through the AMC questionnaires and solving-programming problem tasks. This statistical analysis was carried out with data gathered from the responses of all the students signed-up as participants in the CL sessions. Firstly, a nonparametric analysis was carried out by using the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, through the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found by the nonparametric analysis.

The results obtained by the execution of this R script are:

R script to find significant differences on the estimates of skill/knowledge gains for students with effective participation (02-gpcm-learning-outcomes-effective-analysis.R)

R script to find statistical significant differences on the estimates of skill/knowledge gains (posttest - pretest) obtained through the AMC questionnaires and solving-programming problem tasks. This statistical analysis was carried out with data gathered from the responses of students with effective participation in the CL sessions. Firstly, a nonparametric analysis was carried out by using the Scheirer-Ray-Hare test and the pair-wilcoxon test. After this, through the parametric tests of ANOVA and Tukey post-hoc, we confirmed the statistical significant differences found by the nonparametric analysis.

The results obtained by the execution of this R script are:

Scripts to answer the RQ3

RQ3: Are the participants' motivation and learning outcomes linked in ont-gamified and non-gamified CL sessions?

R script to find significant correlations between motivation scores and gain scores on academic performance tests for all the students (03-scr-correlation-signedup-analysis.R)

R script to statistically find significant correlations between students’ motivation and their gains of skill/knowledge during the empirical study. The students’ motivation for this correlation analysis was carried out employing the motivation scores calculated through the IMI questionnaires, and for all the students signed-up as participants in the CL sessions. The students’ gains of skill/knowledge were calculated by teacher-based rules on the AMC questionnaires and programming problem tasks. Spearman’s Rank method was also used for this correlation analysis.

The results obtained by the execution of this R script are:

R script to find significant correlations between motivation scores and gain scores on academic performance tests for students with effective participation (03-scr-correlation-effective-analysis.R)

R script to statistically find significant correlations between students’ motivation and their gains of skill/knowledge during the empirical study. The students’ motivation for this correlation analysis was carried out employing the motivation scores calculated through the IMI questionnaires, and for students with effective participation in the CL sessions. The students’ gains of skill/knowledge were calculated by teacher-based rules on the AMC questionnaires and programming problem tasks. Spearman’s Rank method was also used for this correlation analysis.

A student with effective participation is a student that, at least one time, interacted with other member of the CL group by following the necessary interactions indicated in the CSCL script

The results obtained by the execution of this R script are:

R script to find significant correlations between the estimates of motivation and the gain of skill/knownedge for all the students (03-irt-correlation-signedup-analysis.R)

R script to statistically find significant correlations between students’ motivation and their gains of skill/knowledge during the empirical study. The students’ motivation for this correlation analysis was carried out employing the IRT-based estimates of motivation calculated through the RSM-based intruments, and for all the students signed-up as participants in the CL sessions. The students’ gains of skill/knowledge were estimated by GPCM-based instruments on the AMC questionnaires and programming problem tasks. Spearman’s Rank method was also used for this correlation analysis.

The results obtained by the execution of this R script are:

R script to find significant correlations between the estimates of motivation and the gain of skill/knownedge for students with effective participation (03-scr-correlation-effective-analysis.R)

R script to statistically find significant correlations between students’ motivation and their gains of skill/knowledge during the empirical study. The students’ motivation for this correlation analysis was carried out employing the IRT-based estimates of motivation calculated through the RSM-based intruments, and for students with effective participantion in the CL sessions. The students’ gains of skill/knowledge were estimated by GPCM-based instruments on the AMC questionnaires and programming problem tasks. Spearman’s Rank method was also used for this correlation analysis.

A student with effective participation is a student that, at least one time, interacted with other member of the CL group by following the necessary interactions indicated in the CSCL script

The results obtained by the execution of this R script are: