**4. Results**

In this section we present and analyze the course results in terms of students' performance and satisfaction.

#### **4.1 Performance analyses**

Dr. Scratch is an online tool (http://drscratch.org/) that assesses Scratch projects with respect to seven "dimensions," namely, logical thinking (LT), data-information representation (IR), user interactivity (IN), flow control (FC), abstraction (AB) and problem decomposition, parallelism (PA), and synchronization (SN). A project can be graded (from 0 to 3) for each dimension in one of the levels, depending on the level of sophistication achieved by the project code [25, 26]. Thus, a total evaluation ranges from 0 to 21 (7 dimensions multiplied by [0–3]). We analyzed 15 different projects. The projects gathered were scored with values ranging between 10 and 20 (see **Table 1**).

Similar to other studies [27], this study revealed challenges with respect to the use of concepts, such as the parallelism and synchronization. Also, very few applications made use of random numbers and logical expressions. On the contrary,

**193**

*Evaluating a Course for Teaching Advanced Programming Concepts with Scratch to Preservice…*

Mean 1.88 1.54 2.16 1.81 1.68 0.72 1.84 Std. dev. 0.38 0.29 0.41 0.32 0.29 0.22 0.41 Minimum 0 0 1 1 1 1 1 Maximum 3 3 3 3 3 2 3

*PA LT FC IN IR AB SN*

**Statistical measure Dimension of computational thinking**

the frequently used coding concepts such as flow control and user interactivity reveal that the students in their projects make an adequate use of specific conditions and foresee users' interaction. Except the fact that Dr. Scratch provides feedback on several aspects which are related to computational thinking, the software categorizes the project developer skills in three different categories/levels: Basic, Developing, and Master. The 15% of the developed apps were "Basic," and 85% were "Developing." There were no projects on the "Master" level. **Image 6** shows an example of how Dr. Scratch categorizes developer skills. The screenshots of the graphical user interface, code parts, and Dr. Scratch scores of five randomly

To evaluate students' self-efficacy in utilizing programming and computational thinking within their future teaching endeavors, we adapted the Teachers' Self-Efficacy in Computational Thinking (TSECT) [4]. We used the first seven of the nine TSECT items (see **Table 2**). All items use a five-point Likert scale with options of strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. TSECT was given as a pre- and posttest before and after the intervention.

selected projects are displayed in the Appendix.

Questionnaire analysis was performed with SPSS 23.

**4.2 Students' self-efficacy analyses**

*Example of Dr. Scratch project evaluation.*

*DOI: http://dx.doi.org/10.5772/intechopen.81714*

**Table 1.**

**Image 6.**

*Project score given by Dr. Scratch.*


*Evaluating a Course for Teaching Advanced Programming Concepts with Scratch to Preservice… DOI: http://dx.doi.org/10.5772/intechopen.81714*

#### **Table 1.**

*Early Childhood Education*

*3.4.2 Evaluation*

and qualitative data:

support tool

ties in your lessons? Why?").

performance and satisfaction.

**4.1 Performance analyses**

**4. Results**

13 weeks. The lessons were 3 hours per week. The course was offered as optional, and the students took part in the study after ethics approvals were received and all participants signed consent forms. All research participants had basic computer skills, but they had no previous experience with neither computational thinking nor

In order to evaluate the course, we examined both cognitive (how effectively they learned) and affective (how enjoyable the experience was, and how motivated by it the students were) factors. Thus, in this study we collected both quantitative

• The quantitative part was conducted in pretest/posttest quasi-experimental design. Moreover, to understand the learning of programming topics, we evaluated students' projects in terms of students' use of the elements of Scratch language as well as the project functionality and appearance. For that reason, students' project(s) were examined by using the Dr. Scratch tool.

• The qualitative approach used a short questionnaire and semi-structured interviews. Data were recorded through field notes, made by the researchers.

○ The conception about the potential of Scratch and CT activities as a learning

The respondents were asked to answer both to closed questions (yes/not) and open questions ("Do you think that Scratch and coding activities can be a useful support learning tool? Why?," "Do you think about introducing some coding activi-

In this section we present and analyze the course results in terms of students'

Dr. Scratch is an online tool (http://drscratch.org/) that assesses Scratch projects with respect to seven "dimensions," namely, logical thinking (LT), data-information representation (IR), user interactivity (IN), flow control (FC), abstraction (AB) and problem decomposition, parallelism (PA), and synchronization (SN). A project can be graded (from 0 to 3) for each dimension in one of the levels, depending on the level of sophistication achieved by the project code [25, 26]. Thus, a total evaluation ranges from 0 to 21 (7 dimensions multiplied by [0–3]). We analyzed 15 different projects. The projects gathered were scored with values ranging between 10 and 20 (see **Table 1**). Similar to other studies [27], this study revealed challenges with respect to the use of concepts, such as the parallelism and synchronization. Also, very few applications made use of random numbers and logical expressions. On the contrary,

This approach aimed at evaluating essentially three points:

○ The intention to introduce a CT curriculum ○ The level of satisfaction about the course

the use of Scratch or any other programming environment.

**192**

*Project score given by Dr. Scratch.*

#### **Image 6.**

*Example of Dr. Scratch project evaluation.*

the frequently used coding concepts such as flow control and user interactivity reveal that the students in their projects make an adequate use of specific conditions and foresee users' interaction. Except the fact that Dr. Scratch provides feedback on several aspects which are related to computational thinking, the software categorizes the project developer skills in three different categories/levels: Basic, Developing, and Master. The 15% of the developed apps were "Basic," and 85% were "Developing." There were no projects on the "Master" level. **Image 6** shows an example of how Dr. Scratch categorizes developer skills. The screenshots of the graphical user interface, code parts, and Dr. Scratch scores of five randomly selected projects are displayed in the Appendix.

## **4.2 Students' self-efficacy analyses**

To evaluate students' self-efficacy in utilizing programming and computational thinking within their future teaching endeavors, we adapted the Teachers' Self-Efficacy in Computational Thinking (TSECT) [4]. We used the first seven of the nine TSECT items (see **Table 2**). All items use a five-point Likert scale with options of strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. TSECT was given as a pre- and posttest before and after the intervention. Questionnaire analysis was performed with SPSS 23.


#### **Table 2.**

*Modified TSECT instrument items.*

A t-test of the pre and post-survey TSECT scale revealed a statistically significant increase in TSECT from pre (*M* = 12.03, *SD* = 4.39) to post (*M* = 18.14, *SD* = 3.59), *t*(14) = 3.98, *p* < .0001. From the students' answers, we can conclude that after the intervention, they feel themselves confident enough to create projects and they plan to incorporate programming as an instructional tool in their future classrooms.

Furthermore, after completion of the course, the researchers conducted a focus group interview with a structured interview form. All the students noted that the added cognitive effort was worthwhile and decided to bring coding activities into the early childhood classroom. The students noted that they experienced a significant shift in mindset during the course. Before the course started, all students identified the lack of CT knowledge and skills as a major challenge. After the course, all of them could successfully define key CT concepts. They expressed a high degree of confidence that they taught the CT lessons effectively contributed to their learning. Moreover, all students noted that they made major leaps in correcting their misconceptions about what CT is and understanding fundamental CT concepts. After the focus group interview, the researchers noted that the majority of the students could explain what CT is and describe the main concepts covered during the course. It is also worth to mention that all students indicated that they would like to continue CT training in the following academic year, if that was possible. They also mentioned that they would recommend the course to other students.

#### **4.3 Limitations**

In this chapter, we studied how a course helped preservice teachers to learn and introduce CT concepts into their daily teaching practices as a new subject to their students. The programming and teaching behaviors that emerged still need to be validated through further studies. Furthermore, since the data was collected from female students from one university department, the findings should be applied to subjects from other disciplines with caution. Moreover, it may be useful to employ a mixed method approach that incorporates long-term practical research methods for a deeper investigation of factors affecting attitudes and intentions toward using Scratch in respect to the students' gender.

## **5. Discussion and conclusion**

Discussions about the appropriateness of technology in early childhood are mostly put aside, and the pressing question is not "Should we introduce

**195**

future.

ing approach.

*Evaluating a Course for Teaching Advanced Programming Concepts with Scratch to Preservice…*

computers?" but "How should we introduce them?" ([11] as cited in [7]). If coding is conceived as a skill that must begin to be taught early in life [8], and new curricula worldwide in preschool and primary education is covering computational thinking, digital technologies and related areas are being introduced, many preservice teachers are having to undergo professional development to be able to deliver the new material [13]. There are a number of obstacles to bringing coding into the classroom. Even putting obstacles such as the cost to training, teachers have a tendency to teach the way they were taught, and systemwide reform is difficult to implement. To properly bring hands-on learning (or coding, robotics) into the classroom, the classroom must change from a teacher lecturing to a

In this paper we described a course that we have developed at the Department of Preschool Education at the University of Crete in an attempt to help preservice teachers to learn CT concepts and programming. Owing to the fact that preservice teachers find it difficult to master the syntax of programming languages in general [23], we believe that the choice of visual programming language is an important factor in learning programming [18]. In this course we chose Scratch as the main programming environment to create an area for preservice teachers for their innovative ideas and a platform to cultivate preservice teachers' computational

The results, like other studies, show that by enhancing the course curriculum with Scratch and development projects in the Scratch environment, students' performance on CT improved significantly. Similar to Kim et al. [23] research results, we also agree that "Scratch helped pre-service teachers focus on what they could do with programming languages (p. 971)." Scratch helped preservice teachers to overcome their programming difficulties (e.g., syntax) and to focus on core aspects

As it is widely known, changes in learning and teaching practice in class can precede changes in teachers' attitudes and beliefs. Thus, the changes in attitude noted in this study suggest that the preservice teachers believe that Scratch would be a useful tool to do their job and using Scratch would enable them to use technology more effectively [4]. Similar to the study of Arpaci [2], preservice teachers think that using Scratch would increase their productivity, enhance their effectiveness, improve their job performance, and ease their job. Another important thing to consider is that students with no prior programming experience noted that Scratch

Based on the success of course, we made the following conclusions:

computational thinking before they teach in kindergarten.

• We believe that training preservice kindergarten teachers to coding is the best strategy to ensure that all in-service kindergarten teachers will have a technological literacy and computational thinking skills. By introducing coding in university, students will have enough time and exposure to acquire solid

• The majority of preservice teachers are willing to invest time and effort in training related to CT skills. They recognize that they need to have a technological literacy and computational thinking skills to be prepared for the

• There are CT education resources such as lessons and teaching materials available online which are suitable for the novice programmers. Those lessons and teaching resources can be implemented as curriculum which reflects a scaffold-

*DOI: http://dx.doi.org/10.5772/intechopen.81714*

teacher being a mentor [7].

of computational thinking [23].

had assisted them in learning programming.

thinking.

*Evaluating a Course for Teaching Advanced Programming Concepts with Scratch to Preservice… DOI: http://dx.doi.org/10.5772/intechopen.81714*

computers?" but "How should we introduce them?" ([11] as cited in [7]). If coding is conceived as a skill that must begin to be taught early in life [8], and new curricula worldwide in preschool and primary education is covering computational thinking, digital technologies and related areas are being introduced, many preservice teachers are having to undergo professional development to be able to deliver the new material [13]. There are a number of obstacles to bringing coding into the classroom. Even putting obstacles such as the cost to training, teachers have a tendency to teach the way they were taught, and systemwide reform is difficult to implement. To properly bring hands-on learning (or coding, robotics) into the classroom, the classroom must change from a teacher lecturing to a teacher being a mentor [7].

In this paper we described a course that we have developed at the Department of Preschool Education at the University of Crete in an attempt to help preservice teachers to learn CT concepts and programming. Owing to the fact that preservice teachers find it difficult to master the syntax of programming languages in general [23], we believe that the choice of visual programming language is an important factor in learning programming [18]. In this course we chose Scratch as the main programming environment to create an area for preservice teachers for their innovative ideas and a platform to cultivate preservice teachers' computational thinking.

The results, like other studies, show that by enhancing the course curriculum with Scratch and development projects in the Scratch environment, students' performance on CT improved significantly. Similar to Kim et al. [23] research results, we also agree that "Scratch helped pre-service teachers focus on what they could do with programming languages (p. 971)." Scratch helped preservice teachers to overcome their programming difficulties (e.g., syntax) and to focus on core aspects of computational thinking [23].

As it is widely known, changes in learning and teaching practice in class can precede changes in teachers' attitudes and beliefs. Thus, the changes in attitude noted in this study suggest that the preservice teachers believe that Scratch would be a useful tool to do their job and using Scratch would enable them to use technology more effectively [4]. Similar to the study of Arpaci [2], preservice teachers think that using Scratch would increase their productivity, enhance their effectiveness, improve their job performance, and ease their job. Another important thing to consider is that students with no prior programming experience noted that Scratch had assisted them in learning programming.

Based on the success of course, we made the following conclusions:


*Early Childhood Education*

**Item Wording**

**Table 2.**

other topics

educational tool

*Modified TSECT instrument items.*

1 I feel confident writing simple programs in Scratch 2 I know how to teach programming concepts with Scratch

3 I can encourage a positive attitude toward programming to my students

7 I can create lessons plans using programming as an educational tool

5 I'm sure myself to use programming as an educational tool within a classroom

A t-test of the pre and post-survey TSECT scale revealed a statistically significant increase in TSECT from pre (*M* = 12.03, *SD* = 4.39) to post (*M* = 18.14, *SD* = 3.59), *t*(14) = 3.98, *p* < .0001. From the students' answers, we can conclude that after the intervention, they feel themselves confident enough to create projects and they plan to incorporate programming as an instructional tool in their future classrooms.

4 I can become a mentor teacher and support my students to use programming as a tool to explore

6 I can adapt methods, lesson plans, and curriculum materials for using programming as an

Furthermore, after completion of the course, the researchers conducted a focus group interview with a structured interview form. All the students noted that the added cognitive effort was worthwhile and decided to bring coding activities into the early childhood classroom. The students noted that they experienced a significant shift in mindset during the course. Before the course started, all students identified the lack of CT knowledge and skills as a major challenge. After the course, all of them could successfully define key CT concepts. They expressed a high degree of confidence that they taught the CT lessons effectively contributed to their learning. Moreover, all students noted that they made major leaps in correcting their misconceptions about what CT is and understanding fundamental CT concepts. After the focus group interview, the researchers noted that the majority of the students could explain what CT is and describe the main concepts covered during the course. It is also worth to mention that all students indicated that they would like to continue CT training in the following academic year, if that was possible. They also men-

In this chapter, we studied how a course helped preservice teachers to learn and introduce CT concepts into their daily teaching practices as a new subject to their students. The programming and teaching behaviors that emerged still need to be validated through further studies. Furthermore, since the data was collected from female students from one university department, the findings should be applied to subjects from other disciplines with caution. Moreover, it may be useful to employ a mixed method approach that incorporates long-term practical research methods for a deeper investigation of factors affecting attitudes and intentions toward using

Discussions about the appropriateness of technology in early childhood are mostly put aside, and the pressing question is not "Should we introduce

tioned that they would recommend the course to other students.

Scratch in respect to the students' gender.

**5. Discussion and conclusion**

**194**

**4.3 Limitations**

In future work, it would be an idea to plan out a more open-ended set of challenges, which would allow students to use most advanced CT concepts. Also, it would be a good idea to integrate in the course smart robots such as Bee-Bot and Kibo or Internetconnected smart toys such as Sphero. Also, as a new version of Scratch, Scratch 3.0 is on the way and it would be a good idea to integrate in a new course the use of smart mobile devices such as tablets as a part of new students' experience.
