**4.1 Methods of evaluation**

*Theorizing STEM Education in the 21st Century*

30% Professional life as a CS engineer and lifelong learning

52% Ergonomics and mental health 46% Master programmes 40% Procrastination

34% Minorities and equal treatment 30% Study motivation and study skills

**Table 4.**

25% Quality in the education—what is that? 23% Studying and working abroad 11% Plagiarism and responsibility

8% Learning outcomes, criteria and assessment

2.Increased understanding of the programme (L ← P relation)

*Results of the evaluation question: 'Which three seminar topics do you think were most fruitful?'*

gramme (L ← L, L ← P and L ← T relations)

7.Follow-up of academic results (T ← L relation)

Let us motivate the last three functions.

and make them more suited to the programme.

eighth function of PIC.

3.Connections between teachers and students (T ← L and L ← T relations)

4.Exchange of experiences between students from different years of the pro-

5.Training in written and oral communication and reflection (curriculum)

6.Information about elective courses and studies abroad (L ← P relation)

8.Covering subjects that other courses are not covering (curriculum)

9.Education of the instructors involved (T ← P and T ← L relations)

10.Quality enhancing evaluation of the programme (all L ⇄ T ⇄ P relations)

The motivation of function 1–7 should be clear from the above characterisation.

There may be important but small subjects that are parts of the overall objectives of the programme but are not included in any ordinary course. This was the case for us for ethics, plagiarism, computer science history and the computer in the societal development. Therefore, we extended PIC with an ethics module and a computer history model and added plagiarism as a seminar topic. This is an example of the

Function 9 concerns the education of the instructors who are acting as mentors, which is of two kinds: First, PIC gives knowledge about the programme, its objectives, contents and courses to the students, but the mentors need to read the preparation material before each seminar, so they will get the same knowledge. Second, the mentors will learn, by reading reflections and listening to the discussions at the seminars, how the students experience their studies and how they study and prioritise. The mentors can then use this knowledge to improve their own courses

Regarding function 10, there are several common problems with ordinary course

evaluations that PIC solves. Many course surveys have low participation, but in PIC the surveys are mandatory. This is possible because the fourth intended learning outcome of the course is 'review critically and reflect on both the setup and

**208**

Six different methods for collecting data have been used in the evaluations:

