**11. Research on model validity**

One method of providing evidence of the efficacy of the model is to examine learner selfreflections. The examples below, from students and teachers, illustrate learner perceptions of various aspects of the model, and particularly the six dimensions of self-regulated learning (motive, methods, time, physical environment, social environment, performance). The examples are taken from weekly, midterm, and final reflections in the student class, and endof-module and end-of-course reflections for teachers. The model elements are indicated in parentheses following each quotation.

Students


#### Instructors

**Purpose Reflection Prompt:**

**Students**

**11. Research on model validity**

parentheses following each quotation.

motivation and needs before I set my goals. (Motive)

Students

Skills)

**Table 2.** Self-reflection Prompts and Theoretical Connections

52 E-Learning - Instructional Design, Organizational Strategy and Management

**Formative Summative** **Reflection Prompt:**

Comment on your goals and to what degree you achieved them.

**Formative Summative** **Model/Theory Elements**

**Teachers**

One method of providing evidence of the efficacy of the model is to examine learner selfreflections. The examples below, from students and teachers, illustrate learner perceptions of various aspects of the model, and particularly the six dimensions of self-regulated learning (motive, methods, time, physical environment, social environment, performance). The examples are taken from weekly, midterm, and final reflections in the student class, and endof-module and end-of-course reflections for teachers. The model elements are indicated in

**•** I learned a lot of things through this wonderful activity about identifying values and setting goals. First of all, I need to understand my motivation before I set goals Sometimes, it was difficult for me to set a meaningful goal for I did not understand my desire and vision. I was like a visitor without a compass and map, and I got lost. Therefore, I need to evaluate my

**•** I reviewed the goals I set for the course to see if I am making progress. I need to see if I have made any changes. I need to make some changes from what I have done before to now in order to get better grades. These are new goals for the second half: first, try to distinguish between important things and unimportant things. Second, try to improve reading speed, and the last one is improve on finding main ideas in readings (Performance, Methods, Target

**•** The course activities are good activities because these activities can help me to become a better learner. Reviewing the activities, I found that I learned a lot. For example, I found out my learning styles, my strengths and weaknesses as an English language learner, and my reading strategies. These three activities help me to study better. Besides, I found out how to schedule my study time and have a study plan and to study in short segments. These two activities help me to have a better time management. (Methods, Time, Performance)

**•** Online distance learning program are quite strenuous compared to traditional classroom learning, notwithstanding this fact I felt it was fun and balanced. The structures set up to help the student know, do and become is just awesome, most significant of all is sharing ideas and learning from each other, furthermore our speaking partners made it more

interesting. (Structure, Dialogue, Social Environment, Collaborative Control)


Compiling these comments and using qualitative analysis methods to determine categories and themes [38] can provide insights into the effectiveness of the course activities and learners' evaluations of their success in achieving course outcomes. The examples above indicate clear evidence that students and teachers benefitted from the structure and dialogue in the course, the opportunity to examine their motivation, goal setting activities, engagement in the social environment, and the instruction and practice activities related to methods and strategies. Outcomes indicate that they acquired new strategies, increased their self-regulation behaviors, and achieved the target skills and knowledge for the courses.

Another way to measure the effectiveness of the model is to have students complete a formal survey with specific questions about course activities and then analyze the data. Questions might be both forced choice (e.g., multiple choice or ranking) and open-ended. Forced choice questions could consist of asking learners to rank the activities in the course grouped by model component (e.g., for dialogue/social environment/collaborative control, these could include the discussion board, video postings, peer review of writing drafts, group writing project), evaluate the instructor on a Likert scale (e.g., value of instructor feedback, response time, knowledge, quality of interactions), or the effectiveness of the course design and content (ease of navigation, technology use, organization of the home page, clarity of instructions). Examples of open-ended questions might include the following: What aspects of the course did you feel were the most valuable? What specific learning (or teaching strategies) were new to you and which did you have the most success with? What were the greatest strengths of your instructor? What suggestions for improvement do you have for your instructor? This approach is a little more structured than the course self-reflections, particularly if quantitative responses are included. These can be collected over several semesters and the results compared to help inform administrators and designers of needed changes in the content and structure of the course.

Additionally, instructors should be given the opportunity to provide feedback about the course materials. As they are teaching, they will notice inconsistencies in the course, unclear instruc‐ tions, portions of the course that students have difficulty with, and poorly worded or incorrect content and test items. They will also have suggestions for content changes and different pedagogical and presentation approaches. While course designers and administrators responsible for the course would not want to implement all of these recommendations or respond immediately except in cases where simple corrections or typos are needed, this feedback should be compiled and analyzed to determine needed revisions to the course. Also, if the course is part of a sequence of courses, feedback should be obtained from instructors teaching the next level course in the series to determine if the course and instructional techniques are preparing learners adequately.

Other ways to evaluate the model would be through course completion and test scores. The latter would be dependent on ensuring that instructors are rating student work and applying rubrics consistently in the case of assignments such as essays or projects. Comparisons could be made between courses in which teachers have been trained and those in which instructors have not been trained in the model or between courses with embedded learning strategy activities and those without.
