**2. Research method**

In this study, the researchers used a quantitative research method based on a survey to collect the required data from a group of students. Using this approach helped the researchers to test the preceding research hypotheses by employing relevant participants, research instruments, and procedure. The details of the research method are as follows.

#### **2.1. Participants**

The sample of the survey comprised a group of 608 undergraduates, who were majoring in information technology at two institutions of higher learning in Malaysia. In terms of gender composition, this sample consisted of 401 female undergraduates and 207 male undergradu‐ ates. Their mean age was 21.5 years, and, on average, they had been using the LMSs for more than 2 years. Their participation in this survey was based on voluntary basis.

#### **2.2. Research instruments**

The research instrument used in this study was mostly based on the questionnaire used by Ref. [18] to collect data pertaining to the constructs LMS use, learner satisfaction, and perceived benefit. There were 11 items in the questionnaire, which were split into three categories. The first category comprised four items to measure LMS use, the second category comprised three items to measure learner satisfaction, and the third category consisted of four items to measure perceived benefit. The participants were asked to state their opinions with regard to the three constructs along 5-Likert-type scales, ranging from "1" (*strongly disagree*) to "5" (*strongly agree*). Cronbach's alpha coefficients for construct reliability measurement of LMS use, learner satisfaction, and perceived benefit were 0.89, 0.88, and 0.91, respectively. These coefficients suggest that the internal consistency of the items are good, exceeding the acceptable value of 0.7 [20]. Table 2 summarizes the 11 items, constructs, and internal consistencies as reported in Ref. [18].


**Table 2.** Items, constructs, and internal consistencies

#### **2.3. Procedure**

research questions that focus on perceived benefit, LMS use, and learner satisfaction were

**a.** Is there a significant difference in perceived benefit between the open source LMS and the

**b.** Is there a significant difference in perceived LMS use between the open source LMS and

**c.** Is there a significant difference in perceived learner satisfaction between the open source

Based on the three research questions, three corresponding research hypotheses were also

**a.** Perceived benefit of the open source LMS will differ significantly from the perceived

**b.** LMS use of the open source LMS will differ significantly from the LMS use of the

**c.** Learner satisfaction of the open source LMS will differ significantly from the learner

In this study, the researchers used a quantitative research method based on a survey to collect the required data from a group of students. Using this approach helped the researchers to test the preceding research hypotheses by employing relevant participants, research instruments,

The sample of the survey comprised a group of 608 undergraduates, who were majoring in information technology at two institutions of higher learning in Malaysia. In terms of gender composition, this sample consisted of 401 female undergraduates and 207 male undergradu‐ ates. Their mean age was 21.5 years, and, on average, they had been using the LMSs for more

The research instrument used in this study was mostly based on the questionnaire used by Ref. [18] to collect data pertaining to the constructs LMS use, learner satisfaction, and perceived benefit. There were 11 items in the questionnaire, which were split into three categories. The first category comprised four items to measure LMS use, the second category comprised three items to measure learner satisfaction, and the third category consisted of four items to measure perceived benefit. The participants were asked to state their opinions with regard to the three

than 2 years. Their participation in this survey was based on voluntary basis.

formulated to guide the study as follows:

benefit of the proprietary LMS.

satisfaction of the proprietary LMS.

formulated as follows:

proprietary LMS.

**2. Research method**

**2.1. Participants**

**2.2. Research instruments**

proprietary LMS as reported by the participants?

370 E-Learning - Instructional Design, Organizational Strategy and Management

the proprietary LMS as reported by the participants?

LMS and the proprietary LMS as reported by the participants?

and procedure. The details of the research method are as follows.

Two groups of participants were formed based on their locations of study. The first group comprised 290 undergraduates (204 females, 86 males) of a private university, who used a proprietary (licensed) LMS. The second group comprised 318 undergraduates (197 females, 121 males) of a public university, who used use a non-proprietary (open source) LMS. The participation of the undergraduates was secured through personal contact of the researchers to allow them to send an online survey questionnaire to the students. This questionnaire also contained a brief introduction of the purpose of the survey and an assurance that their answers would remain confidential. Collected questionnaires were analyzed using Statistical Software for Social Science (Ver. 19.) The statistical procedures to address the research questions were a series of independent t-tests and Pearson correlation. The former procedure was used to examine any significant differences in the perceived constructs. The latter procedure was employed to examine the relations among the constructs. "Pearson correlation is commonly used in social science studies to examine the size and direction of the linear relationship between two continuous variables" [21].

### **3. Findings**

Participants' responses to the questionnaire items were processed to produce the required descriptive statistics, namely, the mean scores, maximum scores, minimum scores, and standard deviations. The overall mean scores (standard deviations) of LMS use, learner satisfaction, and perceived benefit were 3.95 (.68), 3.97 (.69), and 3.78 (.65), respectively, as shown in Table 3.


**Table 3.** Mean scores and standard deviations of the three constructs

An independent-samples t-test indicated that LMS use's mean scores were significantly higher for the group that used open source system (*M* = 4.02, *SD* = 0.62) than for the group that used the proprietary systems (*M* = 3.86, *SD* = 0.73), *t*(606) = 2.91, p < 0.05. The same test also indicated that learner satisfaction's mean scores was significantly higher for the group that used open source system (*M* = 4.04, *SD* = 0.64) than for the group that used the proprietary systems (*M* = 3.87, *SD* = 0.72), *t*(606) = 2.77, *p* < 0.05. Similarly, the perceived benefit's mean score was significantly higher for the group that used open source system (*M* = 3.85, *SD* = 0.52) than for the group that used the proprietary systems (*M* = 3.72, *SD* = 0.76), *t*(606) = 2.32, *p* < 0.05.

Bivariate correlations between pairs of constructs were also computed using the Pearson correlation procedure. Perceived benefit and LMS use were significantly correlated, *r*(606) = 0.11, *p* < 0.05. Likewise, perceived benefit and learner satisfaction were also significantly correlated, *r*(606) = 0.12, p < 0.05. For constructs LMS use and learner satisfaction, their correlation was found to be strong and highly significant, *r*(606) = 0.95, *p* < 0.001. Table 4 summarizes the correlations among perceived benefit, LMS use, and learner satisfaction.

Differences in Perceived Benefit, Use, and Learner Satisfaction between Open Source LMS and Proprietary LMS http://dx.doi.org/10.5772/60670 373


**Table 4.** Pearson correlations among perceived benefit, LMS use, and learner satisfaction

## **4. Discussion and conclusion**

used in social science studies to examine the size and direction of the linear relationship

Participants' responses to the questionnaire items were processed to produce the required descriptive statistics, namely, the mean scores, maximum scores, minimum scores, and standard deviations. The overall mean scores (standard deviations) of LMS use, learner satisfaction, and perceived benefit were 3.95 (.68), 3.97 (.69), and 3.78 (.65), respectively, as

**LMS use Learner Satisfaction Perceived Benefit**

Mean *SD* Mean *SD* Mean *SD*

4.02 0.62 4.04 0.64 3.85 0.52

3.86 0.73 3.87 0.72 3.72 0.76

3.95 0.68 3.97 0.69 3.78 0.65

An independent-samples t-test indicated that LMS use's mean scores were significantly higher for the group that used open source system (*M* = 4.02, *SD* = 0.62) than for the group that used the proprietary systems (*M* = 3.86, *SD* = 0.73), *t*(606) = 2.91, p < 0.05. The same test also indicated that learner satisfaction's mean scores was significantly higher for the group that used open source system (*M* = 4.04, *SD* = 0.64) than for the group that used the proprietary systems (*M* = 3.87, *SD* = 0.72), *t*(606) = 2.77, *p* < 0.05. Similarly, the perceived benefit's mean score was significantly higher for the group that used open source system (*M* = 3.85, *SD* = 0.52) than for the group that used the proprietary systems (*M* = 3.72, *SD* = 0.76), *t*(606) = 2.32, *p* < 0.05.

Bivariate correlations between pairs of constructs were also computed using the Pearson correlation procedure. Perceived benefit and LMS use were significantly correlated, *r*(606) = 0.11, *p* < 0.05. Likewise, perceived benefit and learner satisfaction were also significantly correlated, *r*(606) = 0.12, p < 0.05. For constructs LMS use and learner satisfaction, their correlation was found to be strong and highly significant, *r*(606) = 0.95, *p* < 0.001. Table 4 summarizes the correlations among perceived benefit, LMS use, and learner satisfaction.

between two continuous variables" [21].

372 E-Learning - Instructional Design, Organizational Strategy and Management

**Construct (Measure)**

**Table 3.** Mean scores and standard deviations of the three constructs

**3. Findings**

shown in Table 3.

**Type of LMS**

Open source (*n* = 318)

Proprietary (*n* = 290)

Overall (*N* = 608) One of the major findings of the independent-samples t-test indicates that the participants who used the open source LMS rated the perceived benefit of their LMS significantly higher than their counterpart who used the proprietary LMS. In other words, the difference in perceived benefit between the two types of LMSs as reported by the participants was significant. Thus, this finding supports the first research hypothesis of the study. Similarly, the LMS use of the open source LMS was observed to differ significantly from the LMS use of the proprietary LMS, which lends support for the second research hypothesis of the study. Likewise, another finding of the independent-samples t-test indicates that the participants who used the open source LMS tended to rate Learner satisfaction significantly higher than those who used the proprietary LMS. There was a significant difference between the two groups in this measure, thus supporting the third research hypothesis of the study.

Given the support of all research hypotheses, there is growing evidence to suggest that learning management solutions developed by the open source community might have matured to a level that is on par with licensed solutions – or, as demonstrated in this case, the former might have surpassed the latter in terms of perceived benefit, LMS use, and learner satisfaction. Clearly, when these measures are perceived to be important by users, it can be inferred that the implementation of such a leaning management system is successful to a certain extent. Undisputedly, "there are numerous technical and socio-psychological factors" [22,23,24] "involving all the stakeholders that can determine the success (or failure) of LMSs" [25]. In this study, the perceived benefit (perceived usefulness), learner satisfaction, and LMS use of the open source LMS were highly rated. In addition, based on the correlational analysis, all the three factors were also significantly correlated with each other. More notably, the significant positive association between learner satisfaction and perceived benefit is consistent with earlier findings. This finding reinforces the contention that users will accept an LMS to be beneficial when they "are satisfied with the performance of such a system" [26]. This significant positive relation will in turn make users utilize the system more often and readily to support their learning process. Continued use of such systems will not only benefit students and instructors but administrators as well. In view of this revelation, it is important for both teaching staff and academic administrative personnel to institute several initiatives to high‐ light the benefits of LMS to their students. Through such initiatives, students will be able to recognize and appreciate the immense potential of an LMS. With the right frame of mind, students, especially the freshmen, will be primed to adapt to new, novel learning environment.

Interestingly, in this study, perceived benefit, LMS use, and learner satisfaction of an open source LMS were rated higher than the proprietary LMS. This finding suggests that solutions developed by the open source organizations or individuals have a promising future in providing online learning opportunities to a wide spectrum of users. Though the proprietary LMS entered the educational landscape much earlier than the open source LMS and dominated the online learning environment, the ongoing and relentless efforts by the open source community have made the latter system a formidable solution on par with the former system. Given its relatively inexpensive outlay, many institutions, especially cash-strapped schools, can now afford to implement open source, non-proprietary learning solutions. As demon‐ strated in this study, the open source LMS was rated high by students who used it to support their learning, especially outside the classroom. The high ratings of the perceived benefit, LMS use, and learner satisfaction strongly suggest that "features and functionalities of open source LMSs to support online independent learning have improved over the years to provide the essential teaching and learning needs" [27].

Despite the many advantages of open source LMSs, some factors may hinder successful implementation of such systems. Even though the acquisition of open source LMSs are almost without cost, a highly trained personnel (e.g., a system analyst) is required to manage the systems, which encompasses a wide range of technicalities. Without proper system manage‐ ment, the solution put in place will ultimately become inefficient and ineffective. Like any other database systems, LMSs require constant monitoring, updating, and maintenance over time as the number of users is growing. In addition, there is concern that some of the open source LMSs do not provide the level of security that is needed by certain educational institutions. Hence, important information could be comprised, which is detrimental to the organizations' reputation. Of course, there are other factors as well that can make an open source implementation challenging, but these two factors represent the major concern that can make such adoption either a success or a failure. Given these issues, adopting an open source solution entails appropriate planning that holistically focuses on all aspects pertaining to technical, training, and cost considerations.

In this study, even though the open source Education LMS was highly rated higher than the proprietary Education LMS in terms of the three constructs, the researchers strongly believe that the success of any LMS system – irrespective of being either proprietary or open source – will rely on technical, managerial, institutional, and attitudinal aspects. For example, at the institutional level, universities should not view LMS as a mere technological tool, but more importantly, this system should be implemented with careful, comprehensive planning. To plan such an implementation would entail a rigorous review of existing infrastructure, current teaching and learning practices, and policies. Failure to factor in these aspects in the planning process could make the system underutilized. The researchers opine that for any universities to embark on an LMS project, a dedicated team drawn from various and relevant background should be set up to plan all the necessary details to help guide the selection, installation, testing, and full deployment of a proper LMS system. Ideally, these team members should work in a unit, which may be called (as an example) an e-learning unit under the ambit of the academic affairs department.

recognize and appreciate the immense potential of an LMS. With the right frame of mind, students, especially the freshmen, will be primed to adapt to new, novel learning environment.

Interestingly, in this study, perceived benefit, LMS use, and learner satisfaction of an open source LMS were rated higher than the proprietary LMS. This finding suggests that solutions developed by the open source organizations or individuals have a promising future in providing online learning opportunities to a wide spectrum of users. Though the proprietary LMS entered the educational landscape much earlier than the open source LMS and dominated the online learning environment, the ongoing and relentless efforts by the open source community have made the latter system a formidable solution on par with the former system. Given its relatively inexpensive outlay, many institutions, especially cash-strapped schools, can now afford to implement open source, non-proprietary learning solutions. As demon‐ strated in this study, the open source LMS was rated high by students who used it to support their learning, especially outside the classroom. The high ratings of the perceived benefit, LMS use, and learner satisfaction strongly suggest that "features and functionalities of open source LMSs to support online independent learning have improved over the years to provide the

Despite the many advantages of open source LMSs, some factors may hinder successful implementation of such systems. Even though the acquisition of open source LMSs are almost without cost, a highly trained personnel (e.g., a system analyst) is required to manage the systems, which encompasses a wide range of technicalities. Without proper system manage‐ ment, the solution put in place will ultimately become inefficient and ineffective. Like any other database systems, LMSs require constant monitoring, updating, and maintenance over time as the number of users is growing. In addition, there is concern that some of the open source LMSs do not provide the level of security that is needed by certain educational institutions. Hence, important information could be comprised, which is detrimental to the organizations' reputation. Of course, there are other factors as well that can make an open source implementation challenging, but these two factors represent the major concern that can make such adoption either a success or a failure. Given these issues, adopting an open source solution entails appropriate planning that holistically focuses on all aspects pertaining to

In this study, even though the open source Education LMS was highly rated higher than the proprietary Education LMS in terms of the three constructs, the researchers strongly believe that the success of any LMS system – irrespective of being either proprietary or open source – will rely on technical, managerial, institutional, and attitudinal aspects. For example, at the institutional level, universities should not view LMS as a mere technological tool, but more importantly, this system should be implemented with careful, comprehensive planning. To plan such an implementation would entail a rigorous review of existing infrastructure, current teaching and learning practices, and policies. Failure to factor in these aspects in the planning process could make the system underutilized. The researchers opine that for any universities to embark on an LMS project, a dedicated team drawn from various and relevant background should be set up to plan all the necessary details to help guide the selection, installation, testing, and full deployment of a proper LMS system. Ideally, these team members should work in a

essential teaching and learning needs" [27].

374 E-Learning - Instructional Design, Organizational Strategy and Management

technical, training, and cost considerations.

This team should be given sufficient empowerment to study, formulate, and deploy strategies for effective LMS deployment. This team should examine existing infrastructure to help determine the capabilities of available hardware to support the proposed learning management system. Having the knowledge of the performance of this hardware would help IT personnel to carry out appropriate upgrading or retrofitting work to run the proposed learning management system. It is worthy to note that implementing learning management system is not only about the software per se, but the hardware to run the system is equally important. In other words, the importance of the symbiotic relation between software and hardware to operate such a learning management should not be downplayed. Thus, a thorough review of existing equipment and IT tools is not only important but also mandatory. Likewise, a review of existing teaching and learning practice of a university should be performed with utmost importance. After all, the main aim of deploying LMS is to improve the existing process of teaching and learning, thus this review would help identify weak spots or areas that require redress. Problems associated with teaching and learning in many universities worldwide have increased without respite given the ever-increasing number of students enrolling in diverse academic programs. Face-toface lectures have become a serious problem to many educational institutions as infrastruc‐ ture is stretched to its limits, putting great constraint on it capability. Naturally, lectures have to be complemented by other means, notably online learning to cater vast student populations. Nowadays, many universities have adopted blended learning as a solution to handle the teaching needs and learning needs of lecturers and students, respectively.

In addition, the introduction of learning management systems in institutes of higher learning would entail a sound, appropriate policy to make their implementations a success. Ultimately, these systems would be deemed worthwhile and beneficial if all the stakeholders (students, lecturers, and administrators) could fully utilize these solutions. For example, a part of the policy may contain provisions to necessitate (or to enforce) all lecturers to use the LMS in the following teaching activities: a) uploading lecture notes on LMS, b) making online announce‐ ments, c) setting up online discussion groups, d) conducting online quizzes and tests, e) providing online feedback of students performance, and f) posting online information and news. With all these teaching activities performed on the learning management system, students would be more prone to partake in online learning activities to complement their face-to-face learning. Hence, the use of the system would be more intense, leading students to perceive the system to be highly beneficial. Sustaining this level of teaching and learning activities would in the end make the adoption of the LMS a success.

Overall, the findings of this study provide some assurances that open source education learning management systems are on par with their proprietary counterparts for the constructs learner satisfaction, LMS use, and perceived benefit. In fact, the latter system has been demonstrated to be rated higher than the former system for the three constructs. However, this finding is informative in terms of the continually increasing capability of open source education learning management systems, but not conclusive to stake claim that these open source systems are better than proprietary systems. Many factors are involved in making the adoption of learning management systems a success; thus, the interpretation of this finding should be embraced with caution as this study was based on students' opinions. Other stakeholders' opinions and feedback are needed to ascertain the performance of any learning management systems with some degree of certainty.

In summary, open source education learning management systems are beginning to be seriously viewed as an effective, efficient learning solution from the student perspective. Now, more learning opportunities will be made available to a greater pool of learners across the nation to help them pursue their academic programs in an environment that suits today's educational landscape – more precisely, digital landscape in which more and more contents and materials are in electronic form. Therefore, it is incumbent on the management of univer‐ sities, training centers, and institutions that are currently using expensive learning solutions to seek affordable solutions, which are equally effective, to further enhance their students' online learning experiences. More importantly, schools, especially the public ones, which do not have such systems in place, should try to learn from others that have successfully imple‐ mented the open source education learning management systems so that their pupils can experience online learning at the early age.
