**Differences in Perceived Benefit, Use, and Learner Satisfaction between Open Source LMS and Proprietary LMS**

Ahmad Rafi, Khairulanuar Samsudin and Hafizul Fahri Hanafi

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/60670

#### **Abstract**

Currently, many institutions are using expensive proprietary learning management systems (LMSs). Open source LMSs have been introduced to offer affordable solutions; however, these solutions have not been fully embraced. The researchers undertook a study to examine the differences between these two types of LMSs. This study used a survey to collect data pertaining to perceived benefit, LMS use, and learner satisfaction. The survey sample comprised 608 information technology (IT) major undergraduates from two Malaysian universities. Two groups were set up based on the LMSs used, where the first group (*n* = 290) and the second group (*n* = 318) used the proprietary and open source solutions, respectively. Students were asked to fill out a questionnaire to elicit their opinions concerning constructs *perceived benefit*, *use*, and *learner satisfaction*, and data were analyzed using SPSS (ver. 19). Independent sample t-tests were performed, indicating that there were significant differences in the three constructs, which favored the open source solution. Correla‐ tional analysis showed that each construct correlated significantly with each other, suggesting that each contributes to the overall effectiveness of the system. These findings reinforce the imperative of deploying open source learning solutions that are not only affordable but also effective to support students' needs for effective online learning.

**Keywords:** Learner satisfaction, open source LMS, perceived benefit, proprietary LMS, use

© 2015 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### **1. Introduction**

Today's educational realm is witnessing an ongoing tremendous transformation in the teaching and learning process as the result of the continual advancement of technology. For decades, learners were immersed in learning settings that were dictated by physical learning tools (which was naturally cumbersome), confined learning spaces, and the presence of the teacher. Then, this learning landscape changed to a better setting, with the introduction of the then technology – the monochrome television (TV). The teacher, on certain occasions, showed students the scheduled broadcast over the educational channel containing pre-recorded teaching topics relevant to the current learning objectives. In the same period, many public universities that offered distance-learning courses began to conduct such learning classes by broadcasting live lectures to students in their designated classes, which were geographically spread across the country. Even though this type of learning environment was not exactly perfect, many students managed to learn quite effectively with minimum cost. Later, this learning setting morphed into a revolutionized teaching and learning environment in response to the advancement of the personal computing platform together with the introduction of the Internet, especially the World Wide Web. Hence, the birth of electronic learning (e-learning) was inevitable, bringing in tremendous benefit to the educational, social, and economical spheres. From the educational perspective, students' independent learning becomes more intense with more online materials and contents being delivered over the Internet and Intranet. This intensification of independent learning has shifted the role of instructors – from being the teacher to the facilitator, especially in collaborative learning classes.

Given the enormous economic and educational potential of e-learning, many solutions have been introduced since the late 1990s. These solutions assume many different terms or names, such as course management system (CMS), learning content management system (LCMS), virtual learning environment (VLE), virtual learning system (VLS), learning portal, or elearning platform, which reflects the many flavors of their functionalities. Among these, LMS is the dominant term commonly used in the educational sphere that focuses on learners rather than learning contents. The literature is quite replete with many definitions of LMS. For example, an LMS is "[a] comprehensive, integrated software [application] that the develop‐ ment, delivery, assessment, and administration of courses in traditional face-to-face, blended, or online learning environments" [1]. In a similar tone, Ref. [2] defined an LMS as "... as a software application for the administration, documentation, tracking, reporting and delivery of e-learning education courses or training programs."

Many learning management system (LMS) companies have entered the market to provide online learning solutions to many institutes of higher learning (IHLs). Invariably, these proprietary LMSs were, and still are, prohibitively expensive to other branches of educational sphere, such as public schools, colleges, and training institutions. The licenses of the LMSs are notoriously exorbitant, ranging from tens of thousands to hundreds of thousands (depending on the scale of users). In fact, the costing of LMS covers not only the cost of acquisition, but installation, customization, and maintenance costs as well. To highlight the impact of the preceding factors, the finding of a survey by eLearning Guild survey [3] involving 909 of its members serves as a guideline for any prospective organizations that decide to implement these learning solutions. Depending on the scale and needs of an organization, the cost of acquisition, installation, and customization can range from as low as \$10,000 to more than \$1 million. On top of this cost, the maintenance of such a system will incur additional cost, ranging from \$10,000 to more than \$250,000 annually. Clearly, the overall cost of running these learning management systems is quite staggering, especially for small organizations. Despite these cost constraints, many institutions still prefer to use proprietary LMSs because of several factors, such as ease of upgrades, security, downtime, and support, which are relatively better handled by proprietary systems [4].

In view of the high initial cost of implementation, many non-profit organizations, such as the open source software (OSS) community (which consists of dedicated individuals or teams) have begun developing their own version of LMS, with considerable degree of success. According to a white paper by Ref. [5], "[o]pen-source solutions are software for which the source code is provided under a license that permits users to access, change, and improve it." Likewise, Ref. [6] defined open source software solutions as "... computer solutions or applications that are developed, tested, updated, and distributed among the community members." The development of open source LMSs entails the utilization of open source platforms, such as PHP/MySQL, Java, Python, Ruby on Rails, or on open source content management systems (CMS), such as Joomla and Drupal [7]. In addition, open source LMSs, such as Moodle, Sakai, and Wordpress, are built on content management systems, such as Joomla and Drupal. Initially, open source systems were built for education, but now they have been adopted by both educational organizations and some companies as well [7].

From the initial outlay perspective, "several OSS systems can help mitigate the ever-increasing licensing fee of commercial providers" [8]. In fact, in certain functionalities, they may have surpassed certain performances of the proprietary LMSs. For example, better customization, intuitive navigation, "simple chat tool" [9] and "highly interactive" [10] are some of the features of the OSS systems that users found to be appealing. "Ample evidence can be gleaned from the relevant literature that supports the use of affordable OSS systems to help improve student leaning" [9,11,12]. Then again, the superiority of one system over the others may no longer hold true when the latter may have made further improvements, far exceeding the former. Nonetheless, there are bound to be intrinsic differences between OSS and licensed LMSs, which cover a range of features, functionalities, and characteristics. These differences in functionalities or features could make – depending on the background of a range of stakeholders, such as the end user (e.g., students and lecturers) and the system administrator – certain LMS systems more preferable compared to others. For example, the end user would naturally prefer an LMS system that is easy to use, while the system administrator would desire an LMS system that is easy to maintain. Irrespective of the types of LMSs, these learning solutions should be able to perform the following core functions for educational purposes as follows [2]:


**1. Introduction**

364 E-Learning - Instructional Design, Organizational Strategy and Management

Today's educational realm is witnessing an ongoing tremendous transformation in the teaching and learning process as the result of the continual advancement of technology. For decades, learners were immersed in learning settings that were dictated by physical learning tools (which was naturally cumbersome), confined learning spaces, and the presence of the teacher. Then, this learning landscape changed to a better setting, with the introduction of the then technology – the monochrome television (TV). The teacher, on certain occasions, showed students the scheduled broadcast over the educational channel containing pre-recorded teaching topics relevant to the current learning objectives. In the same period, many public universities that offered distance-learning courses began to conduct such learning classes by broadcasting live lectures to students in their designated classes, which were geographically spread across the country. Even though this type of learning environment was not exactly perfect, many students managed to learn quite effectively with minimum cost. Later, this learning setting morphed into a revolutionized teaching and learning environment in response to the advancement of the personal computing platform together with the introduction of the Internet, especially the World Wide Web. Hence, the birth of electronic learning (e-learning) was inevitable, bringing in tremendous benefit to the educational, social, and economical spheres. From the educational perspective, students' independent learning becomes more intense with more online materials and contents being delivered over the Internet and Intranet. This intensification of independent learning has shifted the role of instructors – from being the

Given the enormous economic and educational potential of e-learning, many solutions have been introduced since the late 1990s. These solutions assume many different terms or names, such as course management system (CMS), learning content management system (LCMS), virtual learning environment (VLE), virtual learning system (VLS), learning portal, or elearning platform, which reflects the many flavors of their functionalities. Among these, LMS is the dominant term commonly used in the educational sphere that focuses on learners rather than learning contents. The literature is quite replete with many definitions of LMS. For example, an LMS is "[a] comprehensive, integrated software [application] that the develop‐ ment, delivery, assessment, and administration of courses in traditional face-to-face, blended, or online learning environments" [1]. In a similar tone, Ref. [2] defined an LMS as "... as a software application for the administration, documentation, tracking, reporting and delivery

Many learning management system (LMS) companies have entered the market to provide online learning solutions to many institutes of higher learning (IHLs). Invariably, these proprietary LMSs were, and still are, prohibitively expensive to other branches of educational sphere, such as public schools, colleges, and training institutions. The licenses of the LMSs are notoriously exorbitant, ranging from tens of thousands to hundreds of thousands (depending on the scale of users). In fact, the costing of LMS covers not only the cost of acquisition, but installation, customization, and maintenance costs as well. To highlight the impact of the preceding factors, the finding of a survey by eLearning Guild survey [3] involving 909 of its

teacher to the facilitator, especially in collaborative learning classes.

of e-learning education courses or training programs."


Figure 1 shows a snapshot of the learning materials interface of an LMS system indicating available lecture and presentation notes to registered students of a particular course, serving as the third core function (i.e., Assemble and deliver learning content rapidly) of any LMS systems as mentioned above.


**Figure 1.** A snapshot of a learning materials interface

Implementing LMSs for learning purposes that involves audiences consisting of students, teachers, and administrators would entail the following features [13]:


Figure 2 shows a snapshot of the group forum interface of an LMS system that can setup to facilitate discussion among a group of students involved in a project or an assignment. Through this online forum, students will be able to discuss their ongoing work without the usual constraints faced by face-to-face discussion, namely, time and place. At any time, at any place every member of the group can compose and post comments to collaborate on that work.


**Figure 2.** A snapshot of a group forum interface

**•** Assemble and deliver learning content rapidly

366 E-Learning - Instructional Design, Organizational Strategy and Management

**•** Personalize content and enable knowledge reuse

**•** Support portability and standards

**Figure 1.** A snapshot of a learning materials interface

**•** Setting the course calendar.

**•** Forum module

systems as mentioned above.

**•** Consolidate training initiatives on a scalable web-based platform

Figure 1 shows a snapshot of the learning materials interface of an LMS system indicating available lecture and presentation notes to registered students of a particular course, serving as the third core function (i.e., Assemble and deliver learning content rapidly) of any LMS

Implementing LMSs for learning purposes that involves audiences consisting of students,

Figure 2 shows a snapshot of the group forum interface of an LMS system that can setup to facilitate discussion among a group of students involved in a project or an assignment. Through this online forum, students will be able to discuss their ongoing work without the

teachers, and administrators would entail the following features [13]:

**•** Registration and Enrollment options to teachers and students.

**•** Setting the different User Roles and user account.

**•** Uploading and Retrieving Assignment and Resources

**•** Adding/Deleting Courses by the University/Educational Bodies.

In general, there are two main categories of learning management system, namely Education LMS and Corporate LMS [7]. The former primarily focuses on learners and learning facilities, launching and monitoring of online learning, and keeping record of learning activities. On the other hand, even though the latter shares similar functionalities as the former, corporate LMS is also equipped with e-Commerce capability, regulatory compliance, competency, perform‐ ance, human capital, and talent management [7]. For Education LMS, there are two categories: a) commercial (proprietary) Education LMS, and b) open source Education LMS. In fact, there is another category involving systems (which is small in number) that were developed by the universities themselves. Examples of the university-built systems include Brigham Young University, Maryland University, University of Phoenix, Western Governors University, and the Oberta University in Catalonia, among others. Currently, there are about 214 commercial Education LMSs available. In contrast, for open source Education LMSs, the number is about 60, and this number is expected to grow enormously as they become more technically mature over the years, enabling improved installation and customization. Table 1 shows some examples of both types of Education LMSs commonly used by universities worldwide.

Leading commercial Education LMS include Blackboard Learn (Blackboard Inc.), De‐ sire2Learn (D2L) Brightspace, Edmodo (Edmodo LLC), Instructure Canvas (Instructure Inc.), Pearson LearningStudio, and Schoology (Schoology, Inc.). For open source Education LMS, popular systems include ATutor (University of Toronto), eFront (Epignosis Ltd.), ILIAS 4 e-Learning, Instructure Canvas (Instructure Inc.), Moodle (open source), OpenOLAT, Sakai, and Chamilo. Clearly, commercial Education LMSs still dominate the educational landscape


**Table 1.** Some examples of commercial and open source Education LMSs used by universities

compared to open source Education LMSs. This is not surprising given the former's earlier adoption by many major corporations for the training of their personnel. However, open source Education LMSs are tailing closely behind their commercial counterparts for reasons as explained earlier. With greater effort by the open source movement, this type of learning systems is poised to make greater inroad in online learning environments in the near future.

In general, IHLs can adopt two categories of LMSs – either licensed (proprietary) systems or non-licensed (OSS) systems. WebCT, Blackboard, MyGuru, eCollege, and LearningSpace are some of the examples of the former category; on the other hand, Moodle, ILIAS, ATutor, and Claroline represent some of the latter systems. Undisputedly, deciding on which one of the two will rely on many aspects, such as user acceptance, technical support, maintenance, training, servicing, and cost of ownership, which will have an overall significant impact on the use of the system.

For any technology implementations, the ultimate aim is to ensure that the intended users (e.g., students, executives, trainees, or personnel) will be able to improve their knowledge and skills after using such systems. More importantly, users must be made to realize that the systems are indispensable to the efforts to make them more competent. From the managerial perspec‐ tive, it becomes the imperative of the instructors, teachers, or administrators to select and implement the right system in their organization lest the implementation will run into problems, which could be costly and damaging. To achieve a successful implementation of any LMS system will entail conditions that help users to appreciate the full potential of the solution. In other words, they should perceive the system to be highly beneficial to their training or leaning. Of course, there are interrelated factors that come into play in shaping the perceived usefulness or perceived benefit of such systems.

To explain the factors and their relations, several researchers have formulated a few models such as *Technology Acceptance Model (TAM)* [14], *Unified Theory of Acceptance and Use of Tech‐ nology (UTAUT)*[15], *DeLone and McLean model (DL&ML)*[16], and *Educational Technology Model (ETM)* [17]. Invariably, some of the newer models were formulated based on older models, thus some having the same underlying constructs, such as system quality, service quality, course quality, learner satisfaction, LMS use, and *perceived benefit* [18]. In this paper, the constructs that were examined were learner satisfaction, LMS use, and perceived benefit given that the remaining constructs mainly deal with the technical aspects of the systems. According to Ref. [19], user (learner) satisfaction, which measures learner's attitude toward the system, is " the extent to which users believe the information system available to them meets their information requirements." Thus, if the user perceives the system to be poor, the system is rendered inferior. In contrast, higher learner satisfaction of the system will lead to higher "intention to use," which in turn improves usage [16].

Based on these interrelations, satisfied learners will perceive the system to be beneficial to their learning and will most likely use the system more persistently. "The construct LMS use measures the extent to which learners use the LMS, which in effect serves as a barometer that shows the success (or failure) of such a system implementation" [16]. With frequent use of the system, learners will be more likely to improve their knowledge and skills – the positive impact of which will resonate throughout the organization. Accordingly, measuring the net benefit of the system entails the evaluation of the system along with the purpose of the system. "One of the practical ways to measure the perceived net benefit is through eliciting learners' perception on the benefit of the system" [18]. In unison, all these factors will have a serious impact of the selection and use of such a LMS. Furthermore, the use of such a system will also be influenced by several mediating, notably demographic factors, which need to be considered when implementing online learning for students.

compared to open source Education LMSs. This is not surprising given the former's earlier adoption by many major corporations for the training of their personnel. However, open source Education LMSs are tailing closely behind their commercial counterparts for reasons as explained earlier. With greater effort by the open source movement, this type of learning systems is poised to make greater inroad in online learning environments in the near future. In general, IHLs can adopt two categories of LMSs – either licensed (proprietary) systems or non-licensed (OSS) systems. WebCT, Blackboard, MyGuru, eCollege, and LearningSpace are some of the examples of the former category; on the other hand, Moodle, ILIAS, ATutor, and Claroline represent some of the latter systems. Undisputedly, deciding on which one of the two will rely on many aspects, such as user acceptance, technical support, maintenance, training, servicing, and cost of ownership, which will have an overall significant impact on the

**No. Commercial Education LMS Open source Education LMSs** 1 Blackboard Learn (Blackboard Inc.)**\*** ATutor (University of Toronto)\*

4 Instructure Canvas (Instructure Inc.) **\*** Instructure Canvas (Instructure Inc.)\*

9 AdrennaLearn (Adrenna Inc.) CourseWork (Stanford University)

10 Academic Systems LMS LMS Global BusinessLMS

12 Education Elements HLMS JoomlaLMS (JoomlaLMS)

14 FrogLearn (FrogEducation Ltd.) EctoLearning (Ecto, LLC) 15 Helix LMS (Helix Education) Sensei (Woothemes)

17 JoomlaLMS (Joomla LMS) Metacoon Metastudy

18 Krawler LMS (Krawler Networks) OpenSWAD

20 Top Scholar (Top Scholar) WeBWorK

16 InYourClass (InYourClass.com) Uzity (Foradian Technologies)

19 McGraw-Hill Connect Whiteboard Courseware System

**Table 1.** Some examples of commercial and open source Education LMSs used by universities

11 Destiny One (Destiny Solutions Inc.) Google Classroom

13 eScholar (eScholar LLC) Open LMS

2 Desire2Learn (D2L) Brightspace **\*** eFront (Epignosis Ltd.)\* 3 Edmodo (Edmodo LLC) **\*** ILIAS 4 e-Learning\*

5 Pearson LearningStudio Moodle (open source)\*

6 Schoology (Schoology, Inc.) **\*** OpenOLAT\* 7 CourseWebs (Case Consulting, LLC) Sakai\* 8 Collaborise Classroom (DemocraSoft) Chamilo\*

368 E-Learning - Instructional Design, Organizational Strategy and Management

use of the system.

\* Leading, popular LMS

As with other computer-based solutions, both proprietary and non-proprietary systems are readily available. Given the availability of both types of the systems, undertaking a compara‐ tive study of these two systems will not only be interesting but purposeful, as the lessons to be learned will help universities' administrators to make an informed decision on the final choice of a particular system type to be used in their organizations. Such a choice will have an overwhelming impact on the learning and teaching process in the long term. Thus, three research questions that focus on perceived benefit, LMS use, and learner satisfaction were formulated to guide the study as follows:


Based on the three research questions, three corresponding research hypotheses were also formulated as follows:

