**4. Proposed solution**

In this section, we describe the focus groups usability technique applied in an OSS project. Firstly, we describe the case study design. Secondly, we specify the characteristics of the selected OSS project. Thirdly, we describe the selected usability technique as prescribed by HCI. We then introduce the adaptations made to the focus groups technique for application in an OSS project. Finally, we report the results of applying this usability technique.

#### **4.1. Case study design**

Case studies are one of the most popular forms of qualitative empirical research [33]. A case study investigates the phenomenon of interest in its real-world context. The phenomenon of interest for this research is the adoption of usability techniques with adaptations, whereas the real-world context is OSS projects. It is not easy to run controlled experiments in the field of OSS because the characteristics of OSS communities (for example, age, availability, expertise, experience, etc.) are unmanageable. Since not all OSS project team members have the same characteristics, it is impossible to minimise the effects of external factors (for example, geographic distribution and time differences). This rules out evaluation by means of an experiment. On this ground, we selected the case study methodology to validate the feasibility of our proposal for adopting a usability technique in an OSS project. We describe the case study following the guidelines set out by Runeson and Host [18]. According to these guidelines, we divide our research into two parts: an exploratory part and a descriptive part. We start by looking at what happens in a real-world scenario and then we describe what happens when we apply the adapted techniques to improve application usability [18].

the focus group in order to achieve the goals set out in this study. Step 2 (Design data collection forms) involves designing a data input form (e.g., to note down the opinions, problems and comments raised by focus group participants). Additionally, a list of specific questions (related, for example, to the user interface and the work environment) has to be compiled and addressed and discussed by the focus group. Step 3 (Conduct the focus group) should take about 1–2 hours. According to Mayhew, a good sized focus group would have between six and eight members. Additionally, she believes that the moderator and note-taker play a very important role with respect to the key information stated by participants [14]. Steps 4 and 5 of the focus groups technique (Analyse data and Draw/document conclusions) address the transcription, analysis and summary of the results to draw and document the focus group conclusions.

Applying a Usability Technique in the Open Source Software Development Process: Experiences…

http://dx.doi.org/10.5772/intechopen.74862

51

The focus groups usability technique cannot be applied directly in the OSS development process because this community has features that do not conform to the HCI world, like, for example: the worldwide geographical distribution of its members, a code-centred world view, a shortage of resources and a culture that can be somewhat alien to interaction developers. Even though usability techniques demand conditions that, as a rule, OSS projects cannot meet, the techniques can be adapted to bring them into line with the idiosyncrasy of the OSS world. In the following, we describe the adaptations of the focus groups usability technique

A usability expert is indispensable for applying Step 1 (Design focus group format) of the technique [14]. This expert is needed to structure the scripted objectives, topics and questions to be analysed when the focus group is conducted. We propose to substitute this expert with the principal developer, an experienced OSS project user or a HCI student under the supervision of a mentor to guide the focus groups format development. With regard to Step 2 (Design a data collection form), we found that the topics to be dealt with in the focus group cannot be physically handed out to participants because they are distributed all over the world. On this ground, we suggest that remote participation in the OSS community should be logged (online forum). Additionally, the outline of the topics should be posted on the same online forum so

that users can recall their experiences with the software system interface under study.

In Step 3 (Conduct focus groups), we found that users are required to meet face to face to participate in technique application. Additionally, we found that a moderator and a note-taker had to be there in person to guide discussions and take notes during focus group application, respectively. This condition cannot be met due to the characteristics of OSS projects. On this ground, we suggest the following adaptations: (i) users will participate remotely in virtual meetings via the online forum; (ii) the moderator will be replaced by the principal developer, an expert OSS project user or a HCI student under the supervision of a mentor, (iii) there will be no note-taker during the conduct of the focus group because the online forum will be logged automatically.

In Step 4 (Analyse data), the information should be organised and then grouped by characteristics (such as age range, gender, occupation, etc.) before analysis. This simplifies the process of data analysis for the purpose of comparing and correctly interpreting the information gathered in the focus group [36]. Finally, Step 5 (Report conclusions) draws the conclusions with respect

**4.4. Adaptations of the focus groups usability technique**

for application in OSS projects.

#### **4.2. ERMaster OSS project characteristics**

We selected ERMaster as the OSS project in which to adopt the focus groups technique. ERMaster is an Eclipse plug-in and is very useful for novice or expert database (DB) designers. There are several OSS project repositories. One of the most popular is Source-Forge.net. This repository classifies OSS projects by categories. Since this technique is related to requirements engineering for product concept development, we looked at projects with a low level of coding (that is, projects where key features were still being added) that were not overly ambitious and were at the very early development stages (alpha version) in order to select a suitable OSS project in which to adopt the selected usability technique. Considering the above, we selected the ERMaster OSS project. Thanks to the characteristics of this project, we can adopt a usability technique related to a requirements activity (product concept development). Therefore, the benefits of applying the technique will have a bigger impact on the development process and software system usability.

#### **4.3. Focus groups usability technique description**

The focus groups technique is a useful tool for evaluating user needs and feelings about a product expressed at group sessions [34]. More formalised definitions in the field of HCI describe the focus groups technique as a qualitative technique whose aim is to gather information about user opinions, problems and concerns at meetings planned for the purpose [13–15]. According to the literature, several authors [13, 15, 35] neither consider the planning required before and after applying the focus groups technique nor propose definite steps for applying the technique either. By contrast, Mayhew proposes a number of specific steps for applying this technique [14]. According to Mayhew [14], the focus groups technique is composed of the five steps described below.

Step 1 (Design the focus groups format) involves designing a script for the purpose of implementing a planned sequence of activities to be performed before, during and after conducting the focus group in order to achieve the goals set out in this study. Step 2 (Design data collection forms) involves designing a data input form (e.g., to note down the opinions, problems and comments raised by focus group participants). Additionally, a list of specific questions (related, for example, to the user interface and the work environment) has to be compiled and addressed and discussed by the focus group. Step 3 (Conduct the focus group) should take about 1–2 hours. According to Mayhew, a good sized focus group would have between six and eight members. Additionally, she believes that the moderator and note-taker play a very important role with respect to the key information stated by participants [14]. Steps 4 and 5 of the focus groups technique (Analyse data and Draw/document conclusions) address the transcription, analysis and summary of the results to draw and document the focus group conclusions.

### **4.4. Adaptations of the focus groups usability technique**

**4.1. Case study design**

50 Trends in E-learning

Case studies are one of the most popular forms of qualitative empirical research [33]. A case study investigates the phenomenon of interest in its real-world context. The phenomenon of interest for this research is the adoption of usability techniques with adaptations, whereas the real-world context is OSS projects. It is not easy to run controlled experiments in the field of OSS because the characteristics of OSS communities (for example, age, availability, expertise, experience, etc.) are unmanageable. Since not all OSS project team members have the same characteristics, it is impossible to minimise the effects of external factors (for example, geographic distribution and time differences). This rules out evaluation by means of an experiment. On this ground, we selected the case study methodology to validate the feasibility of our proposal for adopting a usability technique in an OSS project. We describe the case study following the guidelines set out by Runeson and Host [18]. According to these guidelines, we divide our research into two parts: an exploratory part and a descriptive part. We start by looking at what happens in a real-world scenario and then we describe what happens when

We selected ERMaster as the OSS project in which to adopt the focus groups technique. ERMaster is an Eclipse plug-in and is very useful for novice or expert database (DB) designers. There are several OSS project repositories. One of the most popular is Source-Forge.net. This repository classifies OSS projects by categories. Since this technique is related to requirements engineering for product concept development, we looked at projects with a low level of coding (that is, projects where key features were still being added) that were not overly ambitious and were at the very early development stages (alpha version) in order to select a suitable OSS project in which to adopt the selected usability technique. Considering the above, we selected the ERMaster OSS project. Thanks to the characteristics of this project, we can adopt a usability technique related to a requirements activity (product concept development). Therefore, the benefits of applying the technique will have a bigger impact on the

The focus groups technique is a useful tool for evaluating user needs and feelings about a product expressed at group sessions [34]. More formalised definitions in the field of HCI describe the focus groups technique as a qualitative technique whose aim is to gather information about user opinions, problems and concerns at meetings planned for the purpose [13–15]. According to the literature, several authors [13, 15, 35] neither consider the planning required before and after applying the focus groups technique nor propose definite steps for applying the technique either. By contrast, Mayhew proposes a number of specific steps for applying this technique [14]. According to Mayhew [14], the focus groups technique is composed of the

Step 1 (Design the focus groups format) involves designing a script for the purpose of implementing a planned sequence of activities to be performed before, during and after conducting

we apply the adapted techniques to improve application usability [18].

**4.2. ERMaster OSS project characteristics**

development process and software system usability.

**4.3. Focus groups usability technique description**

five steps described below.

The focus groups usability technique cannot be applied directly in the OSS development process because this community has features that do not conform to the HCI world, like, for example: the worldwide geographical distribution of its members, a code-centred world view, a shortage of resources and a culture that can be somewhat alien to interaction developers. Even though usability techniques demand conditions that, as a rule, OSS projects cannot meet, the techniques can be adapted to bring them into line with the idiosyncrasy of the OSS world. In the following, we describe the adaptations of the focus groups usability technique for application in OSS projects.

A usability expert is indispensable for applying Step 1 (Design focus group format) of the technique [14]. This expert is needed to structure the scripted objectives, topics and questions to be analysed when the focus group is conducted. We propose to substitute this expert with the principal developer, an experienced OSS project user or a HCI student under the supervision of a mentor to guide the focus groups format development. With regard to Step 2 (Design a data collection form), we found that the topics to be dealt with in the focus group cannot be physically handed out to participants because they are distributed all over the world. On this ground, we suggest that remote participation in the OSS community should be logged (online forum). Additionally, the outline of the topics should be posted on the same online forum so that users can recall their experiences with the software system interface under study.

In Step 3 (Conduct focus groups), we found that users are required to meet face to face to participate in technique application. Additionally, we found that a moderator and a note-taker had to be there in person to guide discussions and take notes during focus group application, respectively. This condition cannot be met due to the characteristics of OSS projects. On this ground, we suggest the following adaptations: (i) users will participate remotely in virtual meetings via the online forum; (ii) the moderator will be replaced by the principal developer, an expert OSS project user or a HCI student under the supervision of a mentor, (iii) there will be no note-taker during the conduct of the focus group because the online forum will be logged automatically.

In Step 4 (Analyse data), the information should be organised and then grouped by characteristics (such as age range, gender, occupation, etc.) before analysis. This simplifies the process of data analysis for the purpose of comparing and correctly interpreting the information gathered in the focus group [36]. Finally, Step 5 (Report conclusions) draws the conclusions with respect to the opinions expressed by users. We did not identify any adverse conditions for the last two steps, and therefore no adaptations had to be made. **Table 1** summarises the steps, identified adverse conditions and suggested adaptations for the focus groups technique [14]. There are mainly two adaptations. First, users participate online via a forum. Secondly, the usability expert is replaced by a developer, expert user or a HCI student under the supervision of a mentor. In this particular case, a HCI student under the supervision of a mentor substituted the expert.

According to HCI prescriptions, design tips for a new application feature output by the focus groups technique are appraised by a usability expert [15]. In the adapted focus groups technique, the end users submitted their designs and opinions via web artefacts (like forums and emails) and not at face-to-face meetings due to the characteristics of the OSS communities. **Table 2** presents the steps and tasks of the adapted focus groups technique that is proposed as a result of this study for its application to an OSS project.

#### **4.5. Case study results**

We applied the focus groups technique in the ERMaster project. We had trouble recruiting real users because it took a long time to get permission from the principal developer. We had

> to contact the principal developer by means of several media (email and personal wiki) before he gave us his consent. Six ERMaster users participated in the focus groups technique application. After designing the focus group format taking into account the stated topics and objectives, we proceeded to phrase the questions in order to apply the focus groups technique.

8. Present the results. • Present the focus group results in the forum to inform the OSS community.

OSS project.

applied.

4. Design the data collection form. • Design a form on a spreadsheet to register data. 5. Conduct the focus group. • Conduct the focus group through the online forum

The questions should be aligned with the objectives addressed in the focus groups and are related to the ERMaster application work environment. The focus groups questions are designed to evaluate usability issues such as ease of learning, efficiency of use, memorability, errors and satisfaction [35]. By studying these factors, we focus on user-centred design, an issue neglected in OSS development projects. Previously, we published the call for participation on the ERMaster forum. **Figure 1** shows an example of a response to the questionnaire,

summary of results obtained from the focus groups questionnaire given by the users on the

We expected a higher rate of participation from the ERMaster community. Since it took a long time to get the principal developer's permission and users did not show much interest in participating in our research, we only managed to recruit six participants. The focus group was moderated by the principal developer. However, he did not comment on the opinions of the users posted on the online forum. We believe that the principal developer did not get involved in the open online forum because he was not unduly concerned about improving the usability of the OSS application. The presence of a note-taker was unnecessary in order to conduct the focus group, as, on one hand, the comments posted on the open online forum were logged

automatically and, on the other, the researchers kept all the emails that they received.

forum. **Table 4** presents the questions and the

• Develop step by step the Focus Group script to apply it to the chosen

http://dx.doi.org/10.5772/intechopen.74862

53

• Ask the project administrator for permission to act with the community of real users that are working with the chosen OSS project.

• Decide what format of focus group to use when the technique is

• Produce a report with the conclusions and recommendations that

• Register the results in the previously designed form.

Applying a Usability Technique in the Open Source Software Development Process: Experiences…

resulted from the data analysis of the focus group.

**Table 3** shows the (unstructured) format design.

**Table 2.** Steps and tasks of the adapted focus groups technique.

**Steps Tasks**

1. Design the format of the focus group

3. Define the topics/questions for the

6. Analyse and interpret the data obtained during the focus group.

and recommendations.

7. Produce a report on the conclusions

2. Determine who will be the participating users of the focus group.

Script.

focus group.

given by one used from the official ERMaster<sup>2</sup>

https://sourceforge.net/p/ermaster/discussion/855766/thread/77445c51/

official ERMaster forum.

2


**Table 1.** Steps, adverse conditions and proposed adaptations for the focus groups technique.


**Table 2.** Steps and tasks of the adapted focus groups technique.

to the opinions expressed by users. We did not identify any adverse conditions for the last two steps, and therefore no adaptations had to be made. **Table 1** summarises the steps, identified adverse conditions and suggested adaptations for the focus groups technique [14]. There are mainly two adaptations. First, users participate online via a forum. Secondly, the usability expert is replaced by a developer, expert user or a HCI student under the supervision of a mentor. In this particular case, a HCI student under the supervision of a mentor substituted the expert.

According to HCI prescriptions, design tips for a new application feature output by the focus groups technique are appraised by a usability expert [15]. In the adapted focus groups technique, the end users submitted their designs and opinions via web artefacts (like forums and emails) and not at face-to-face meetings due to the characteristics of the OSS communities. **Table 2** presents the steps and tasks of the adapted focus groups technique that is proposed

We applied the focus groups technique in the ERMaster project. We had trouble recruiting real users because it took a long time to get permission from the principal developer. We had

> • The expert may be a developer, an expert user or a HCI student (supervised by a mentor).

> • The expert may be a developer, an expert user or a HCI student (supervised by a mentor). • The outlines and the list of topics to be discussed are published on the online forum.

• Users participate remotely via the online forum. • The moderator may be an expert OSS project user or a HCI student (supervised by a mentor). • The number of participants in the online forum

• The time available for submitting opinions to the online forum will depend on each

• The log of each participant in the community is recorded in the online forum (i.e., the forum will play the role of a virtual assistant to take

• The focus group conclusions and recommendations are published in forums and distributed by electronic mailing lists to the OSS

is unlimited.

participant.

notes).

community.

as a result of this study for its application to an OSS project.

**Steps Adverse conditions Proposed adaptations**

• Usability expert participation is

• Usability expert participation in the

• The list of topics to be discussed in the focus groups cannot be handed out as printed matter because the users do

• Users are required to participate in person to apply this technique. • A moderator, which could be the principal developer or an experienced user, and a note-taker are required to

• The number of participants is limited

• (There are no adverse conditions. The adaptation is the result of the OSS community work method and is not a response to an adverse condition.)

**Table 1.** Steps, adverse conditions and proposed adaptations for the focus groups technique.

• The time taken to conduct a focus group is limited (1–4 hours). • There is no assistant to take notes.

4. Analyse data. • No adverse conditions were identified. • N/A

required.

project is required.

not attend in person.

attend in person.

(4–9)

**4.5. Case study results**

1. Design the focus group script

52 Trends in E-learning

2. Design the data collection form.

3. Conduct the focus group.

5. Report conclusions.

format.

to contact the principal developer by means of several media (email and personal wiki) before he gave us his consent. Six ERMaster users participated in the focus groups technique application. After designing the focus group format taking into account the stated topics and objectives, we proceeded to phrase the questions in order to apply the focus groups technique. **Table 3** shows the (unstructured) format design.

The questions should be aligned with the objectives addressed in the focus groups and are related to the ERMaster application work environment. The focus groups questions are designed to evaluate usability issues such as ease of learning, efficiency of use, memorability, errors and satisfaction [35]. By studying these factors, we focus on user-centred design, an issue neglected in OSS development projects. Previously, we published the call for participation on the ERMaster forum. **Figure 1** shows an example of a response to the questionnaire, given by one used from the official ERMaster<sup>2</sup> forum. **Table 4** presents the questions and the summary of results obtained from the focus groups questionnaire given by the users on the official ERMaster forum.

We expected a higher rate of participation from the ERMaster community. Since it took a long time to get the principal developer's permission and users did not show much interest in participating in our research, we only managed to recruit six participants. The focus group was moderated by the principal developer. However, he did not comment on the opinions of the users posted on the online forum. We believe that the principal developer did not get involved in the open online forum because he was not unduly concerned about improving the usability of the OSS application. The presence of a note-taker was unnecessary in order to conduct the focus group, as, on one hand, the comments posted on the open online forum were logged automatically and, on the other, the researchers kept all the emails that they received.

<sup>2</sup> https://sourceforge.net/p/ermaster/discussion/855766/thread/77445c51/


**Figure 1.** Focus groups online forum executed on SourceForge.

**Points to be considered Summary of results**

2. Age The ages of participants in the focus group were in the range of 36 to

Applying a Usability Technique in the Open Source Software Development Process: Experiences…

http://dx.doi.org/10.5772/intechopen.74862

55

4. Occupation The majority of users work in Information Technology related areas.

dictionary obtainment).

and Eclipse.

and options.

83% considered menus were complete.

17% of participants consider they have an intermediate level of experience using ERMaster and 83% state they have an advanced level using this tool.

The majority of participants have used this tool between 1 and 3 years.

They all expressed that the graphic environment allows them to design DER with ease and that it reduces the amount of coding work due to it being a good multiuse tool to work with different Database motors.

83% believed they should only need one or two shortcuts to perform certain specific activities (for example, DER design and DER data

The majority of participants believed ERMaster's graphic environment did not need any improvements. However, they do mention the tool should be installed independently, i.e. without needing the prior installation of Java

All participants mention ERMaster's query environment is pleasant to work with; the majority consider the toolbars, options and colours are

All users considerate easy to learn how to use and to use the icons, menus

sober and adequate for the work done with the application.

40 years old.

3. Gender Participants were predominantly male (83%)

1. E-mail address:

5. What experience do you have with ERMaster

6. How long have you been using

7. What do you like about working with the ERMaster environment?

menus are suitable for purpose?

environment pleasant to work

12. Are the ERMaster icons, menus, options, etc., easy to understand?

8. Do you think the ERMaster

9. What utilities, menus, options, etc., would you like to change or

add to in ERMaster?

10. How good do you think the environment is for entityrelationship diagram design?

11. Is the ERMaster query

with?

ERMaster?

**Table 3.** Focus groups technique format.

The principal developer sent an introduction and formal invitation to the ERMaster project community to participate in the open online forum. For many application users, however, their forum registration is the only record available as they did not post any opinions. Some users answered the questions and submitted their responses by email instead of publishing them on the forum. Other users eventually responded to one or two questions related to their major field of interest but failed to complete the entire questionnaire. A few other users stated that they were happy with the tool and did not answer any of the questions. We then screened all the feedback, and selected the contributors who answered all or most of the questions. As a result of this screening, we got a sample of six users for our research.

The noteworthy results of the application of the focus groups technique considering the data gathered include: (i) novice users had problems with installation (because it is an Eclipse Applying a Usability Technique in the Open Source Software Development Process: Experiences… http://dx.doi.org/10.5772/intechopen.74862 55


The principal developer sent an introduction and formal invitation to the ERMaster project community to participate in the open online forum. For many application users, however, their forum registration is the only record available as they did not post any opinions. Some users answered the questions and submitted their responses by email instead of publishing them on the forum. Other users eventually responded to one or two questions related to their major field of interest but failed to complete the entire questionnaire. A few other users stated that they were happy with the tool and did not answer any of the questions. We then screened all the feedback, and selected the contributors who answered all or most of the questions. As

The noteworthy results of the application of the focus groups technique considering the data gathered include: (i) novice users had problems with installation (because it is an Eclipse

a result of this screening, we got a sample of six users for our research.

**Activities Scenarios Actors**

forum

forum

6. Conduct the online forum. SourceForge forum • OSS community

SourceForge web site online

SourceForge web site online

Focus groups application data collection form (Excel)

Report containing the conclusions and recommendations of the focus groups data analysis

Emails • Principal developer, expert user

Focus group format • ERMaster principal developer

Question format design • I2-TIC master student

Emails and SourceForge forum • I2-TIC master student

Results reporting • I2-TIC master student

belonging to the ERMaster community or a HCI student (under the supervi-

sion of a mentor)

• I2-TIC master student

• I2-TIC master student

• ERMaster principal developer • I2-TIC master student

• ERMaster principal developer • I2-TIC master student

• ERMaster principal developer • I2-TIC master student

1. Determine the focus groups

2. Encourage the OSS community to participate in the forum, considering

3. Briefly explain the aim and benefits of applying the technique in the OSS

4. Determine the topics to be addressed (with regard to the user interface and work environment).

the focus group topics.

researchers.

purpose).

conclusions.

data.

5. Design the questions in line with

7. Review the focus group participant responses (forum/email). Email was an easy option for users due to time constraints. The responses to the questions were sent to one of the

8. Compile the data and enter in data collection form (using an Excel spreadsheet designed for the

9. Analyse and interpret the collected

**Table 3.** Focus groups technique format.

10. Submit a report with the

objectives.

54 Trends in E-learning

its importance.

project



comments in the online forum and ended up emailing their opinions to one of the researchers. Since the focus groups participants had a medium level of experience with respect to both the ERMaster tool and the field of computing, they did not pinpoint any major problems which

Applying a Usability Technique in the Open Source Software Development Process: Experiences…

http://dx.doi.org/10.5772/intechopen.74862

57

The adoption of the adapted focus groups technique was acceptable as this technique requires a small number of participants to get reliable results. With regard to our proposal of substituting a developer, expert user or HCI student under the supervision of a mentor for the usability expert, the expert was replaced in this case by a HCI student supervised by a mentor. Note that this student was in his final year of the Master of Information and Communication Technologies Research and Innovation at the Autonomous University of Madrid and was taking two HCI courses. Additionally, the student was supervised by two usability experts. On this ground, there is no risk of the proposed adaptation for the selected technique having a

We can conclude that the results of the adoption of the focus groups usability technique were not what we expected. Firstly, we banked on the participation of a large number of users based on the statistics provided on the application web site. Secondly, it was hard to contact and recruit users to participate in the research. Note that OSS community members are all volunteers, and they participate in their spare time. Despite all these problems, however, the adaptation of the focus groups technique was reliable for adoption in the ERMaster project, as

The main limitation of our research is the number of case studies (only one). This is preliminary research. Therefore, more cases studies are required to validate the proposed adaptations. Note that there are other usability techniques (for example, user profiles, heuristic evaluation) that might benefit from the proposed adaptations (e.g., HCI students supervised by a mentor standing in for experts) to enhance technique adoption in the OSS development process. Briefly, the results of our research are not very generalizable because we conducted only one case study. Therefore, the focus groups technique needs to be applied to other OSS projects. However, the preliminary results provide a basis on which we can build to improve the performance of other case studies.

The lessons that we have learned from applying the focus groups technique in the OSS project

• OSS project administrators (particularly of small projects) must start to attach importance and become aware of the impact of the issues dealt with by HCI in the development of usable software. We believe that this could motivate users to participate in the application usability techniques. This finding is consistent with the proposals of other authors [23, 37].

• As the developer team of a small OSS project is likely to have limited knowledge of usability techniques and interaction design, we suggest recruiting a community user who has some knowledge of or is enthusiastic about these issues to contribute in the early stages of

novice users may have had.

**6. Lessons learned**

the OSS project development.

are as follows:

negative impact on the quality of the software.

it does not take many users to get a reliable result.

**Table 4.** Summary of the results obtained from the questionnaire.

plug-in), (ii) expert users regard ERMaster is being a tool that is easy to learn, easy to use and easy to understand and had no trouble remembering how to use it, (iii) ERMaster is designed ergonomically using menus, action bars and easy access icons, but some users requested the addition of options of interest (for example, export DBs to Excel), and (iv) users consider the ERMaster work environment to be adequate, as there are Help and Query tools.
