**3. Research methodology and the survey results**

A blend of research methods consisting of focus group studies, pilot tests, and surveys have been used and discussed.

#### **3.1. Focus group studies**

successful e-commerce/IS use can create net benefits concerning financial and operational performance for SMEs, even in developing countries [2]. A significant number of SMEs have, however, failed in adopting e-commerce while many businesses are not satisfied with their e-commerce systems. The research stated, therefore, that quantifying the value contribution of e-commerce has become an issue for managers seeking to justify the enormous expendi-

Among the research methods, the evaluation has been one of the five top research areas on the adoption of e-commerce along with trust, technology acceptance and technology application, e-commerce task-related application, and e-markets, which resulted from the analysis of a total of 1064 e-commerce-related articles and 33,173 references published in leading e-commerce journals between 2006 and 2010 [4]. Researchers have also enunciated the need for evaluating e-commerce success as avoiding failure again, learning from experience, indicating actual business benefits, the requirement for adoption guidelines, and for further improvement and

Investigation to date does not clearly show how to evaluate the success of e-commerce systems [6]. In the investigation of e-commerce, many research studies use the IS success model to evaluate e-commerce systems [7]. For example, the original or updated DeLone and McLean model have been widely used for evaluating the degree of IS/e-commerce success [8]. The Technology Acceptance Model (TAM) and its extensions have also been used as reliable

IS success approach may not, however, be methodologically and theoretically feasible in e-commerce among SMEs [10]. The literature has noted that the difficulties existed in using such IS models and its extensions [5]. The main difficulty has been highlighted that the determinants of e-commerce might be dissimilar to the concepts in IS success studies [5, 10]. Other difficulties could be focused on the involvement of top management, beyond Internet technol-

In the literature, no strong consensus or well-known comprehensive and integrated theoretical framework currently exists [10–12]. Research frameworks also lack a theoretical approach to defining and evaluating e-commerce success among SMEs [10–12]. Exploring more effective methods to describe and evaluate e-commerce success, thereby, becomes a challenging task [13, 14]. This research seeks to help fill the gap by proposing a new model to evaluate

In the literature, satisfaction is a very important element for a successful long-term relationship with e-commerce adoption/success [9]. Research highlighted that satisfaction rather than system use was adopted as an appropriate measure of e-commerce success [15]. Any

and robust models for predicting the user acceptance of e-commerce [9].

tures involved in new IT investment [3].

40 Management of Information Systems

development [5].

ogy and lack of experiences [5].

e-commerce success from a business perspective.

**2.1. Using business satisfaction for evaluating e-commerce success**

**2. CSFs and E-commerce success**

A focus group study involves a formalised process of bringing a small group of people together for an interactive and spontaneous discussion or interview of one particular topic or concept [26]. With origins in sociology, focus group study became widely used in market research during the 1980s and was used for more diverse research applications in the 1990s [27]. Focus group studies might help prepare for a survey by providing sufficient information about the survey objective, by defining and improving indicators and about preventing possible errors [28].

Most experts agree that the optimal number of participants in any type of focus group interview is from six to 12 members in nondirective interactive communications facilitated by a moderator who prepares and uses a loosely constructed set of relevant questions [27–29]. In this research, a target of 18 SMEs business managers (nine for each in Australia and China) were formed to define and improve indicators preventing possible errors.

#### **3.2. Pilot tests**

A pilot test is an important and essential step in checking the rigour of the survey instrument and the need for any final modification before conducting the survey proper [28, 30–32]. The objectives of this step were to examine the validity of each item in the survey and to avoid any misleading cultural differences due to inaccurate translation [33], as it is extremely difficult even for experienced social scientists to write a questionnaire [30].

Research advises that a pilot test of 20–50 cases is usually sufficient to discover the major flaws in a questionnaire before they damage the main study [30]. Research further suggested that researchers use open questions in pilot tests and then develop closed-question responses from the answers given to the open questions for large-scale surveys [26].

Twenty businesses were involved in pilot tests (10 for each in Australia and China) in this research, which were carried out with open questions to modify the proposed questionnaires and any errors.

#### **3.3. Surveys and survey results**

The survey samples were selected first. Since most computer programs use standard error algorithms based on the assumption of simple random samples, the standard errors reported in the literature often underestimate sampling error [34]. The goal of the sampling is to collect data representative of a population within the limits of random error, which the researcher then uses to generalise findings from a drawn sample back to a population [35]. It is critical that the chosen respondents are representative of the study population [31]. The random sampling method was chosen in this research to select samples [26] as follows:


Research recommends a mailing sequence for sending the survey questionnaire followed by a reminder sent about 1 week later [36]. Two follow-up reminder letters should then be sent to those not responding while the first should arrive about 1 week after sending the questionnaire and the second a week later [26]. If the higher response rate is necessary, phone calls can be made to the non-respondents about 2 weeks after two reminders [36]. Follow-up notices with personally requesting non-responders' participation may increase response rates to some extent [36, 37]. This research adopted the following data collection sequence represented based on research advice [32, 36–38]:

• Step 1: sending survey forms with an invitation letter.

research during the 1980s and was used for more diverse research applications in the 1990s [27]. Focus group studies might help prepare for a survey by providing sufficient information about the survey objective, by defining and improving indicators and about preventing pos-

Most experts agree that the optimal number of participants in any type of focus group interview is from six to 12 members in nondirective interactive communications facilitated by a moderator who prepares and uses a loosely constructed set of relevant questions [27–29]. In this research, a target of 18 SMEs business managers (nine for each in Australia and China)

A pilot test is an important and essential step in checking the rigour of the survey instrument and the need for any final modification before conducting the survey proper [28, 30–32]. The objectives of this step were to examine the validity of each item in the survey and to avoid any misleading cultural differences due to inaccurate translation [33], as it is extremely difficult

Research advises that a pilot test of 20–50 cases is usually sufficient to discover the major flaws in a questionnaire before they damage the main study [30]. Research further suggested that researchers use open questions in pilot tests and then develop closed-question responses from

Twenty businesses were involved in pilot tests (10 for each in Australia and China) in this research, which were carried out with open questions to modify the proposed questionnaires

The survey samples were selected first. Since most computer programs use standard error algorithms based on the assumption of simple random samples, the standard errors reported in the literature often underestimate sampling error [34]. The goal of the sampling is to collect data representative of a population within the limits of random error, which the researcher then uses to generalise findings from a drawn sample back to a population [35]. It is critical that the chosen respondents are representative of the study population [31]. The random

Research recommends a mailing sequence for sending the survey questionnaire followed by a reminder sent about 1 week later [36]. Two follow-up reminder letters should then be sent

sampling method was chosen in this research to select samples [26] as follows:

• Stage 2: random sampling of small clusters within each selected big clusters.

• Stage 3: the sampling of elements from within the sampled small clusters.

were formed to define and improve indicators preventing possible errors.

even for experienced social scientists to write a questionnaire [30].

the answers given to the open questions for large-scale surveys [26].

sible errors [28].

42 Management of Information Systems

**3.2. Pilot tests**

and any errors.

**3.3. Surveys and survey results**

• Stage 1: random sampling of big clusters.


In this research, a total of 2401 surveys were successfully sent to SMEs in Australia and China including Australian SMEs (1528) and Chinese SMEs (873). The usable response rate for the surveys was 7.54% (181 out of 2401) including Australian SMEs (69) and Chinese SMEs (112) [39]. Consequently, a total of 15 items from item 1 to item 15 were finally identified (**Table 1**).


**Table 1.** The 15 items identified.
