*5.2.2.3 Difficulty level*

The difficulty level index (P) ranges from 0 to 1 and represent the number of students answering the question correctly. The higher the difficulty level value, the higher the number of students answering the question correctly, and vice versa. **Table 3** shows that the "moderate" category is the majority incident regarding the question's difficulty level. On average, the P values for A-tier, R-tier and B-tier are 0.58, 0.53 and 0.42, respectively and fall in the "moderate" category. These values imply that the level of the questions is relevant for secondary school students.

## *5.2.2.4 Discriminatory index*

Discriminatory index (D) compares the number of students answering the questions correctly between high achievement students and low achievement ones. The higher the D indices, the higher the number of students answering the question correctly from high achievement students, and vice versa (**Table 4**).

On average, the D values for A-tier, R-tier and B-tier are 0.53, 0.52 and 0.62, respectively and fall in the "moderate" category. These values imply that the instrument can differentiate students with high achievement and those with low achievement.


*The Multi-Tier Instrument in the Area of Chemistry and Science DOI: http://dx.doi.org/10.5772/intechopen.100098*

#### **Table 3.**

*The difficulty level of questions of the FTI-ENES.*


#### **Table 4.**

*Discriminatory indices of questions of the FTI-ENES.*

### *5.2.2.5 Distractor effectiveness*

The distractor effectiveness parameter represents whether each wrong option in the A and R tiers is functional. An option is considered functional when it is chosen by at least one student [32]. **Table 5** demonstrates that all the options are functional, implying the homogeneity of the options.


**Table 5.**

*Distractor effectiveness for each option each question of the FTI-ENES.*

### **6. Conclusions**

A two-tier instrument that was initially developed by Treagust [15] is the pioneer of a multi-tier instrument. The next generation of multi-tier instruments, including three-tier, four-tier, and five-tier, responds to the drawbacks of the two-tier, which is the inability to distinguish an actual unscientific understanding and the role of guessing. We also believe that an additional drawing tier, as shown by the work of Anam et al. [28], is a rational exercise to be applied in future assessment purposes. By adopting the procedure of two-tier development, we suggest a more straightforward procedure to develop a multi-tier instrument including Mapping concept, Developing the multiple-choice question with free responses (MCQ-FR), Validating the MCQ-FR, Testing and Collecting Students' Unscientific Understanding, Developing the prototype multi-tier instrument, and Validating the Prototype and refining the final multi-tier instrument. A wrong answer-wrong reason (WAWR) combination accompanied by a high confidence rating index (CRI) is the parameter to justify students' unscientific understanding level. In this paper, we suggest employing a five scale CRI instead of 6 because it provides a better clarity of students to express his/her level of confidence. We also suggest that using a CRI of 3 as a limit between *genuine* and *spurious* unscientific understanding will ensure a robust justification regarding students' unscientific understanding and lack of knowledge.

The FTI-ENES instrument developed in this study consists of 13 questions covering the topic of electrolyte and non-electrolyte solutions. The instrument's validity and reliability revealed that it is applicable to be used in identifying students' understanding of electrolyte and non-electrolyte solution. Even though the scope of the concepts covered in this study is relevant for secondary chemistry school, it may also be transferable for fresh university students, particularly to identify their basic chemistry knowledge gained from their learning experiences in their secondary school chemistry. Other detailed examples of the application of this procedure in developing multi-tier instruments can be found in our previous works, including in chemical kinetics [7], acid–base properties of salt solution [33, 34], and thermochemistry [30].
