**6. Findings**

Six students were enrolled in the distance section of LIBR 1100 in the fall of 2010. Five of these students actively participated in all the course assignments. One student hardly participated at all and received several 0 point scores. All of the study's findings are based on the input of the five students who actively participated. They were required to take 12 quizzes and could earn a maximum of 8 points on eleven of the quizzes and five points on one of them. Table 4 includes the students' scores on all 12 quizzes. The great majority of the questions on the quizzes were graded automatically by Blackboard. However, the instructor who taught the online course in the fall of 2010 reviewed the handful of fill-in questions and, using her own judgment, determined whether the fill-in answers were correct or not. She did not use a grading rubric while reviewing the questions.

The students also took 6 practicums, and their scores on these are reported in Table 5. Practicum one required the students to determine a thesis for their annotated bibliography assignment. Practicum two required that they find books on the topic of the thesis using keywords derived from the thesis statement. They searched for the books in online catalogs.

Assessment Methods of Student Learning

improved on in the post-assessment test.

What are the 3 Boolean operators?

Question 1

a – Add b – If c – Not d – Then e – And f – Or g – Sum

Correct

Question 14

Correct

PR = Pre-Assessment PO = Post-Assessment

PR = Pre-Assessment PO = Post-Assessment

pre- to post-assessment. This improvement was encouraging.

(c, e, f) 2 40 Correct

What information is needed for a book citation?

a – Credentials, revisions, date of publication b – Author, title, date, publisher information

c – Author, title, publisher, volume and issue number, date

(b) 1 20 Correct

in Web-Based Distance Courses: A Case Study 201

An average score of the group of five students who took both the pre- and post-assessment tests was determined by adding the percentages of the students who answered each question correctly and then determining the average of the total. The average score on the pre-assessment test was 64 percent, and the average score on the post-assessment test was 82.7 percent. Thus, as a group, the students increased their average score by 18.7 points from

Table 6 includes questions one and fourteen on the pre- and post-assessment tests. These two questions illustrate the kinds of questions found on the tests and the choice of answers the students could choose from. Also they both illustrate the handful of questions the students, as a group, did poorly on in the pre-assessment test and then significantly

PR-1 Frequency Percent PO-1 Frequency Percent

Incorrect 3 60 Incorrect 0 0

PR-14 Frequency Percent PO-14 Frequency Percent

Incorrect 4 80 Incorrect 1 20

Table 6. Test questions 1 and 14 and their pre- and post-assessment results.

(c, e, f) 5 100

(b) 4 80


Table 4. Student Scores on Quizzes (fall 2010).

Practicum three required that they find government documents on their topic using appropriate databases. Practicum four required that they find magazine and newspaper articles, and practicum five that they find articles in scholarly journals. Finally, practicum six required that they find appropriate Websites on their topic using a search engine such as Google to find them. All of their sources were found online using the Internet and, after the students created accurate citations for them using either the American Psychological Association or the Modern Language Association style manual, became a substantial portion of their annotated bibliography assignment. All the practicums were graded by the instructor who did not use a grading rubric. The maximum number of points possible for each of these practicums was 20.


Table 5. Student Scores on Practicums (fall 2010).

Five students completed the annotated bibliography assignment. Three A's (the scores were 95, 95, and 90 out of 100 possible points) and two B's (88 and 85) were awarded, with a score of 90 or higher corresponding to an A letter grade and a score of 80 to 89 corresponding to a B letter grade. The instructor determined these scores and letter grades using her own judgment without the use of a grading rubric or an automatic system.

Number 1 2 3 4 5 6 7 8 9 10 11 12

1 8 8 8 8 8 7 8 8 8 8 8 5 2 8 6 8 8 8 8 8 8 8 8 8 5 3 8 8 8 8 8 6 7 8 8 8 8 5 4 8 6 6 7 8 6 7 7 7 7 7 5 5 8 6 6 7 6 6 6 7 7 8 7 4

Practicum three required that they find government documents on their topic using appropriate databases. Practicum four required that they find magazine and newspaper articles, and practicum five that they find articles in scholarly journals. Finally, practicum six required that they find appropriate Websites on their topic using a search engine such as Google to find them. All of their sources were found online using the Internet and, after the students created accurate citations for them using either the American Psychological Association or the Modern Language Association style manual, became a substantial portion of their annotated bibliography assignment. All the practicums were graded by the instructor who did not use a grading rubric. The maximum number of points possible for

Number 1 2 3 4 5 6

Five students completed the annotated bibliography assignment. Three A's (the scores were 95, 95, and 90 out of 100 possible points) and two B's (88 and 85) were awarded, with a score of 90 or higher corresponding to an A letter grade and a score of 80 to 89 corresponding to a B letter grade. The instructor determined these scores and letter grades using her own

1 20 20 20 20 20 20 2 20 20 20 19 19 20 3 20 20 19 18 19 20 4 20 20 15 18 10 19 5 20 19 17 6 19 18

20 20 20 20 20 20

8 8 8 8 8 8 8 8 8 8 8 5

Quiz

Student Number

Highest Possible Score

Table 4. Student Scores on Quizzes (fall 2010).

each of these practicums was 20.

Practicum

Student Number

Highest Possible Score

Table 5. Student Scores on Practicums (fall 2010).

judgment without the use of a grading rubric or an automatic system.

An average score of the group of five students who took both the pre- and post-assessment tests was determined by adding the percentages of the students who answered each question correctly and then determining the average of the total. The average score on the pre-assessment test was 64 percent, and the average score on the post-assessment test was 82.7 percent. Thus, as a group, the students increased their average score by 18.7 points from pre- to post-assessment. This improvement was encouraging.

Table 6 includes questions one and fourteen on the pre- and post-assessment tests. These two questions illustrate the kinds of questions found on the tests and the choice of answers the students could choose from. Also they both illustrate the handful of questions the students, as a group, did poorly on in the pre-assessment test and then significantly improved on in the post-assessment test.


Table 6. Test questions 1 and 14 and their pre- and post-assessment results.

Assessment Methods of Student Learning

test.

assessment test.

**7. Conclusions** 

by the assessments to improve the course.

in Web-Based Distance Courses: A Case Study 203

documents. Perhaps question eleven is also too easy. Question twelve asked the students to examine a citation to a journal article and identify its citation style. Modern Language Association style was the correct answer. Sixty percent of the students answered the question correctly in the pre-assessment test, and 60 percent did so in the post-assessment

Most of the students did well on question thirteen and could identify the features of an annotated bibliography. Eighty percent of the students answered this question correctly in the pre-assessment test, and 80 percent did so in the post-assessment test. Question fourteen asked "What information is needed for a book citation." Twenty percent answered question fourteen correctly in the pre-assessment test, and 80 percent did so in the post-assessment test. Apparently the students had learned something about book citations in the course. Question fifteen asked which statements were correct in a list that supposedly included two citations, an ISBN number, a URL address, and a call number. Eighty percent of the students answered the question correctly in the pre-assessment test, and 100 percent in the post-

This case study examined student learning-outcomes assessment methods that are suitable for Web-based courses. It concentrated on those assessment methods used by instructors at Texas Tech University Library in a one-hour, Web-based credit course developed to teach library research skills to distance students. The instructors selected these methods because they thought that the methods offered the greatest potential for assisting them in reaching the course's assessment goals, included implementation processes that promised to be relatively easy to accomplish, and had attributes that the students would find user-friendly and that would discourage plagiarism. The study also reported the planning process the course instructors follow each year in identifying assessment methods appropriate for their course, in developing and implementing the methods, and how they use the data collected

The instructors who teach the library research course participate as team members in the process of continuously improving the course's content and the teaching and learning that occurs in the course. This team effort has evolved over the years into a structured yearly cycle of planning, developing, marketing, implementing, assessing, and improving the course. Among other things, reviewing learning goals, identifying learning outcomes, creating learning activities, enacting learning activities, gathering data to check learning, interpreting data, and enacting decisions occur in this planning cycle. The data collected from the various assessment instruments play an important role in the process because the findings that come from the data help to identify where improvement is needed. Blackboard Learn, the assessment module in Blackboard, will soon be used to track the progress of students enrolled in the online course and will facilitate accurate reporting of the Library's

Data collected by the assessment instruments employed in the fall of 2010 disclosed findings that were consulted during the summer of 2011 when plans for improving the learning and teaching experiences that would take place in the fall of 2011 were made. All five students did well on all the quizzes. Scores ranged from six to eight points with a maximum possible

impact on the learning and teaching that takes place at Texas Tech University.

Considerable improvement was made on the first question in the post-assessment survey. The students were asked to identify the three Boolean operators. Forty percent of the group answered the question correctly in the pre-assessment test, and 100 percent answered it correctly in the post-assessment test (See Table 3 for a summary of pre- and post-assessment test scores). Eighty percent of the students answered the second question correctly on both the pre-assessment test and the post-assessment test. Question two asked the students to identify the least likely resource for finding citations to articles. The answer was the Texas Tech University Libraries' online catalog.

Question three asked the students what to look for in determining the authority of an Internet site. Eighty percent answered the question correctly in both the pre-assessment test and the post-assessment test. Question four was challenging for the students. When asked to identify the correct statements in a list that included supposed examples of a book's call number, an ISBN number, a citation to a book, a citation to an article, and a URL address, only 40 percent (two students) answered the question correctly by identifying the correct examples on the list in the pre-assessment test and 60 percent (three students) answered it correctly on the post-assessment test.

Question five asked the students to identify the "word search" that would give them books most directly related to gang violence. One-hundred percent of the students correctly identified "gangs AND violence" as the correct answer in the pre-assessment test and 100 percent also selected the correct answer in the post-assessment test. The results of this and the first question in the test suggest that, by the end of the course, all the students in the group understood what Boolean operators were and how they worked. However, it would also seem that question five is probably too easy and should be replaced by a more difficult question.

Eighty percent of the students could identify primary research sources in question six in the pre-assessment test, and 100 percent in the post-assessment test. Considerable improvement took place on question seven. The students were asked to identify "typical scholarly research sources" from a list. Twenty percent of the students selected the correct answer in the pre-assessment test, and 80 percent selected the correct answer in the post-assessment test.

In question eight, 60 percent (three students) in the pre-assessment test and 100 percent in the post-assessment test correctly identified the kinds of information that can be found in the Texas Tech University Libraries' online catalog. All of the students answered question nine correctly in the pre-assessment test, thus indicating that they were aware that full-text magazine articles cannot be found in the catalog. Eighty percent answered this question correctly in the post-assessment test. This indicates regression in learning for this particular bit of knowledge. Question ten asked the students which of two databases—ABI/Inform or Lexis-Nexis Academic Universe—contained full-text newspaper articles. Twenty percent identified the correct answer (Lexis-Nexis Academic Universe) in the pre-assessment test, and 40 percent did so in the post-assessment test. This would indicate that the majority of the students are not aware of the content of this particular database and perhaps the content of databases in general.

All of the students did well on question eleven in both the pre- and post-assessment tests. This question required knowledge of the difference between PDF and HTML full-text documents. Perhaps question eleven is also too easy. Question twelve asked the students to examine a citation to a journal article and identify its citation style. Modern Language Association style was the correct answer. Sixty percent of the students answered the question correctly in the pre-assessment test, and 60 percent did so in the post-assessment test.

Most of the students did well on question thirteen and could identify the features of an annotated bibliography. Eighty percent of the students answered this question correctly in the pre-assessment test, and 80 percent did so in the post-assessment test. Question fourteen asked "What information is needed for a book citation." Twenty percent answered question fourteen correctly in the pre-assessment test, and 80 percent did so in the post-assessment test. Apparently the students had learned something about book citations in the course. Question fifteen asked which statements were correct in a list that supposedly included two citations, an ISBN number, a URL address, and a call number. Eighty percent of the students answered the question correctly in the pre-assessment test, and 100 percent in the postassessment test.
