**3. University ranking systems**

These days, the Internet has become the main source of scientific information, both for the academic community and for the society. The whole society has been turning to the Internet as a primary medium for presentation of information to the public. On that ground, the fact that web publications are a primary tool for communication within the educational system and that they reflect the complete picture of quality and performances of universities has become very important [51]. Bearing in mind the development of digital world, the influence of electronic publications is significantly greater than the influence of written media or printed versions of journals and books today. Websites are the cheapest and the most efficient way to stimulate all of the three academic missions: to educate, to research, and to transfer knowledge [51]. This fact is one of the main reasons why web data have been extensively used for evaluation, inter alia, of universities and research institutions in the last couple of years.

Ranking is a process in which one defines positions of elements in a group in regard to a total system so that for any two elements in a sequence, the first one is ranked "as higher than," "as lower than," or "as equal to" the second item of a sequence [46].

Ranking process appears in many fields whether they are academic or of other type. In a case of academic space, ranking may be applied in different parts of academic space, starting from ranking of professors and ranking of researchers and research centers to ranking of universities. Ranking of universities is an especially interesting field of application of ranking.

Currently, one implements a process of comparing and evaluating universities in the domain of academic and research performances with the existing system of ranking of universities. Most of the academic institutions rely on data obtained from the ranking system of universities which serve them as indicators of a progress of an institution over time in regard to other academic institutions [47]. Besides this, information from these ranking lists often serve as a basis for applying for and obtaining financial assets from founders or other institutions on the basis of a position on these lists [48]. On the other hand, potential beneficiaries of services of a university use these lists to evaluate academic institutions to decide which one to attend and to evaluate which one provides better options for education and further employment.

The study [47] identified 24 ranking systems. Thirteen ranking systems were analyzed into details since their lists were active during the last couple of years, i.e., from 2015 to 2016. Other ranking systems were excluded from further analysis because they did not publish information and did not include indicators of their performances or published their ranking methodologies. The study evaluated between 500 and 5000 institutions. The oldest ranking system, *Carnegie Classification*, was established in 1973. All other ranking systems were first published between 2003 and 2015. The study mentioned that three ranking systems were led by universities, two were led by agencies, five by consultancy or independent groups, and one was led by an institution established by a government.

In the analysis from the study [47], 4 systems for ranking out of 13 for evaluation claimed that they used their results to evaluate quality or performances of researches. Nine of thirteen systems use a total number of publications as an indicator for evaluation of quality or performances of researches—this is usually defined as a number of *peer-reviewed* articles from bases of Thomson Reuters' Web of Science Core Collections or SCOPUS which is maintained by Elsevier. On average, 33.8% ranking results are ascribed to publications and quotations or to various versions of these metrics [47].

Ranking systems that strongly rely on metrics related to publications and quotations are *Leiden Ranking*, *Shanghai*, *SCImago*, *URAP*, *US News and World Report*, and *EU U-Multirank* systems. The fact that SCImago ranking system takes the presence on the web into consideration by Google metrics [49], which is 20% of the total result of ranking, is very interesting. Similarly, Webometrics ranking list includes all universities of the world which are present on the web in the ranking system. The objective of this list is to encourage universities and their personnel to increase visibility of universities through creating more websites of university organizations and institutions. A survey of percentual participation and application of individual indicators applied by various systems is given in the work into details [47]. According to [47] current indicators are not adequate for an accurate assessment of results of researches, and they need to be amended and expanded to satisfy a standardized criterium.

### **3.1 Webometrics Ranking of World Universities**

Several research teams have been working on the development of web indicators since the mid-1990s. Realizing possibilities of this kind of ranking, the European Commission started several projects for this purpose: EICSTES (www.eicstes.org currently inactive), WISER (www.wiserweb.org—currently inactive), and www. webindicators.org (currently inactive).

After noting capabilities and importance of web search engines as the main agent to access information being processed and being processed by the web [53], one created new indicators [54, 11] which should have been milestones to solve

**27**

web every 6 months.

by the European Union.

*Advantages and Disadvantages of the Webometrics Ranking System*

data were the basis of information used in analyses [56, 51].

problems arising from instability of results of browsing via web search engines [55]

Most of the bibliometric indicators, such as a number of publications or quotations, are easily available. However, the problem with such access is that in this way only a limited number of information about activities, researchers, and observed institutions are available since only formal publications are taken into consideration. Actually, scientometric tasks should contain more elements, and more

However, including additional elements in an analysis, particularly when they are not easily available, may complicate the analysis and sometimes may be inapplicable when it comes to a global work plan. Among other things, there is an attitude that publications are not the only indicator of evaluation of professors. There are, inter alia, materials for lectures, raw data, slides from lectures, software, and bibliographic or linked lists (bookmarks), which are also deemed as relevant information

Besides these data, a structure and a content of all kinds of administrative information provided by an institution also have their value. All these elements speak for themselves when published publicly in the virtual world, the web world, and are very good indicators of an academic level of an educational institution. The fact that if someone is not on the web she/he does not exist supports the previous statement. Web space provides a comprehensive way to describe a wide range of activities of an institution where scientific publications represent only one of components which

Today, highly ranked researchers, institutions, and universities publish millions of pages with various materials composed of hundreds of departments and services,

Until now one has talked about webometrics methodology and systems for ranking of universities generally. However, the topic of this chapter is oriented toward a specific system of ranking of universities which applies webometrics methodology for the world's ranking of universities. This chapter will elaborate on Webometrics Ranking of World Universities, which was developed by and is under the competence of Cybermetrics Lab (Spanish National Research Council, CSIC) [50], who developed indicators called web ranking (WR) for the ranking process and who initially considered the following elements in the ranking process [51]: a number of published websites (S); a number of files contained, including PDF, ps, doc, and PPT form of documents (R); a number of articles collected via Google Scholar (GS)

Webometrics Ranking of World Universities is the largest list for academic ranking of higher education institutions. From 2004, Cybermetrics Lab has implemented an independent, objective, free, open scientific exercise for provision of reliable, multidimensional, updated, and useful information about performances of universities from all over the world on the basis of their presence and impact on the

*Cybermetrics Lab* has been developing quantitative studies on the Academic Web Network since the mid-1990s. The first indicator was introduced during the EASST/4S conference in Bielefeld (1996), and collection of web data from European universities started in 1999 with a support of EICSTES project financed

hundreds of research teams, and thousands of students on their websites.

database system (Sc); and a total number of external links (V).

The first catalogs of universities were created with projects EICSTES and WEISER, and the first preliminary list of these universities based on web indicators was published in 2004. This application of cybermetrics or webometrics techniques did not significantly differ from similar scientometric proposals where bibliometric

and artifacts arising from calculation of Web Impact Factors [6] [51].

*DOI: http://dx.doi.org/10.5772/intechopen.87207*

variables should be added [51].

may be found on websites.

about a professor's dedication to students [51].

### *Advantages and Disadvantages of the Webometrics Ranking System DOI: http://dx.doi.org/10.5772/intechopen.87207*

*Scientometrics Recent Advances*

employment.

versions of these metrics [47].

a standardized criterium.

**3.1 Webometrics Ranking of World Universities**

webindicators.org (currently inactive).

Currently, one implements a process of comparing and evaluating universities in the domain of academic and research performances with the existing system of ranking of universities. Most of the academic institutions rely on data obtained from the ranking system of universities which serve them as indicators of a progress of an institution over time in regard to other academic institutions [47]. Besides this, information from these ranking lists often serve as a basis for applying for and obtaining financial assets from founders or other institutions on the basis of a position on these lists [48]. On the other hand, potential beneficiaries of services of a university use these lists to evaluate academic institutions to decide which one to attend and to evaluate which one provides better options for education and further

The study [47] identified 24 ranking systems. Thirteen ranking systems were analyzed into details since their lists were active during the last couple of years, i.e., from 2015 to 2016. Other ranking systems were excluded from further analysis because they did not publish information and did not include indicators of their performances or published their ranking methodologies. The study evaluated between 500 and 5000 institutions. The oldest ranking system, *Carnegie Classification*, was established in 1973. All other ranking systems were first published between 2003 and 2015. The study mentioned that three ranking systems were led by universities, two were led by agencies, five by consultancy or independent groups, and one was led by an institution established by a government.

In the analysis from the study [47], 4 systems for ranking out of 13 for evaluation claimed that they used their results to evaluate quality or performances of researches. Nine of thirteen systems use a total number of publications as an indicator for evaluation of quality or performances of researches—this is usually defined as a number of *peer-reviewed* articles from bases of Thomson Reuters' Web of Science Core Collections or SCOPUS which is maintained by Elsevier. On average, 33.8% ranking results are ascribed to publications and quotations or to various

Ranking systems that strongly rely on metrics related to publications and quotations are *Leiden Ranking*, *Shanghai*, *SCImago*, *URAP*, *US News and World Report*, and *EU U-Multirank* systems. The fact that SCImago ranking system takes the presence on the web into consideration by Google metrics [49], which is 20% of the total result of ranking, is very interesting. Similarly, Webometrics ranking list includes all universities of the world which are present on the web in the ranking system. The objective of this list is to encourage universities and their personnel to increase visibility of universities through creating more websites of university organizations and institutions. A survey of percentual participation and application of individual indicators applied by various systems is given in the work into details [47]. According to [47] current indicators are not adequate for an accurate assessment of results of researches, and they need to be amended and expanded to satisfy

Several research teams have been working on the development of web indicators since the mid-1990s. Realizing possibilities of this kind of ranking, the European Commission started several projects for this purpose: EICSTES (www.eicstes.org currently inactive), WISER (www.wiserweb.org—currently inactive), and www.

After noting capabilities and importance of web search engines as the main agent to access information being processed and being processed by the web [53], one created new indicators [54, 11] which should have been milestones to solve

**26**

problems arising from instability of results of browsing via web search engines [55] and artifacts arising from calculation of Web Impact Factors [6] [51].

The first catalogs of universities were created with projects EICSTES and WEISER, and the first preliminary list of these universities based on web indicators was published in 2004. This application of cybermetrics or webometrics techniques did not significantly differ from similar scientometric proposals where bibliometric data were the basis of information used in analyses [56, 51].

Most of the bibliometric indicators, such as a number of publications or quotations, are easily available. However, the problem with such access is that in this way only a limited number of information about activities, researchers, and observed institutions are available since only formal publications are taken into consideration. Actually, scientometric tasks should contain more elements, and more variables should be added [51].

However, including additional elements in an analysis, particularly when they are not easily available, may complicate the analysis and sometimes may be inapplicable when it comes to a global work plan. Among other things, there is an attitude that publications are not the only indicator of evaluation of professors. There are, inter alia, materials for lectures, raw data, slides from lectures, software, and bibliographic or linked lists (bookmarks), which are also deemed as relevant information about a professor's dedication to students [51].

Besides these data, a structure and a content of all kinds of administrative information provided by an institution also have their value. All these elements speak for themselves when published publicly in the virtual world, the web world, and are very good indicators of an academic level of an educational institution. The fact that if someone is not on the web she/he does not exist supports the previous statement. Web space provides a comprehensive way to describe a wide range of activities of an institution where scientific publications represent only one of components which may be found on websites.

Today, highly ranked researchers, institutions, and universities publish millions of pages with various materials composed of hundreds of departments and services, hundreds of research teams, and thousands of students on their websites.

Until now one has talked about webometrics methodology and systems for ranking of universities generally. However, the topic of this chapter is oriented toward a specific system of ranking of universities which applies webometrics methodology for the world's ranking of universities. This chapter will elaborate on Webometrics Ranking of World Universities, which was developed by and is under the competence of Cybermetrics Lab (Spanish National Research Council, CSIC) [50], who developed indicators called web ranking (WR) for the ranking process and who initially considered the following elements in the ranking process [51]: a number of published websites (S); a number of files contained, including PDF, ps, doc, and PPT form of documents (R); a number of articles collected via Google Scholar (GS) database system (Sc); and a total number of external links (V).

Webometrics Ranking of World Universities is the largest list for academic ranking of higher education institutions. From 2004, Cybermetrics Lab has implemented an independent, objective, free, open scientific exercise for provision of reliable, multidimensional, updated, and useful information about performances of universities from all over the world on the basis of their presence and impact on the web every 6 months.

*Cybermetrics Lab* has been developing quantitative studies on the Academic Web Network since the mid-1990s. The first indicator was introduced during the EASST/4S conference in Bielefeld (1996), and collection of web data from European universities started in 1999 with a support of EICSTES project financed by the European Union.

These efforts are a continuation of scientometric research Cybermetrics Lab which started in 1994 and which was presented on a conference of the International Society for Scientometrics and Informetrics (ISSI, 1995–2011) and International Conferences on Scientific and Technology Indicators (STI-ENID, 1996–2012) and published in journals with a great impact effect (*Journal of Informetrics*, *Journal of American Society for Information Science and Technology*, *Scientometrics*, *Journal of Information Science*, *Processing Information and Management*, *Research Assessment*, and others). In 1997 one started issuing journal *Cybermetrics* dedicated to published works about webometrics.

After publishing of ranking of the University of Jiao Tong in Shanghai, Academic Ranking of World Universities (ARWU) [52] in 2003, team Cybermetrics Lab decided to approve the main innovations proposed by Liu and his team. It was suggested that ranking should be done on a basis of publicly available web data, combining variables in a composite directory and with a real global coverage. The first edition was published in 2004, and it has been issued two times a year since 2006. After 2008 the portal has included webometrics ranking for research centers, hospitals, repositories, and business schools.

### *3.1.1 Composite indicator*

Probably one of the most important contributions of *Shanghai* ranking was introduction of the **composite indicator**, which combines a system of weighing factors with a set of indicators. Traditional bibliometric indexes are made on ratios such as Garfield's journal impact factor which is based on variables which follow the power law and are useless for description of huge and complicated scenarios.

Ingwersen's proposal from 1998 [6] for a similarly designed *Web Impact Factor* which uses ratio *links/websites* (L/W) was equally useless due to mathematical artifacts which it generates.

Following the *Shanghai* model up, Cybermetrics Lab developed an indicator which transforms relation L/W into the following formula *aL + bW*, where *L* and *W* should be normalized in advance and *a* and *b* are weights which add 100%. Cybermetrics Lab strongly discouraged the usage of WIF due to its serious disadvantages. The composite indicator may be designed with different groups of variables and weights according to the needs of programmers and models. Webometrics applies "a priori" scientific model for the creation of a composite indicator. Other ranking lists chose arbitrary weights for very dependable variables and even combine raw values *with ratios.* None of them follows up a logic relation among variables related to activities and influential variables, i.e., each group represents 50% out of the total measure of weight.

Values should be normalized before any combination of variables, but the practice of application of percentage is mainly inaccurate due to *power law* distribution of data.

Webometrics *log* normalizes variables before combining in the ratio of 1:1 between activity/presence and visibility/influence of a group of indicators.

### *3.1.2 Collection of data for webometrics ranking*

Collection of a great quantity of data from the Internet, where one has to go through thousands of sites, may be done only automatically. One of the possibilities is to use commercial or free-of-charge crawlers, but adjustment of such systems for adjusted needs may be a complicated and difficult task, and it requires a significant participation of human and computer resources [57]. On the other hand, web search engines already have well-designed and tested systems for this need,

**29**

**Table 1.**

*Advantages and Disadvantages of the Webometrics Ranking System*

Search, Bing, Exalead, and Alexa [11] are used for these purposes.

and they do regular updates of their databases and have many tools which enable automatization of work so that machines may be easily adjusted to extract required data. Furthermore, web search engines are the main agents in navigation process on the web, and therefore the presence of a web domain in their databases represents an indicator of visibility on the Internet. Commercial web search engines also have limitations, which often include inconsistent and rounded-off results of browsing, favoritism in geographic and language coverage of results of browsing, or frequent and nontransparent changes in their work procedures. Due to the mentioned problems, one uses several web search engines together in practice, when collecting data. The most popular search engines such as Google (and Google Scholar), Yahoo

Webometrics ranking system [58] performs an evaluation and ranking of universities of the world two times a year (January/February and June/July) by its own developed methodology. Webometrics ranking methodology includes several phases and applies several systems so that data necessary for ranking and analyses

According to [51], there are three key aspects that need to be measured in the

• Visibility, number of certain cases of appearance on other web hosts which refer to the analyzed web host (quotations of websites-hosts = number of external

Bibliometrics has traditionally ignored frequency of appearance of a journal on various locations or sources of data and has focused on an impact of a journal, i.e., relation between a number of quotations and a number of published articles in the journal. A similar approach was proposed in the case of Webometrics

Webometrics ranking performs monitoring of a certain group of parameters (criteria) (**Table 1**), but only size and visibility of a web host are included in the

**Criteria Indicator Sources Weight**

Visibility Number of external links (*V*) Yahoo, Exalead, Live 50%

Exalead

Number of papers (*Sc*) Google Scholar 12.5%

Google 12.5%

25%

Size Number of pages (*S*) Google, Yahoo, Live,

Number of rich files (PDF, PPT, DOC, and PS) (*R*)

Luminosity Number of external outlinks Subdomains Number of subdomains Popularity Number of visits

*Criteria and weights used in the calculation of the WR indicator [51].*

*DOI: http://dx.doi.org/10.5772/intechopen.87207*

*3.1.3 The webometrics ranking weighing model*

may be updated and collected in time.

• Size, i.e., quantity of published information

incoming links) obtained by a domain

• Popularity, which represents a number of visits to a website

academic web space:

ranking.

*Advantages and Disadvantages of the Webometrics Ranking System DOI: http://dx.doi.org/10.5772/intechopen.87207*

*Scientometrics Recent Advances*

works about webometrics.

*3.1.1 Composite indicator*

artifacts which it generates.

the total measure of weight.

*3.1.2 Collection of data for webometrics ranking*

tion of data.

hospitals, repositories, and business schools.

These efforts are a continuation of scientometric research Cybermetrics Lab which started in 1994 and which was presented on a conference of the International Society for Scientometrics and Informetrics (ISSI, 1995–2011) and International Conferences on Scientific and Technology Indicators (STI-ENID, 1996–2012) and published in journals with a great impact effect (*Journal of Informetrics*, *Journal of American Society for Information Science and Technology*, *Scientometrics*, *Journal of Information Science*, *Processing Information and Management*, *Research Assessment*, and others). In 1997 one started issuing journal *Cybermetrics* dedicated to published

After publishing of ranking of the University of Jiao Tong in Shanghai, Academic Ranking of World Universities (ARWU) [52] in 2003, team Cybermetrics Lab decided to approve the main innovations proposed by Liu and his team. It was suggested that ranking should be done on a basis of publicly available web data, combining variables in a composite directory and with a real global coverage. The first edition was published in 2004, and it has been issued two times a year since 2006. After 2008 the portal has included webometrics ranking for research centers,

Probably one of the most important contributions of *Shanghai* ranking was introduction of the **composite indicator**, which combines a system of weighing factors with a set of indicators. Traditional bibliometric indexes are made on ratios such as Garfield's journal impact factor which is based on variables which follow the

Ingwersen's proposal from 1998 [6] for a similarly designed *Web Impact Factor* which uses ratio *links/websites* (L/W) was equally useless due to mathematical

Following the *Shanghai* model up, Cybermetrics Lab developed an indicator which transforms relation L/W into the following formula *aL + bW*, where *L* and *W* should be normalized in advance and *a* and *b* are weights which add 100%. Cybermetrics Lab strongly discouraged the usage of WIF due to its serious disadvantages. The composite indicator may be designed with different groups of variables and weights according to the needs of programmers and models. Webometrics applies "a priori" scientific model for the creation of a composite indicator. Other ranking lists chose arbitrary weights for very dependable variables and even combine raw values *with ratios.* None of them follows up a logic relation among variables related to activities and influential variables, i.e., each group represents 50% out of

Values should be normalized before any combination of variables, but the practice of application of percentage is mainly inaccurate due to *power law* distribu-

Webometrics *log* normalizes variables before combining in the ratio of 1:1 between activity/presence and visibility/influence of a group of indicators.

Collection of a great quantity of data from the Internet, where one has to go through thousands of sites, may be done only automatically. One of the possibilities is to use commercial or free-of-charge crawlers, but adjustment of such systems for adjusted needs may be a complicated and difficult task, and it requires a significant participation of human and computer resources [57]. On the other hand, web search engines already have well-designed and tested systems for this need,

power law and are useless for description of huge and complicated scenarios.

**28**

and they do regular updates of their databases and have many tools which enable automatization of work so that machines may be easily adjusted to extract required data. Furthermore, web search engines are the main agents in navigation process on the web, and therefore the presence of a web domain in their databases represents an indicator of visibility on the Internet. Commercial web search engines also have limitations, which often include inconsistent and rounded-off results of browsing, favoritism in geographic and language coverage of results of browsing, or frequent and nontransparent changes in their work procedures. Due to the mentioned problems, one uses several web search engines together in practice, when collecting data. The most popular search engines such as Google (and Google Scholar), Yahoo Search, Bing, Exalead, and Alexa [11] are used for these purposes.
