**International Higher Education Rankings at a Glance: How to Valorise the Research in Social Sciences and Humanities?**

José M. Gómez-Sancho and Carmen Pérez-Esparrells *The University of Zaragoza & The Autonomous University of Madrid, Spain* 

#### **1. Introduction**

As part of the international comparison of higher education, the last decade witnessed an explosion of rankings of higher education institutions (hereinafter HEIs) at both national and international level. Consequently, in almost all countries HEIs are compared in national rankings compiled by both the public and private sector; worldwide, furthermore, so-called global university rankings have been elaborated and most countries are concerned about their implications. This process is relentless, and despite being sharply criticised, such rankings or league tables are increasingly being used to compare HEIs. Their use is partly responsible for increased competition among universities at national level and, more importantly, among the most prestigious institutions at international level.

The international university rankings attempt "simplistically" to assess the "quality of higher education", a concept which is far from clear, particularly as an abstract term, and often misunderstood as to what it measures. Quality is not one-dimensional, but rather multidimensional: the quality of teaching, the quality of research, quality as a combination of activities (teaching and learning, research, development and innovation), quality as an institutional mission, etc. As a result of this multidimensionality there is no consensus on what constitutes "quality" or "excellence" in higher education. Furthermore, quality is not homogeneous within each HEI, as its schools, departments or programs vary. In general, although the majority of the more established rankings attempt to measure precisely academic and scientific quality in both teaching and research, experience has shown that the most popular global rankings (i.e. the rankings of world-class universities) in fact reflect many factors related to institutional reputation and prestige, and there is an acknowledged lack of an appropriate battery of performance indicators at international level to comprehensively measure the total quality of HEIs and to consider all the fields of knowledge in which they work. So far, the critical element is the management of knowledge and how to generate "good" rankings. The theory behind of rankings seems clear, it has a substantive meaning but it is not trouble-free the construction of indicators for all the areas (European University Association, 2011).

We examine the major global rankings (the ARWU, the QS World University rankings and the new THES World University Rankings) and other international rankings specialized in

International Higher Education Rankings at a Glance:

and to improve its strategic planning.

their interdisciplinary characteristics (Liu, 2009).

(A&HCI).

teaching.

How to Valorise the Research in Social Sciences and Humanities? 359

university reputation or size. Finally, all of them use a relative small number of indicators (based in the acquisition and publication of "reliable data" on university webs, HEIs leaders,

ARWU is the most widely known internationally ranking, yet also one of the most controversial. Its original intention was to rank world-class universities and, eventually, to begin benchmarking top Chinese universities using internationally comparable third party data available to all to check (Liu, 2009). Its ultimate objective was to measure scientific productivity and the project was performed by solely the academic interests of the University of Shanghai, to analyse its position in a worldwide classification of universities

The ARWU was born as a "generalist" ranking, including factors related to research, teaching and the reputation and size of HEIs. It lists the top 500 institutions worldwide, on the basis of four criteria: the quality of education (the number of alumni who have earned a Nobel Prize or have been awarded a Fields Medal); the quality of Faculty (the number of researchers who have earned a Nobel Prize in physics, chemistry, medicine or economics and/or a Fields medal in mathematics, and the number of highly cited researchers in the fields established by the Web of Knowledge or WoK); research output (the number of articles published in the journals Nature and Science; and the number of publications in the Science Citation Index (SCI) and Social Science Citation Index (SSCI) of the Web of Science or WoS), weighted by the size of the institution (academic performance of institution in relation to the number of full-time equivalent academic staff). The ISI WoK includes a multitude of databases, among which are the Journal Citation Report, the Derwent Innovation Index, the Essential Science Indicator, the ISIHighlyCited.com or the WoS, which in turn includes, among others, the SCI, the SSCI and the Arts & Humanities Citation Index

Since 2007 this has listed the top 100 universities, in five main fields: Natural Sciences and Mathematics, Engineering and Information Technology, Life Sciences and Agriculture, Medicine and Pharmacy, and Social Sciences. Since 2009 it has also provided information regarding the top 100 universities in more specific areas: Mathematics, Physics, Pharmacy, Computer Science and Economics. Arts and Humanities are not ranked because of the technical difficulties in finding internationally comparable indicators with reliable data. Psychology and other cross-disciplinary fields are not included in the ranking because of

The criteria of this ranking can be grouped alternatively, as in Table 1, which shows more clearly that research activity is the basis of almost all the indicators used, using the number of Nobel Prize winners and Fields Medal winners to evaluate not only research but also

Despite being strongly criticised, the popularity of the ARWU ranking has steadily increased. One important criticism concerns its bias towards large universities and, above all, towards research-based universities. Although the ranking corrects for institutional size (per capita academic performance of institution), this factor only accounts for 10% of the total ranking, while other indicators are expressed in absolute terms and, therefore, benefit the larger HEIs. In turn, the bias in favour of research means ARWU is a "partial" ranking and, more worryingly, ignores teaching activities. Such activities are perhaps those most appreciated

governmental agencies, etc.) and assign weights (if any) arbitrarily to each of them.

assessing research undertaken by HEIs (HEEACT, CWTS-Leiden and SIR). None of these lists provide a complete picture of the sector, especially for social sciences and humanities, but there is no doubt that research world-class is inherently international. Most of these international rankings confer great importance to research activities or focus their assessments only on this issue, as shown the proliferation of rankings about research only.

The aim of this chapter is two-fold. First, we discuss the advantages and disadvantages of the international university rankings, pointing out the deficiencies in the treatment of social sciences and humanities research which virtually disable the concept of an "ideal" ranking of higher education institutions. Second, we review the current literature which attempts to bridge the gap between Sciences (Natural Sciences, Life Sciences, Medicine and Pharmacy) and Social Sciences and Humanities. This challenge cannot be sidestepped, especially when ranking providers are classifying and comparing institutions engaged in different activities such as teaching, research and the transfer of knowledge and involved in different fields of research and in different teaching programs in the context of the internationalisation of higher education.

### **2. Some features of international rankings of Higher Education Institutions**

The increasing globalisation of higher education has resulted in an explosion in the global rankings of HEIs. Great interest has been shown all over the world in the state of the question, and there exist an increasing number of empirical studies proposing classifications of them. This section offers, firstly, a synthetic review of global university rankings from differing approaches; secondly, the most important league tables which have classified HEIs attending only research are reviewed and focusing the treatment and problems in social sciences and humanities; and finally, discussing the main advantages and disadvantages of these rankings as a whole.

#### **2.1 Global university rankings**

Rankings of HEIs have become established in the age of globalisation as a result of the internationalisation of higher education, which has led to increasingly global competition and a proliferation of so-called worldwide rankings, also known as ARWU (Academic Rankings of World Universities), after the name of the first such ranking, published in 2003. Therefore, the university ranking phenomenon is a very current phenomenon. We now describe the main characteristics (performance indicators, admissions criteria, etc.) of three most widely known internationally global HEI rankings.The first, developed now by the Shangai Ranking Consultancy and the Institute of Higher Education of the Shanghai Jiao Tong University, is the Shangai Academic Ranking of World Universities (or ARWU). The second and third, published by The Times Higher Education World University Ranking, in cooperation with Quacquarelli Symonds until 2009 and together with Thomson Reuters since 2010. Both of them rank universities on the basis of various criteria, such as prestige, peer review, awards received and research, among others.

All the above rankings analyse a considerable number of HEIs but usually present only those universities in the leading positions (Top-100, Top-200 or following top hundreds until Top-500). Some divide their lists by world regions or fields of knowledge. These kind of global rankings attempt to link research indicators to other issues related to teaching,

assessing research undertaken by HEIs (HEEACT, CWTS-Leiden and SIR). None of these lists provide a complete picture of the sector, especially for social sciences and humanities, but there is no doubt that research world-class is inherently international. Most of these international rankings confer great importance to research activities or focus their assessments only on this issue, as shown the proliferation of rankings about research only. The aim of this chapter is two-fold. First, we discuss the advantages and disadvantages of the international university rankings, pointing out the deficiencies in the treatment of social sciences and humanities research which virtually disable the concept of an "ideal" ranking of higher education institutions. Second, we review the current literature which attempts to bridge the gap between Sciences (Natural Sciences, Life Sciences, Medicine and Pharmacy) and Social Sciences and Humanities. This challenge cannot be sidestepped, especially when ranking providers are classifying and comparing institutions engaged in different activities such as teaching, research and the transfer of knowledge and involved in different fields of research and in different teaching programs in the context of the internationalisation of

**2. Some features of international rankings of Higher Education Institutions**  The increasing globalisation of higher education has resulted in an explosion in the global rankings of HEIs. Great interest has been shown all over the world in the state of the question, and there exist an increasing number of empirical studies proposing classifications of them. This section offers, firstly, a synthetic review of global university rankings from differing approaches; secondly, the most important league tables which have classified HEIs attending only research are reviewed and focusing the treatment and problems in social sciences and humanities; and finally, discussing the main advantages and disadvantages of

Rankings of HEIs have become established in the age of globalisation as a result of the internationalisation of higher education, which has led to increasingly global competition and a proliferation of so-called worldwide rankings, also known as ARWU (Academic Rankings of World Universities), after the name of the first such ranking, published in 2003. Therefore, the university ranking phenomenon is a very current phenomenon. We now describe the main characteristics (performance indicators, admissions criteria, etc.) of three most widely known internationally global HEI rankings.The first, developed now by the Shangai Ranking Consultancy and the Institute of Higher Education of the Shanghai Jiao Tong University, is the Shangai Academic Ranking of World Universities (or ARWU). The second and third, published by The Times Higher Education World University Ranking, in cooperation with Quacquarelli Symonds until 2009 and together with Thomson Reuters since 2010. Both of them rank universities on the basis of various criteria, such as prestige,

All the above rankings analyse a considerable number of HEIs but usually present only those universities in the leading positions (Top-100, Top-200 or following top hundreds until Top-500). Some divide their lists by world regions or fields of knowledge. These kind of global rankings attempt to link research indicators to other issues related to teaching,

higher education.

these rankings as a whole.

**2.1 Global university rankings** 

peer review, awards received and research, among others.

university reputation or size. Finally, all of them use a relative small number of indicators (based in the acquisition and publication of "reliable data" on university webs, HEIs leaders, governmental agencies, etc.) and assign weights (if any) arbitrarily to each of them.

ARWU is the most widely known internationally ranking, yet also one of the most controversial. Its original intention was to rank world-class universities and, eventually, to begin benchmarking top Chinese universities using internationally comparable third party data available to all to check (Liu, 2009). Its ultimate objective was to measure scientific productivity and the project was performed by solely the academic interests of the University of Shanghai, to analyse its position in a worldwide classification of universities and to improve its strategic planning.

The ARWU was born as a "generalist" ranking, including factors related to research, teaching and the reputation and size of HEIs. It lists the top 500 institutions worldwide, on the basis of four criteria: the quality of education (the number of alumni who have earned a Nobel Prize or have been awarded a Fields Medal); the quality of Faculty (the number of researchers who have earned a Nobel Prize in physics, chemistry, medicine or economics and/or a Fields medal in mathematics, and the number of highly cited researchers in the fields established by the Web of Knowledge or WoK); research output (the number of articles published in the journals Nature and Science; and the number of publications in the Science Citation Index (SCI) and Social Science Citation Index (SSCI) of the Web of Science or WoS), weighted by the size of the institution (academic performance of institution in relation to the number of full-time equivalent academic staff). The ISI WoK includes a multitude of databases, among which are the Journal Citation Report, the Derwent Innovation Index, the Essential Science Indicator, the ISIHighlyCited.com or the WoS, which in turn includes, among others, the SCI, the SSCI and the Arts & Humanities Citation Index (A&HCI).

Since 2007 this has listed the top 100 universities, in five main fields: Natural Sciences and Mathematics, Engineering and Information Technology, Life Sciences and Agriculture, Medicine and Pharmacy, and Social Sciences. Since 2009 it has also provided information regarding the top 100 universities in more specific areas: Mathematics, Physics, Pharmacy, Computer Science and Economics. Arts and Humanities are not ranked because of the technical difficulties in finding internationally comparable indicators with reliable data. Psychology and other cross-disciplinary fields are not included in the ranking because of their interdisciplinary characteristics (Liu, 2009).

The criteria of this ranking can be grouped alternatively, as in Table 1, which shows more clearly that research activity is the basis of almost all the indicators used, using the number of Nobel Prize winners and Fields Medal winners to evaluate not only research but also teaching.

Despite being strongly criticised, the popularity of the ARWU ranking has steadily increased. One important criticism concerns its bias towards large universities and, above all, towards research-based universities. Although the ranking corrects for institutional size (per capita academic performance of institution), this factor only accounts for 10% of the total ranking, while other indicators are expressed in absolute terms and, therefore, benefit the larger HEIs. In turn, the bias in favour of research means ARWU is a "partial" ranking and, more worryingly, ignores teaching activities. Such activities are perhaps those most appreciated

International Higher Education Rankings at a Glance:

change in the original indicators and the idiosyncrasy of this region.

HEIs cited above the average and to the detriment of those below.

International outlook - staff, students and research (7.5 per cent worth.)

and arrangements with overseas institutions (Harvey, 2008).

How to Valorise the Research in Social Sciences and Humanities? 361

Company has expanded the QS ranking to include the top 500 universities (http://www.topuniversities.com), classified, as Table 1 shows, according to six distinct indicators: academic peer review (composite score drawn from peer review survey); employer review (score based on responses to employer survey); Faculty student ratio; citations per faculty (citations of Scopus divided by the number of full-time equivalent staff at the institution); international Faculty (proportion of overseas staff); and international Students (proportion of international students). For 2011, this company has developed a specific ranking for Latin America called QS University Rankings Latin America. This ranking has in common with the QS World University ranking some indicators (academic reputation, employer reputation, faculty student ratio, international staff and international students) but includes new indicators (staff with PHD, web impact, papers per Faculty, citation per paper). In addition, the company has established other weights, justifying this

Similarly to ARWU, it compiles a league table based on the various activities performed by universities, but also includes peer review (by academics from all over the world, experts in the relevant knowledge fields, employees, etc.), which provides a multidimensional view of the relative strengths of the world's leading universities. Furthermore, the weight of research activity is more nuanced, as citations received only account for 20 per cent of the total score.

The QS Ranking is well-known internationally, yet also highly controversial, due to its use of biannual assessments made by academic staff and employers, leading to two serious shortcomings. Firstly, it is a highly subjective ranking; the peer review results are strongly influenced by the country composition of the sample of experts consulted and also by the (unnamed) employers surveyed. Secondly, as Ioannidis et al. (2007) indicate, it is unlikely that any expert possesses a global view of the inner workings of teaching at institutions worldwide. Its remaining indicators are also highly questionable. The citations system is also biased in various ways, as mentioned above: not only does it favour the larger HEIs and those more specialised in scientific fields, it also generates a new bias in favour of those

The new THE World University rankings (2010 and 2011) have some changes in relation to QS World University Rankings, with more data variables, with 13 variables in total, as shown in table 1. This new ranking weights divided between: Teaching - the learning environment (30 per cent of the overall ranking score); Research - volume, income and reputation (30 per cent); Citations - Influence research (30 per cent ); Industry income - innovation (2.5 per cent) and

However, despite being the last ranking in time, this league table has the same problems as the previous rankings: research has much more worth of what the indicator said, since the majority of variables used are directly or indirectly connected with it as well they combine new indicators into a final score as a "simple" calculation. With regard to the internationalisation of universities, an international staff and international students may be a good proxy of the ability to attract foreign talent and of the demand for education (from foreign students), yet the number of foreign students or the proportion of foreign academic staff is no guarantee of quality. In some cases, such as the UK, HEIs are unable to state how many international students (or even academic staff) they have, due to varying definitions


So urce: Aguillo et al. (2010) and autho rs ' elabo ratio n

Table 1. Methodology of global university rankings

by society in general (students, parents, employers and other stakeholders) and, moreover, as various authors have argued, it is very difficult to accept that the Nobel Prizes and Fields Medals won by former students and current researchers are the most appropriate indicators for assessing the quality of education. As European University Association (2011, p.16) alleged, there is a demand for more "democratic" rankings. As other global league tables, ARWU classifies a few hundred of universities which are the "best" in the world: "in so doing, they are created problems for thousands of "normal" universities which simply do their job".

The technical and methodological problems inherent in research outputs mean that such results are incomplete and therefore biased in many ways (Van Raan, 2005). Firstly, by using the number of articles published in the SCI and SSCI, outputs are restricted to research published in scientific journals; omitting alternative formats such as books, which in some fields are extremely important (especially in Social Sciences and Humanities). This criticism can be extrapolated to other classifications, as discussed below. Secondly, the abovementioned databases have a much stronger representation of journals in the Sciences fields (Life, Physical, Health) compared to others such as Social Sciences or, especially, and Humanities, which is virtually non-existent.

According to Harvey (2008), the status afforded by the ARWU ranking suggests that once a league table is published, its methodology is no longer questioned and its statistical data are taken to be objective, especially if this ranking reinforces commonly held prejudices and preconceptions regarding reputation. Universities from Anglo-Saxon countries and in general, from the North hemisphere perform better. Together with the unquestionable quality of many English language HEIs, these benefit from the fact that English is the international language of research, and many indicators are based on citations which benefit such HEIs. As Altbach (2006) remarks, Americans tend to cite Americans.

The following two global rankings arise from the splitting of the ranking that was published by The Times Higher Education Supplement in collaboration with QS Company, was the THE-QS World University Ranking, published annually since 2004. After 2010, the QS

Nature & Science 20% Proportion of internationally coauthored

Citations (Scopus) 20%

Employers reputation 10%

by society in general (students, parents, employers and other stakeholders) and, moreover, as various authors have argued, it is very difficult to accept that the Nobel Prizes and Fields Medals won by former students and current researchers are the most appropriate indicators for assessing the quality of education. As European University Association (2011, p.16) alleged, there is a demand for more "democratic" rankings. As other global league tables, ARWU classifies a few hundred of universities which are the "best" in the world: "in so doing, they are created problems for thousands of "normal" universities which simply do

The technical and methodological problems inherent in research outputs mean that such results are incomplete and therefore biased in many ways (Van Raan, 2005). Firstly, by using the number of articles published in the SCI and SSCI, outputs are restricted to research published in scientific journals; omitting alternative formats such as books, which in some fields are extremely important (especially in Social Sciences and Humanities). This criticism can be extrapolated to other classifications, as discussed below. Secondly, the abovementioned databases have a much stronger representation of journals in the Sciences fields (Life, Physical, Health) compared to others such as Social Sciences or, especially, and

According to Harvey (2008), the status afforded by the ARWU ranking suggests that once a league table is published, its methodology is no longer questioned and its statistical data are taken to be objective, especially if this ranking reinforces commonly held prejudices and preconceptions regarding reputation. Universities from Anglo-Saxon countries and in general, from the North hemisphere perform better. Together with the unquestionable quality of many English language HEIs, these benefit from the fact that English is the international language of research, and many indicators are based on citations which benefit

The following two global rankings arise from the splitting of the ranking that was published by The Times Higher Education Supplement in collaboration with QS Company, was the THE-QS World University Ranking, published annually since 2004. After 2010, the QS

such HEIs. As Altbach (2006) remarks, Americans tend to cite Americans.

SIZE Size Size of institution 10% Included in indicators Included in indicators

Alumni Nobel & Fields 10%

PRESTIGE Prestige Staff Nobel & Fields 20% Academic reputation 40%

Criteria ARWU QS Nº HEIs RANKING 500 700

Research impact Highly cited researchers 20%

Table 1. Methodology of global university rankings

Humanities, which is virtually non-existent.

So urce: Aguillo et al. (2010) and autho rs ' elabo ratio n

Research output

Quality of education

RESEARCH

TEACHING

their job".

SCI & SSCI 20% Papers per academic and staff 6%

research papers 2.5%

THE 400

Research income from industry 2.5% Research income (scaled) 6% Citation impact normalised (WoK) 30% Income per academic 2.25%

PhD awards / bachelor's awards 2.25% PhD awards per academic 6% Academic reputation (teaching / research) 15 / 18%

Student/staff ratio 20% Undergraduates admitted per academic 4.5% International students 5% Ratio of international to domestic students 2.5% International staff 5% Ratio of international to domestic staff 2.5% Company has expanded the QS ranking to include the top 500 universities (http://www.topuniversities.com), classified, as Table 1 shows, according to six distinct indicators: academic peer review (composite score drawn from peer review survey); employer review (score based on responses to employer survey); Faculty student ratio; citations per faculty (citations of Scopus divided by the number of full-time equivalent staff at the institution); international Faculty (proportion of overseas staff); and international Students (proportion of international students). For 2011, this company has developed a specific ranking for Latin America called QS University Rankings Latin America. This ranking has in common with the QS World University ranking some indicators (academic reputation, employer reputation, faculty student ratio, international staff and international students) but includes new indicators (staff with PHD, web impact, papers per Faculty, citation per paper). In addition, the company has established other weights, justifying this change in the original indicators and the idiosyncrasy of this region.

Similarly to ARWU, it compiles a league table based on the various activities performed by universities, but also includes peer review (by academics from all over the world, experts in the relevant knowledge fields, employees, etc.), which provides a multidimensional view of the relative strengths of the world's leading universities. Furthermore, the weight of research activity is more nuanced, as citations received only account for 20 per cent of the total score.

The QS Ranking is well-known internationally, yet also highly controversial, due to its use of biannual assessments made by academic staff and employers, leading to two serious shortcomings. Firstly, it is a highly subjective ranking; the peer review results are strongly influenced by the country composition of the sample of experts consulted and also by the (unnamed) employers surveyed. Secondly, as Ioannidis et al. (2007) indicate, it is unlikely that any expert possesses a global view of the inner workings of teaching at institutions worldwide. Its remaining indicators are also highly questionable. The citations system is also biased in various ways, as mentioned above: not only does it favour the larger HEIs and those more specialised in scientific fields, it also generates a new bias in favour of those HEIs cited above the average and to the detriment of those below.

The new THE World University rankings (2010 and 2011) have some changes in relation to QS World University Rankings, with more data variables, with 13 variables in total, as shown in table 1. This new ranking weights divided between: Teaching - the learning environment (30 per cent of the overall ranking score); Research - volume, income and reputation (30 per cent); Citations - Influence research (30 per cent ); Industry income - innovation (2.5 per cent) and International outlook - staff, students and research (7.5 per cent worth.)

However, despite being the last ranking in time, this league table has the same problems as the previous rankings: research has much more worth of what the indicator said, since the majority of variables used are directly or indirectly connected with it as well they combine new indicators into a final score as a "simple" calculation. With regard to the internationalisation of universities, an international staff and international students may be a good proxy of the ability to attract foreign talent and of the demand for education (from foreign students), yet the number of foreign students or the proportion of foreign academic staff is no guarantee of quality. In some cases, such as the UK, HEIs are unable to state how many international students (or even academic staff) they have, due to varying definitions and arrangements with overseas institutions (Harvey, 2008).

International Higher Education Rankings at a Glance:

Productivity

Impact

Number of citations 1998-2008

Number of citations 2007-2008

Average citations 1998-2008

Number of highly-cited papers 1998-2008 15% Number of artícles in highimpact journals 2008 15%

Table 2. Methodology of research university rankings

H Index 2007-2008

Criteria Nº HEIs IN RANKING

RESEARCH

So urce: Autho rs ' elabo ratio n

HEIs.

institutions and countries?

How to Valorise the Research in Social Sciences and Humanities? 363

Light Green

Dark Green

Orange

Excellence Excellence Rate

As Table 2 shows, this ranking produces a synthetic indicator using eight variables in groups related to productivity, impact and excellence, neutralising the effect of institutional size by using the h-index indicator and adding the citation average. In our opinion, the effects of these additions are slight, since neither is suitable for dealing with data from fields with very different publication behaviours, while they both favour those fields which cite above the average. Four indicators are related to the number of full-time equivalent (FTE) faculty members (the number of articles in the previous eleven and the previous two years, and the number of citations in the previous eleven and the previous two years). In our view, this factor makes a interesting contribution to the ranking but in turn raises some important questions; for example, how has it been possible to obtain homogeneous data from different

Another drawback to the HEEACT ranking is that, as in many others, the weighting of the variables is arbitrary. The double counting of articles has also been criticised, as these are allocated to all institutions involved in their elaboration; thus, an article produced in collaboration by various institutions is counted several times, as if it were an article produced by each of the participating HEIs. In certain fields (astronomy, for example) it is not unusual for articles to be signed by over 200 authors and include over one hundred

Furthermore, the variables used reinforce rather than correct the biases possessed by the ISI databases. The system benefits those fields better represented in the sample, those with the greatest propensity to be cited, and those which, in general, accumulate a higher percentage of citations in the two years prior to the year evaluated. Its indiscriminate use may cause,

In 2008 the Centre for Science and Technology Studies (CWTS) at Leiden University developed a set of four research rankings. The Centre enjoys great prestige in Europe and

therefore, undesirable effects (Gomez-Sancho and Mancebón-Torrubia, 2009).

(MNCS2)

10% 30%

10%

10%

20% 50%

Number of artícles 1998-2008 10% 20% Yellow Number of publications (P) Output Number of publications Number of artícles 2008 10% Blue Citations per publication (CPP) Int. Coll. International collaboration

HEEACT CWTS SIR 500 500 3042

> Citations per publication by the size-independent, fieldnormalised average impact (CPP/FCSm)

> Citations per publication by the size-independent, fieldnormalised average impact

> Number of publications multiplied by citations per publication by field-normalised average (P\*CPP/FCSm)

Normalized Impact

High Quality Publications

Specialitation Index

Relationship of an institution's average scientific impact and the world average

Ratio of publications in the most influential scholarly journals of the world (first quartile) in their categories

Thematic concentration / dispersion of an institution's scientific output

Percentage of an institution's scientific output is included into the set formed by the 10% of the most cited papers in their respective scientific fields

Due to the many criticisms, we can see a new strategy between the ranking providers in an attempt to relativize the results. To do this, besides the traditional global ranking, currently they offer new university rankings by field of knowledge or geographical location.

Taking into account the drawbacks of these rankings and their bias, a problem that can emerge is that universities are more worried for improving ranking positions than for improving quality. In an attempt to improve their positions in these three global rankings, HEIs can tempted to improve their performance specifically in those fields which are measured by ranking indicators (European University Association, 2011). At the same time, their funding policies may be benefiting those areas (and faculty, schools, and centres) related especially to natural sciences and medicine to the detriment of the areas of social sciences and humanities. For this reason, self-regulation may not be sufficient and in autumn 2010, the International Rakings Expert Group (IREG) announced that it would start a rankings audit exercise.

#### **2.2 Research university rankings**

The following three league tables, much more recent, focus exclusively on research performance: these are the Performance Ranking of Scientific Papers for World Universities, elaborated by the Higher Education and Accreditation Council of Taiwan, the Leiden World Ranking, constructed by the Centre for Science and Technology Studies (CWTS) of Leiden University and the SCImago Institutions Ranking, compiled by the SCImago Research Group. These three rankings, devoted exclusively to research, are focused on bibliographic indicators, whether new or combinations of existing measurements.

Obviously, the sharp criticism aimed at these three major global rankings has recently led to the elaboration of new lists with reduced objectives i.e. the analysis of merely one aspect of research performance, namely scientific papers. Therefore, the Social Sciences, Humanities and Engineering are worst classified by principle in these three such rankings. The methodology employed by these three research rankings is outlined in Table 2.

The HEEACT ranking has been published since 2007 and provides results for the top 500 research universities and their location by continent and country. Beginning in 2008, this HEEACT has additionally classified the top 300 universities in six broad knowledge fields: Agriculture, Clinical Medicine, Engineering, Life Sciences, Natural Sciences and Social Sciences (which can in turn be classified by continent and country).

This ranking is based on data from various sources, supplied by the ISI Web of Knowledge (WoK). Comparability is thereby affected, for various reasons. Firstly, it focuses exclusively on papers published in scientific journals, ignoring publications in other formats (books, reports, projects, etc.). Secondly, the databases underrepresent publications in Social Sciences and Engineering and, above all, Arts &Humanities. Although the Arts & Humanities Citation Index (A&HCI) is also available, the HEEACT ranking does not include the database because "it may fail to objectively and accurately represent the research performance of arts and humanities researchers". As European University Association (2011) said, the database mainly indexes English-language journals, while arts and humanities researchers often publish in their native languages and in various forms of publications (e.g. books). According to the rankers, focusing on data obtained from SCI and SSCI allows for fairer comparisons across universities globally.

Due to the many criticisms, we can see a new strategy between the ranking providers in an attempt to relativize the results. To do this, besides the traditional global ranking, currently

Taking into account the drawbacks of these rankings and their bias, a problem that can emerge is that universities are more worried for improving ranking positions than for improving quality. In an attempt to improve their positions in these three global rankings, HEIs can tempted to improve their performance specifically in those fields which are measured by ranking indicators (European University Association, 2011). At the same time, their funding policies may be benefiting those areas (and faculty, schools, and centres) related especially to natural sciences and medicine to the detriment of the areas of social sciences and humanities. For this reason, self-regulation may not be sufficient and in autumn 2010, the International Rakings Expert Group (IREG) announced that it would start

The following three league tables, much more recent, focus exclusively on research performance: these are the Performance Ranking of Scientific Papers for World Universities, elaborated by the Higher Education and Accreditation Council of Taiwan, the Leiden World Ranking, constructed by the Centre for Science and Technology Studies (CWTS) of Leiden University and the SCImago Institutions Ranking, compiled by the SCImago Research Group. These three rankings, devoted exclusively to research, are focused on bibliographic

Obviously, the sharp criticism aimed at these three major global rankings has recently led to the elaboration of new lists with reduced objectives i.e. the analysis of merely one aspect of research performance, namely scientific papers. Therefore, the Social Sciences, Humanities and Engineering are worst classified by principle in these three such rankings. The

The HEEACT ranking has been published since 2007 and provides results for the top 500 research universities and their location by continent and country. Beginning in 2008, this HEEACT has additionally classified the top 300 universities in six broad knowledge fields: Agriculture, Clinical Medicine, Engineering, Life Sciences, Natural Sciences and Social

This ranking is based on data from various sources, supplied by the ISI Web of Knowledge (WoK). Comparability is thereby affected, for various reasons. Firstly, it focuses exclusively on papers published in scientific journals, ignoring publications in other formats (books, reports, projects, etc.). Secondly, the databases underrepresent publications in Social Sciences and Engineering and, above all, Arts &Humanities. Although the Arts & Humanities Citation Index (A&HCI) is also available, the HEEACT ranking does not include the database because "it may fail to objectively and accurately represent the research performance of arts and humanities researchers". As European University Association (2011) said, the database mainly indexes English-language journals, while arts and humanities researchers often publish in their native languages and in various forms of publications (e.g. books). According to the rankers, focusing on data obtained from SCI and

indicators, whether new or combinations of existing measurements.

Sciences (which can in turn be classified by continent and country).

SSCI allows for fairer comparisons across universities globally.

methodology employed by these three research rankings is outlined in Table 2.

they offer new university rankings by field of knowledge or geographical location.

a rankings audit exercise.

**2.2 Research university rankings** 


Table 2. Methodology of research university rankings

As Table 2 shows, this ranking produces a synthetic indicator using eight variables in groups related to productivity, impact and excellence, neutralising the effect of institutional size by using the h-index indicator and adding the citation average. In our opinion, the effects of these additions are slight, since neither is suitable for dealing with data from fields with very different publication behaviours, while they both favour those fields which cite above the average. Four indicators are related to the number of full-time equivalent (FTE) faculty members (the number of articles in the previous eleven and the previous two years, and the number of citations in the previous eleven and the previous two years). In our view, this factor makes a interesting contribution to the ranking but in turn raises some important questions; for example, how has it been possible to obtain homogeneous data from different institutions and countries?

Another drawback to the HEEACT ranking is that, as in many others, the weighting of the variables is arbitrary. The double counting of articles has also been criticised, as these are allocated to all institutions involved in their elaboration; thus, an article produced in collaboration by various institutions is counted several times, as if it were an article produced by each of the participating HEIs. In certain fields (astronomy, for example) it is not unusual for articles to be signed by over 200 authors and include over one hundred HEIs.

Furthermore, the variables used reinforce rather than correct the biases possessed by the ISI databases. The system benefits those fields better represented in the sample, those with the greatest propensity to be cited, and those which, in general, accumulate a higher percentage of citations in the two years prior to the year evaluated. Its indiscriminate use may cause, therefore, undesirable effects (Gomez-Sancho and Mancebón-Torrubia, 2009).

In 2008 the Centre for Science and Technology Studies (CWTS) at Leiden University developed a set of four research rankings. The Centre enjoys great prestige in Europe and

International Higher Education Rankings at a Glance:

the 100% of the highly cited papers in their respective fields.

careful attention to the many phenomena that influence citation rates".

Savino, 2007; Van Raan, 2005).

America, Europe, Australia, Asia, the Pacific, etc.)

**2.3 Positive and negative aspects of rankings of higher education institutions** 

This proliferation of international rankings has led to many authors worldwide analysing their advantages and disadvantages (Altbach, 2006; Carey, 2006; Dill and Soo, 2005; Dill and van Vaught, 2010; Harvey, 2008; Hazelkorn, 2007 and 2011; Huisman, 2008; Marginson and Van Der Wende, 2007; Nkomo, 2009; Stella and Woodhouse, 2006; Thakur, 2007; Usher and

The impact upon HEIs of such uses of rankings is, in some aspects, highly positive. League tables allow improving data collection and providing more information on the Internet, causing HEIs to place increasing importance upon the information thus provided. There is no doubt that rankings can provide free publicity worldwide, especially for universities in the top positions in their country, region or area of influence (North America, Latin

Furthermore, rankings influence student choice of institutions at which to pursue their studies, particularly in so-called "overseas markets" (as a consequence of globalisation and the internationalisation of higher education). The impact (both inside and outside HEIs) of published rankings is increasing, due to their dissemination by the media. With regard to the recruitment of faculty and students of students, rankings may offer a good "business card" for the best placed HEIs, and such universities can compete more effectively for top scholars and the most able students. Other key stakeholders in higher education, such as

How to Valorise the Research in Social Sciences and Humanities? 365

world; those ranked in the first quartile (25%) in their categories as ordered by SCImago Journal. The two new indicators are: the specialization index indicates which measures the extent of the matrix concentration/dispersion of an institution's scientific output, indicating generalist vs. specialized institutions respectively. This indicator is computed according to the Gini Index; and excellence rate, that is to say, proportion of output in the set formed by

To sum-up, the last two above mentioned research rankings are a "good" research evaluation framework for science policy-makers, research managers and so on. As European University Association (2011) shown, ranking providers are trying to improve the methodology they use. In other words, improving the calculation methods is not enough; rankings should make efforts to cover all research areas on an equal basis. However, the improvements are often technical rather than conceptual. Despite these types of rankings having substantially improved the definition of research excellence, their failure to assess university teaching quality has been the subject of criticism, even sharper than the three major global rankings reviewed. This is because good research does not ensure good teaching and learning i.e. the quality of education. While it is difficult to find comparable indicators of teaching performance at a global level, it is unacceptable to exclude this important mission of HEIs, at least at international and national level. Moreover, regarding the journal citation impact factor itself, European University Association (2011) notes that, especially in social sciences and humanities, rankings do not correlate very well with impact factors. In the above fields and in engineering, other sources, such as books and proceedings, are important as well. A warning on this issue can even be found on the Thomson-Reuter's website which states that "the impact factor should not be used without

worldwide, and its rankings are based on its own bibliographic indicators, giving them greater objectivity than the rankings reviewed above. Its more elaborate rankings (Dark Green, Light Green and Orange)have succeeded in eliminating some of the serious problems involved in the use of Web of Science and Scopus, by correctly identifying the institutions to which researchers belong and corrections for practically all errors and inconsistencies in the raw publication and citation data.

The CWTS employs a multi-indicator approach, with an identical methodology and database for the five colour-coded rankings; these are yellow, green (light and dark), orange and blue (Table 2), and give the top 100 and the top 250 European institutions and now, they show the top 500 worldwide. The data used are the papers published between 2004 and 2008 and the citations between 2004 and 2009. Each ranking focuses on a specific indicator; for example, the yellow ranking only takes into account the number of publications (which favours larger institutions) and, as in the previous cases, articles by researchers from more than one institution are counted twice, but in this case an effort is made to correctly identify HEIs. The blue ranking uses the number of citations per publication but is not normalised by field, to the detriment (as the authors warn) of certain universities such as those specialising in engineering or social sciences, as the number of citations per publication is significantly lower than in other fields such as medicine and health sciences as a whole. The two green rankings (light and dark) use two different indicators to relativize the previous indicator by normalising citation counts for the field and the year in which a publication was published. Thus, the light green rank use citations per publication which compare with mean field citation score (CPP/FCS and is called "crown indicator"), while the dark green ranking use the mean-normalised citation score (MNCS2 and is called "alternative crown indicator), as Waltman et al. (2011) have explained. Finally, the orange ranking multiplies the total number of publications by their relative impact in their given field. The important improvements made by the CWTS have not exempted it from criticism. Some such criticisms once again allege that it benefits larger universities (except for the green ranking) and focuses only on research (scientific papers), ignoring the other activities in which HEIs engage. More specific criticisms have focused on the inappropriateness of normalising bibliographic indicators by knowledge field (Opthof and Leydesdorff, 2010), an issue which remains highly controversial (van Raan et al., 2010).

The last of the three research rankings was published for the first time in 2010 by the Spanish research group SCImago and takes the name of SCImago Institutions Rankings (SIR). It differs from the previous two in its data source; the publication and citation data used in its elaboration are obtained from the Scopus database compiled by Elsevier. It provides a ranking which includes all types of research institutions instead of being restricted to universities. It is much broader than the other league tables examined here; it comprises 3,042 institutions in the 3rd edition, all of which have published over 100 papers per year. The data it collects cover the years 2005-2009.

In the last edition (October 2011) SIR comprises six indicators (two of them are new): output, measured by the number of publications; international collaboration, measured by the percentage of articles produced through such joint contribution; normalized impact, the values, expressed in percentages, show the relationship of an institution's average scientific impact and the world average; High quality publications indicator is the ratio of publications that an institution publishes in the most influential scholarly journals of the

worldwide, and its rankings are based on its own bibliographic indicators, giving them greater objectivity than the rankings reviewed above. Its more elaborate rankings (Dark Green, Light Green and Orange)have succeeded in eliminating some of the serious problems involved in the use of Web of Science and Scopus, by correctly identifying the institutions to which researchers belong and corrections for practically all errors and inconsistencies in the

The CWTS employs a multi-indicator approach, with an identical methodology and database for the five colour-coded rankings; these are yellow, green (light and dark), orange and blue (Table 2), and give the top 100 and the top 250 European institutions and now, they show the top 500 worldwide. The data used are the papers published between 2004 and 2008 and the citations between 2004 and 2009. Each ranking focuses on a specific indicator; for example, the yellow ranking only takes into account the number of publications (which favours larger institutions) and, as in the previous cases, articles by researchers from more than one institution are counted twice, but in this case an effort is made to correctly identify HEIs. The blue ranking uses the number of citations per publication but is not normalised by field, to the detriment (as the authors warn) of certain universities such as those specialising in engineering or social sciences, as the number of citations per publication is significantly lower than in other fields such as medicine and health sciences as a whole. The two green rankings (light and dark) use two different indicators to relativize the previous indicator by normalising citation counts for the field and the year in which a publication was published. Thus, the light green rank use citations per publication which compare with mean field citation score (CPP/FCS and is called "crown indicator"), while the dark green ranking use the mean-normalised citation score (MNCS2 and is called "alternative crown indicator), as Waltman et al. (2011) have explained. Finally, the orange ranking multiplies the total number of publications by their relative impact in their given field. The important improvements made by the CWTS have not exempted it from criticism. Some such criticisms once again allege that it benefits larger universities (except for the green ranking) and focuses only on research (scientific papers), ignoring the other activities in which HEIs engage. More specific criticisms have focused on the inappropriateness of normalising bibliographic indicators by knowledge field (Opthof and Leydesdorff, 2010), an issue which

The last of the three research rankings was published for the first time in 2010 by the Spanish research group SCImago and takes the name of SCImago Institutions Rankings (SIR). It differs from the previous two in its data source; the publication and citation data used in its elaboration are obtained from the Scopus database compiled by Elsevier. It provides a ranking which includes all types of research institutions instead of being restricted to universities. It is much broader than the other league tables examined here; it comprises 3,042 institutions in the 3rd edition, all of which have published over 100 papers

In the last edition (October 2011) SIR comprises six indicators (two of them are new): output, measured by the number of publications; international collaboration, measured by the percentage of articles produced through such joint contribution; normalized impact, the values, expressed in percentages, show the relationship of an institution's average scientific impact and the world average; High quality publications indicator is the ratio of publications that an institution publishes in the most influential scholarly journals of the

raw publication and citation data.

remains highly controversial (van Raan et al., 2010).

per year. The data it collects cover the years 2005-2009.

world; those ranked in the first quartile (25%) in their categories as ordered by SCImago Journal. The two new indicators are: the specialization index indicates which measures the extent of the matrix concentration/dispersion of an institution's scientific output, indicating generalist vs. specialized institutions respectively. This indicator is computed according to the Gini Index; and excellence rate, that is to say, proportion of output in the set formed by the 100% of the highly cited papers in their respective fields.

To sum-up, the last two above mentioned research rankings are a "good" research evaluation framework for science policy-makers, research managers and so on. As European University Association (2011) shown, ranking providers are trying to improve the methodology they use. In other words, improving the calculation methods is not enough; rankings should make efforts to cover all research areas on an equal basis. However, the improvements are often technical rather than conceptual. Despite these types of rankings having substantially improved the definition of research excellence, their failure to assess university teaching quality has been the subject of criticism, even sharper than the three major global rankings reviewed. This is because good research does not ensure good teaching and learning i.e. the quality of education. While it is difficult to find comparable indicators of teaching performance at a global level, it is unacceptable to exclude this important mission of HEIs, at least at international and national level. Moreover, regarding the journal citation impact factor itself, European University Association (2011) notes that, especially in social sciences and humanities, rankings do not correlate very well with impact factors. In the above fields and in engineering, other sources, such as books and proceedings, are important as well. A warning on this issue can even be found on the Thomson-Reuter's website which states that "the impact factor should not be used without careful attention to the many phenomena that influence citation rates".

#### **2.3 Positive and negative aspects of rankings of higher education institutions**

This proliferation of international rankings has led to many authors worldwide analysing their advantages and disadvantages (Altbach, 2006; Carey, 2006; Dill and Soo, 2005; Dill and van Vaught, 2010; Harvey, 2008; Hazelkorn, 2007 and 2011; Huisman, 2008; Marginson and Van Der Wende, 2007; Nkomo, 2009; Stella and Woodhouse, 2006; Thakur, 2007; Usher and Savino, 2007; Van Raan, 2005).

The impact upon HEIs of such uses of rankings is, in some aspects, highly positive. League tables allow improving data collection and providing more information on the Internet, causing HEIs to place increasing importance upon the information thus provided. There is no doubt that rankings can provide free publicity worldwide, especially for universities in the top positions in their country, region or area of influence (North America, Latin America, Europe, Australia, Asia, the Pacific, etc.)

Furthermore, rankings influence student choice of institutions at which to pursue their studies, particularly in so-called "overseas markets" (as a consequence of globalisation and the internationalisation of higher education). The impact (both inside and outside HEIs) of published rankings is increasing, due to their dissemination by the media. With regard to the recruitment of faculty and students of students, rankings may offer a good "business card" for the best placed HEIs, and such universities can compete more effectively for top scholars and the most able students. Other key stakeholders in higher education, such as

International Higher Education Rankings at a Glance:

How to Valorise the Research in Social Sciences and Humanities? 367

activities, such as continuous education) or national government goals (in particular, to encourage universities in the developing world) (IHEP, 2008). Lastly, rankings can also become a threat to equity, because they may limit access by disadvantaged students or may

The unstoppable growth of rankings in Europe has led to the publication of a series of guidelines for rankings of European institutions. The International Ranking Expert Group produced the Berlin Principles on Ranking of Higher Education Institutions, which require that rankings must correspond to the problems or goals evaluated. The Group also concluded that multidimensional evaluations are more appropriate than a single overall score, especially in the classification and comparison of the missions and objectives of academic institutions with clearly different cultures and/or languages; consequently, it is more appropriate to distinguish between different types of HEIs and between different programmes and disciplines, particularly at the national level (or even at the regional level: Europe, America, Asia, the Pacific, etc.). In other words, the complexity of measuring and validly ranking the academic quality of institutions such as universities which pursue multiple goals and produce multiple outputs suggests that multidimensional rankings are likely to be more valid, all things considered, than synthetic rankings. This is the target of the two recent European initiatives undertaken by the European Commission to increase the visibility of European universities through two tools: the design of institutional profiles (U-

In summary, the "power of seduction" of university rankings poses new challenges to elaborate alternative indicators which measure more precisely the different activities undertaken by HEIs (Nkomo, 2009). As Marginson and Van Der Wende (2007) note, it is necessary to reshape rankings to make them transparent, free from vested interests and methodologically consistent; although the producers of rankings try to employ an adequate methodological basis, it must be underlined that rankings may produce certain "perverse" effects in terms of improving ranking positions such as EUA (2011) said: inequitable student selection by elite universities ("reputation survey by students can be manipulated by directly telling students to lie"), an increase in HEI size to be able to compete ("merging universities just to get onto league tables") and "even bibliographic may be flawed due to the manipulation of the data (Assessment of University-Based Research, 2010). These consequences, in an attempt to improve their positions in the rankings, may already be

be used only by students from higher-income families (Harvey, 2008).

Map) and a multi-dimensional global ranking (called U-Multirank).

occurring to some extent in parts of the academic and scientific worlds.

**matter?** 

**3. Humanities and social sciences in the university rankings: What is the** 

The popularity of university rankings worldwide has meant that nobody (media, families, even governments) has questioned the indicators selected in the rankings and their weights. HEIs which two decades ago were compared only to others in their own country are now compared to peer institutions around the world. Universities in the Anglo-Saxon countries, which have produced rankings for over twenty years, obtain the best results. Together with the undeniable quality of many Anglo-Saxon HEIs, these are favoured by the fact that English has become the language of research and graduate programs, thereby benefiting English-speaking countries (Altbach, 2006). Moreover, the idiosyncrasy of many fields, social sciences and humanities among them, and the lack of good indicators at international

industries governments, etc. have an in direct interest in rankings, which have an important impact on research funding and fundraising strategies. Faculty and administrative staff have a direct interest in rankings,, while students are able to employ rankings in their decisions to study at institutions abroad.

Policymakers are also increasingly aware of the positions achieved by the HEIs under their responsibility. Some European countries (Germany, France, Spain, etc.) have implemented policies in favour of the construction of "international campuses of excellence", indirectly employing league tables to allocate public funding, by taking into account HEI performance in research or by "merging" public research institutions and technology centres with nearby cities and the socioeconomic environment. Finally, HEIs and leaders in higher education are strongly influenced by international rankings. Used correctly, these can stimulate strategies of reinforcing strategic research lines, increasing international projection, attracting international talent, creating a powerful network for knowledge transfer, etc.

Nevertheless, today's rankings have been (and will continue to be) strongly criticised; some such objections question the general usefulness of HEI rankings, while specific methodological criticisms have been described above. Some authors have gone so far as to describe such league tables as "nonsense" (Adler and Harzing, 2009); in addition to their many limitations, they allegedly measure quality in a "naïve" way (Harvey, 2008). Others argue that rankings are constructed using indicators of what can be measured rather than what is important to measure (Stella and Woodhouse, 2006), due principally to the lack of homogeneous data. Thus, global university rankings do not provide a complete picture of HEIs worldwide, as many of the indicators employed are only poor proxies of the attributes they purport to quantify.

Rankings are skewed principally toward research and, within research, toward articles published in scientific journals. This fact introduces serious doubts on the validity of ranking positions, since they represent neither all HEI activity nor all the research they perform. This partial analysis favours certain knowledge fields, publication formats (scientific papers) and research published in English. This situation is, to some extent, the result of funding for academic research and doctoral research education in all the developed countries also being strongly biased towards the sciences, medicine, and engineering (Dill and van Vught, 2010), meaning that rankings emphasising research in these fields may be justified in part as reflecting old biases and interests. It should also be noted that private universities outside the USA generally eschew research and doctoral training in the sciences and engineering, and thus these latter activities are more clearly a "public good" in most countries, undersupplied by the market; consequently, large public subsidies for these fields can be justified by a "market failure" argument. In contrast, many research fields in the humanities and social sciences, as well as non-university vocational and professional training, are dominated by the private university sector. Furthermore, the majority of rankings do not consider institutional size, meaning the top-ranked universities are the largest institutions (Dill and Soo, 2005, Usher and Savino, 2007). Thirdly, the weights applied to the indicators are often subjective and tend to be arbitrary and vary over time (Stella and Woodhouse, 2006).

Finally, the use of rankings has serious drawbacks for HEIs themselves. According to Carey (2006), they constitute a loss of freedom and independence for HEIs to control their "brand" and the terms of their success. Universities themselves have little influence on the methodology employed by international rankings. Indeed, rankings can conflict with other HEI priorities (for example greater attention to excellence in teaching or other specific

industries governments, etc. have an in direct interest in rankings, which have an important impact on research funding and fundraising strategies. Faculty and administrative staff have a direct interest in rankings,, while students are able to employ rankings in their

Policymakers are also increasingly aware of the positions achieved by the HEIs under their responsibility. Some European countries (Germany, France, Spain, etc.) have implemented policies in favour of the construction of "international campuses of excellence", indirectly employing league tables to allocate public funding, by taking into account HEI performance in research or by "merging" public research institutions and technology centres with nearby cities and the socioeconomic environment. Finally, HEIs and leaders in higher education are strongly influenced by international rankings. Used correctly, these can stimulate strategies of reinforcing strategic research lines, increasing international projection, attracting

Nevertheless, today's rankings have been (and will continue to be) strongly criticised; some such objections question the general usefulness of HEI rankings, while specific methodological criticisms have been described above. Some authors have gone so far as to describe such league tables as "nonsense" (Adler and Harzing, 2009); in addition to their many limitations, they allegedly measure quality in a "naïve" way (Harvey, 2008). Others argue that rankings are constructed using indicators of what can be measured rather than what is important to measure (Stella and Woodhouse, 2006), due principally to the lack of homogeneous data. Thus, global university rankings do not provide a complete picture of HEIs worldwide, as many of the indicators employed are only poor proxies of the attributes they purport to quantify.

Rankings are skewed principally toward research and, within research, toward articles published in scientific journals. This fact introduces serious doubts on the validity of ranking positions, since they represent neither all HEI activity nor all the research they perform. This partial analysis favours certain knowledge fields, publication formats (scientific papers) and research published in English. This situation is, to some extent, the result of funding for academic research and doctoral research education in all the developed countries also being strongly biased towards the sciences, medicine, and engineering (Dill and van Vught, 2010), meaning that rankings emphasising research in these fields may be justified in part as reflecting old biases and interests. It should also be noted that private universities outside the USA generally eschew research and doctoral training in the sciences and engineering, and thus these latter activities are more clearly a "public good" in most countries, undersupplied by the market; consequently, large public subsidies for these fields can be justified by a "market failure" argument. In contrast, many research fields in the humanities and social sciences, as well as non-university vocational and professional training, are dominated by the private university sector. Furthermore, the majority of rankings do not consider institutional size, meaning the top-ranked universities are the largest institutions (Dill and Soo, 2005, Usher and Savino, 2007). Thirdly, the weights applied to the indicators are often subjective and tend to be

Finally, the use of rankings has serious drawbacks for HEIs themselves. According to Carey (2006), they constitute a loss of freedom and independence for HEIs to control their "brand" and the terms of their success. Universities themselves have little influence on the methodology employed by international rankings. Indeed, rankings can conflict with other HEI priorities (for example greater attention to excellence in teaching or other specific

international talent, creating a powerful network for knowledge transfer, etc.

arbitrary and vary over time (Stella and Woodhouse, 2006).

decisions to study at institutions abroad.

activities, such as continuous education) or national government goals (in particular, to encourage universities in the developing world) (IHEP, 2008). Lastly, rankings can also become a threat to equity, because they may limit access by disadvantaged students or may be used only by students from higher-income families (Harvey, 2008).

The unstoppable growth of rankings in Europe has led to the publication of a series of guidelines for rankings of European institutions. The International Ranking Expert Group produced the Berlin Principles on Ranking of Higher Education Institutions, which require that rankings must correspond to the problems or goals evaluated. The Group also concluded that multidimensional evaluations are more appropriate than a single overall score, especially in the classification and comparison of the missions and objectives of academic institutions with clearly different cultures and/or languages; consequently, it is more appropriate to distinguish between different types of HEIs and between different programmes and disciplines, particularly at the national level (or even at the regional level: Europe, America, Asia, the Pacific, etc.). In other words, the complexity of measuring and validly ranking the academic quality of institutions such as universities which pursue multiple goals and produce multiple outputs suggests that multidimensional rankings are likely to be more valid, all things considered, than synthetic rankings. This is the target of the two recent European initiatives undertaken by the European Commission to increase the visibility of European universities through two tools: the design of institutional profiles (U-Map) and a multi-dimensional global ranking (called U-Multirank).

In summary, the "power of seduction" of university rankings poses new challenges to elaborate alternative indicators which measure more precisely the different activities undertaken by HEIs (Nkomo, 2009). As Marginson and Van Der Wende (2007) note, it is necessary to reshape rankings to make them transparent, free from vested interests and methodologically consistent; although the producers of rankings try to employ an adequate methodological basis, it must be underlined that rankings may produce certain "perverse" effects in terms of improving ranking positions such as EUA (2011) said: inequitable student selection by elite universities ("reputation survey by students can be manipulated by directly telling students to lie"), an increase in HEI size to be able to compete ("merging universities just to get onto league tables") and "even bibliographic may be flawed due to the manipulation of the data (Assessment of University-Based Research, 2010). These consequences, in an attempt to improve their positions in the rankings, may already be occurring to some extent in parts of the academic and scientific worlds.

#### **3. Humanities and social sciences in the university rankings: What is the matter?**

The popularity of university rankings worldwide has meant that nobody (media, families, even governments) has questioned the indicators selected in the rankings and their weights. HEIs which two decades ago were compared only to others in their own country are now compared to peer institutions around the world. Universities in the Anglo-Saxon countries, which have produced rankings for over twenty years, obtain the best results. Together with the undeniable quality of many Anglo-Saxon HEIs, these are favoured by the fact that English has become the language of research and graduate programs, thereby benefiting English-speaking countries (Altbach, 2006). Moreover, the idiosyncrasy of many fields, social sciences and humanities among them, and the lack of good indicators at international

International Higher Education Rankings at a Glance:

those league tables in the first positions.

that are translated into the results and figures shown below.

List which includes all journals covered by scientific products.

**Field Papers Citations**

Table 3. Papers and Citations grouped by field in WoS

MOLECULAR BIOLOGY &

PHARMACOLOGY &

How to Valorise the Research in Social Sciences and Humanities? 369

Secondly, the abovementioned databases have a much stronger representation of journals in the Natural Sciences and Medicine fields compared to others such as Social Sciences or, especially, Arts and Humanities, which is virtually non-existent. Thus, an exceptionally "good" university specialised in Humanities or in Social Sciences will never be classified in

Moreover, a more detailed analysis of the two major databases worldwide, Web of Knowledge and Scopus, unequivocally shows tiny prominence of both the Social Sciences and Humanities in such databases. This fact involves many disadvantages in the rankings

The Web of Knowledge from Thomson Reuters, the most used in the rankings discussed in this chapter, is a compendium of databases. Its main sources are scientific journals, although recently included scientific conferences. To know how many journals are used to obtain all the information needed to generate the database before, we have to turn to Master Journal

> **Cites per Paper**

CLINICAL MEDICINE 2,094,266 26,401,171 12.61 21.23 21.23 25.42 25.42 CHEMISTRY 1,180,298 13,118,786 11.11 11.97 33.20 12.63 38.06 BIOLOGY & BIOCHEMISTRY 554,085 9,187,524 16.58 5.62 38.82 8.85 46.90 PHYSICS 865,207 7,471,580 8.64 8.77 47.59 7.19 54.10

GENETICS 276,962 6,764,744 24.42 2.81 50.40 6.51 60.61 NEUROSCIENCE & BEHAVIOR 297,238 5,601,258 18.84 3.01 53.41 5.39 66.00 PLANT & ANIMAL SCIENCE 550,056 4,153,348 7.55 5.58 58.99 4.00 70.00 ENGINEERING 816,830 3,910,309 4.79 8.28 67.27 3.77 73.77 MATERIALS SCIENCE 459,129 3,268,171 7.12 4.66 71.93 3.15 76.92 ENVIRONMENT/ECOLOGY 269,152 2,989,116 11.11 2.73 74.65 2.88 79.79 PSYCHIATRY/PSYCHOLOGY 245,935 2,715,784 11.04 2.49 77.15 2.62 82.41 GEOSCIENCES 278,333 2,640,333 9.49 2.82 79.97 2.54 84.95 IMMUNOLOGY 121,714 2,572,216 21.13 1.23 81.20 2.48 87.43 MICROBIOLOGY 165,426 2,524,589 15.26 1.68 82.88 2.43 89.86

TOXICOLOGY 185,581 2,224,182 11.98 1.88 84.76 2.14 92.00 SOCIAL SCIENCES, GENERAL 463,123 2,121,452 4.58 4.70 89.46 2.04 94.04 SPACE SCIENCE 122,081 1,743,100 14.28 1.24 90.70 1.68 95.72 AGRICULTURAL SCIENCES 207,058 1,432,603 6.92 2.10 92.80 1.38 97.10 ECONOMICS & BUSINESS 167,433 1,025,992 6.13 1.70 94.49 0.99 98.09 COMPUTER SCIENCE 262,878 999,267 3.8 2.67 97.16 0.96 99.05 MATHEMATICS 262,443 894,707 3.41 2.66 99.82 0.86 99.91 MULTIDISCIPLINARY 17,807 89,253 5.01 0.18 100.00 0.09 100.00 So urce: Es s ential Science Indicato rs (updated as o f September 1, 2011 to co ver a 10-year + 6 mo nth perio d, J anuary 1, 2001-J une30, 2001) and autho rs ' elabo ratio n

**Percentage of** 

**Accumulated Percentage of Papers**

**Percentage of Citations**

**Accumulated Percentage of Citations**

**Papers**

level, has led to a weaker position of these fields (programs and universities) at global university rankings.

Therefore, the problem in the rankings for the fields such as Social Sciences and Humanities lies in the minor weight that their relevant activities (both teaching and research) have in the indicators employed in these league tables. With regard to teaching activity, it is symptomatic that in almost all the countries, Social Sciences and Arts and Humanities are "catching" together more than a half of university students, not being rewarded their efforts in those global rankings. Moreover, these fields (and subfields) are penalized because, in many cases, the educational effort is higher in those studies and programs because they are continually increasing the number of students per group, more than in other fields.

Given the difficulty of finding qualitative indicators for this crucial activity, perhaps the most recognized one by society, ranking providers have chosen to reflect university research performance focusing on *scientific papers* (and bibliographic indicators) and to discard the search of other teaching and learning indicators which measure more accurately the teaching quality and excellence. As EUA (2011) has described, the quality of teaching is measured indirectly at the best. This criticism itself invalidates the results provided by the so called generalist international rankings, since one university could be in the top of these league tables only because of the excellence of its research activities, independently of its teaching performance.

If we consider higher education a key goal in our societies, it is essential that teaching and learning activities are measured and evaluated to provide an accurate picture of universities performance. This fact increases the urgency of developing internationally comparable indicators that allow teachers and scholars to measure this core activity. In this context, OECD is elaborating an Assessment of Higher Education Learning Outcomes (AHELO). This project is an attempt to compare HEIs internationally on the basis of actual learning outcomes. Three testing instruments will be developed within AHELO: one for measuring generic skills and two for testing discipline-specific skills, in economics and engineering. In these initial phases of the project, the developers have yet to find answers to a number of questions, including whether it is possible to develop instruments to capture learning outcomes that are perceived as valid in diverse national and institutional contexts.

Regarding research activity, as we have shown, the technical and methodological problems inherent in measuring research outputs mean that such results are incomplete and therefore biased in many ways. Firstly, the main format for the measuring of scientific knowledge are the articles published in the SCI and SSCI, that is to say, outputs are restricted to research published exclusively on papers published in scientific journals, ignoring publications in other formats (books, reports, projects, etc.). In the Web of Science, in the database sources such as SCI-EXPANDED, SSCI, A&HCI, the majority (more than 90%) of them are catalogued as articles, and later as proceeding paper, review, meeting abstracts, notes, editorial material, letters, book reviews, books, corrections, discussions, reprints, etc. Currently, Conference Proceedings Citation Index- Science (CPCI-S) and Conference Proceedings Citation Index-Social Science & Humanities (CPCI-SSH) are included. The problem of including these new databases is double: firstly, because many of these publications appear later as a traditional scientific journal articles (incurring double counting) and, secondly, the requirements of the reviewers for inclusion are usually much lower than in the scientific journals.

level, has led to a weaker position of these fields (programs and universities) at global

Therefore, the problem in the rankings for the fields such as Social Sciences and Humanities lies in the minor weight that their relevant activities (both teaching and research) have in the indicators employed in these league tables. With regard to teaching activity, it is symptomatic that in almost all the countries, Social Sciences and Arts and Humanities are "catching" together more than a half of university students, not being rewarded their efforts in those global rankings. Moreover, these fields (and subfields) are penalized because, in many cases, the educational effort is higher in those studies and programs because they are

Given the difficulty of finding qualitative indicators for this crucial activity, perhaps the most recognized one by society, ranking providers have chosen to reflect university research performance focusing on *scientific papers* (and bibliographic indicators) and to discard the search of other teaching and learning indicators which measure more accurately the teaching quality and excellence. As EUA (2011) has described, the quality of teaching is measured indirectly at the best. This criticism itself invalidates the results provided by the so called generalist international rankings, since one university could be in the top of these league tables only because of the excellence of its research activities, independently of its

If we consider higher education a key goal in our societies, it is essential that teaching and learning activities are measured and evaluated to provide an accurate picture of universities performance. This fact increases the urgency of developing internationally comparable indicators that allow teachers and scholars to measure this core activity. In this context, OECD is elaborating an Assessment of Higher Education Learning Outcomes (AHELO). This project is an attempt to compare HEIs internationally on the basis of actual learning outcomes. Three testing instruments will be developed within AHELO: one for measuring generic skills and two for testing discipline-specific skills, in economics and engineering. In these initial phases of the project, the developers have yet to find answers to a number of questions, including whether it is possible to develop instruments to capture learning

outcomes that are perceived as valid in diverse national and institutional contexts.

reviewers for inclusion are usually much lower than in the scientific journals.

Regarding research activity, as we have shown, the technical and methodological problems inherent in measuring research outputs mean that such results are incomplete and therefore biased in many ways. Firstly, the main format for the measuring of scientific knowledge are the articles published in the SCI and SSCI, that is to say, outputs are restricted to research published exclusively on papers published in scientific journals, ignoring publications in other formats (books, reports, projects, etc.). In the Web of Science, in the database sources such as SCI-EXPANDED, SSCI, A&HCI, the majority (more than 90%) of them are catalogued as articles, and later as proceeding paper, review, meeting abstracts, notes, editorial material, letters, book reviews, books, corrections, discussions, reprints, etc. Currently, Conference Proceedings Citation Index- Science (CPCI-S) and Conference Proceedings Citation Index-Social Science & Humanities (CPCI-SSH) are included. The problem of including these new databases is double: firstly, because many of these publications appear later as a traditional scientific journal articles (incurring double counting) and, secondly, the requirements of the

continually increasing the number of students per group, more than in other fields.

university rankings.

teaching performance.

Secondly, the abovementioned databases have a much stronger representation of journals in the Natural Sciences and Medicine fields compared to others such as Social Sciences or, especially, Arts and Humanities, which is virtually non-existent. Thus, an exceptionally "good" university specialised in Humanities or in Social Sciences will never be classified in those league tables in the first positions.

Moreover, a more detailed analysis of the two major databases worldwide, Web of Knowledge and Scopus, unequivocally shows tiny prominence of both the Social Sciences and Humanities in such databases. This fact involves many disadvantages in the rankings that are translated into the results and figures shown below.

The Web of Knowledge from Thomson Reuters, the most used in the rankings discussed in this chapter, is a compendium of databases. Its main sources are scientific journals, although recently included scientific conferences. To know how many journals are used to obtain all the information needed to generate the database before, we have to turn to Master Journal List which includes all journals covered by scientific products.


Table 3. Papers and Citations grouped by field in WoS

International Higher Education Rankings at a Glance:

Life Sciences (3955)

Physical Sciences (6475)

Health Sciences (6235)

Social Sciences (6003)

So urce: Sco pus and autho rs ' ellabo ratio n

because all of them have their pros and cons.

Field (number of Journals, etc.) Subfield

How to Valorise the Research in Social Sciences and Humanities? 371

Table 4. Number of Scientific Publications grouped by field and subfield in Scopus (2011)

In both databases, therefore, there is a bias that "damages" to Social Sciences and Humanities, the latter field even more pronounced. This is because these databases were not designed with an evaluation mission (or an assessment role) in the beginning, but later they provided data that allowed the development of quality indicators of papers and journals. More recently they have expanded their uses to evaluate institutions and even faculties and schools. In our view, the policymakers must be cautious with all these assessment processes because they are not yet recognized as suitable indicators by most experts and specialists,

Although this is not the aim of this chapter a detailed analysis of each of the indicators, it identifies the most representative ones. Clearly, both the number of articles and the citations received are the main indicators, i.e., those that derive all the others. In the Wok, particularly

General (89) General 89

Agricultural and Biological Sciences(all) 1589 Biochemistry, Genetics and Molecular Biology(all) 1619 Immunology and Microbiology(all) 433 Neuroscience(all) 403 Pharmacology, Toxicology and Pharmaceutics(all) 633 Chemical Engineering(all) 517 Chemistry(all) 751 Computer Science(all) 1173 Earth and Planetary Sciences(all) 900 Energy(all) 321 Engineering(all) 2186 Environmental Science(all) 972 Materials Science(all) 990 Mathematics(all) 1024 Physics and Astronomy(all) 922 Medicine(all) 5706 Nursing(all) 466 veterinary(all) 177 Dentistry(all) 131 Health Professions(all) 328 Arts and Humanities(all) 1756 Business, Management and Accounting(all) 943 Decision Sciences(all) 230 Economics, Econometrics and Finance(all) 655 Psychology(all) 884 Social Sciences(all) 3524

Number of Journals, Trade Journals, Books Series & Conference Proceedings

In 2011, 16772 journals were included. The best known databases are: the Science Citation Index Expanded (SCI), which includes data from 8401 journals, the Social Science Citation Index (SSCI) with data from 2995 journals, Arts & Humanities Citation Index (A & HCI) composed of 1656 journals. Among them, the Journal Citation Report (JCR) highlights, where the main journals are evaluated with different indicators. In 2010 edition, it included 8073 journals of sciences, 2731 in the social sciences and there was no edition for Arts & Humanities. The journals included in JCR total figures are less than the previous databases, because on the one hand, there are journals included in both editions and on the other hand, JCR contains journals that have changed their names or merged recently, but they remain to perform the calculations of the indicators for the new journals that have emerged recently.

This is already a first bias in favour of journals covering the fields called *scientific*, but this bias becomes overwhelming if we focus on the distribution of the number of articles or citations. Essential Science Indicators figures are conclusive. Firstly, Arts and Humanities fields are not considered and Social Sciences fields are playing a secondary role. Therefore, the papers published in the last decade are concentrated in the Sciences fields (and among them, 5 of the 22 areas have more than 50 percent), leaving underrepresented areas such as Social Sciences in general with 4.70 percent, or Economic & Business with a 1.70 percent. The conclusion is even worse when the quality through citation is measured: four areas have the 50 per cent of the citations and Social Sciences in general (with 2.04 percent) and Economics & Business (with 0.99 percent) are very residual.

To sum-up, the assessment process of research activity significantly impairs the development of both, the Social Sciences and the Arts and Humanities. They are a minority in the journals considered. They are considered marginally by focusing on the number of articles (where the Arts & Humanities fields are vanished). Finally, they are residual when they measure the quality of articles for the citations received.

Scopus is the other international database that provides research indicators on which the results of the current rankings are based, but this database has not been widely used as the well known WoK. Scopus is somewhat broader, but the problem can be to include "questionable" quality journals. This database tries to be positioned opposite the WoK, including more scientific publications worldwide (not only American) and more scientific publications in undervalued fields, belonging to the Social Sciences and Humanities. Table 4 shows the current journals, books, etc. that are the sources of this database. Taking into account the results, it should be noted that publications may be included in more than one field or subfield (up to 4).

It is important to note that the Social Sciences (which include Arts and Humanities) have a greater number of publications than the former database. But, as with WoK, when analysing the number of papers and citations, these fields are again penalized. This database also provides interesting information regarding to one of the biases that research results have among the different science fields and art fields. These are the different formats used for the dissemination of the results (Table 5). In summary, articles in journals are the "way out" almost exclusively in Natural Sciences and Medicine, while in other fields besides the articles, they include other formats. Thus, in Engineering, Architecture and Computer Science the proceedings have much more interest and in Social Sciences and Humanities both books and book chapters are very important.

In 2011, 16772 journals were included. The best known databases are: the Science Citation Index Expanded (SCI), which includes data from 8401 journals, the Social Science Citation Index (SSCI) with data from 2995 journals, Arts & Humanities Citation Index (A & HCI) composed of 1656 journals. Among them, the Journal Citation Report (JCR) highlights, where the main journals are evaluated with different indicators. In 2010 edition, it included 8073 journals of sciences, 2731 in the social sciences and there was no edition for Arts & Humanities. The journals included in JCR total figures are less than the previous databases, because on the one hand, there are journals included in both editions and on the other hand, JCR contains journals that have changed their names or merged recently, but they remain to perform the calculations of the indicators for the new journals that have emerged recently. This is already a first bias in favour of journals covering the fields called *scientific*, but this bias becomes overwhelming if we focus on the distribution of the number of articles or citations. Essential Science Indicators figures are conclusive. Firstly, Arts and Humanities fields are not considered and Social Sciences fields are playing a secondary role. Therefore, the papers published in the last decade are concentrated in the Sciences fields (and among them, 5 of the 22 areas have more than 50 percent), leaving underrepresented areas such as Social Sciences in general with 4.70 percent, or Economic & Business with a 1.70 percent. The conclusion is even worse when the quality through citation is measured: four areas have the 50 per cent of the citations and Social Sciences in general (with 2.04 percent) and Economics

To sum-up, the assessment process of research activity significantly impairs the development of both, the Social Sciences and the Arts and Humanities. They are a minority in the journals considered. They are considered marginally by focusing on the number of articles (where the Arts & Humanities fields are vanished). Finally, they are residual when

Scopus is the other international database that provides research indicators on which the results of the current rankings are based, but this database has not been widely used as the well known WoK. Scopus is somewhat broader, but the problem can be to include "questionable" quality journals. This database tries to be positioned opposite the WoK, including more scientific publications worldwide (not only American) and more scientific publications in undervalued fields, belonging to the Social Sciences and Humanities. Table 4 shows the current journals, books, etc. that are the sources of this database. Taking into account the results, it should be noted that publications may be included in more than one

It is important to note that the Social Sciences (which include Arts and Humanities) have a greater number of publications than the former database. But, as with WoK, when analysing the number of papers and citations, these fields are again penalized. This database also provides interesting information regarding to one of the biases that research results have among the different science fields and art fields. These are the different formats used for the dissemination of the results (Table 5). In summary, articles in journals are the "way out" almost exclusively in Natural Sciences and Medicine, while in other fields besides the articles, they include other formats. Thus, in Engineering, Architecture and Computer Science the proceedings have much more interest and in Social Sciences and Humanities

& Business (with 0.99 percent) are very residual.

both books and book chapters are very important.

field or subfield (up to 4).

they measure the quality of articles for the citations received.


So urce: Sco pus and autho rs ' ellabo ratio n

Table 4. Number of Scientific Publications grouped by field and subfield in Scopus (2011)

In both databases, therefore, there is a bias that "damages" to Social Sciences and Humanities, the latter field even more pronounced. This is because these databases were not designed with an evaluation mission (or an assessment role) in the beginning, but later they provided data that allowed the development of quality indicators of papers and journals. More recently they have expanded their uses to evaluate institutions and even faculties and schools. In our view, the policymakers must be cautious with all these assessment processes because they are not yet recognized as suitable indicators by most experts and specialists, because all of them have their pros and cons.

Although this is not the aim of this chapter a detailed analysis of each of the indicators, it identifies the most representative ones. Clearly, both the number of articles and the citations received are the main indicators, i.e., those that derive all the others. In the Wok, particularly


International Higher Education Rankings at a Glance:

second mission and even the third mission of the HEIs.

done in these fields of knowledge apart from scientific papers?

the HEIs. We must devote time and resources to these challenges.

Vol.8, No.1, (March), pp.72–95, ISSN 1537-260X

so prevalent on university agendas actually.

**5. Acknowledgments** 

opinions are our own.

1084-0613

**6. References** 

How to Valorise the Research in Social Sciences and Humanities? 373

The solution is to make "more countable" the research in these fields, Social Sciences and Humanities, and to increase the visibility in the global university rankings of the so called

With respect to the measurement of the research in these fields, although there has been an increasing advance in the diffusion of results within the "scientific" standard in many of the sub-fields (economics, history or archaeology would be paradigmatic examples), some of them are still scarce. Should we change the way of spread the results giving less weight to scientific journal articles? Or should the databases providers try to capture what is being

The international efforts under development (especially initiatives from the European Commission) have stimulated the transparency of HEIs activities, through the description of university performance (institutional profiles) which allows benchmarking. This transformation implies also to measure the impact of HEIs activities on many educational aspects. No longer can international rankings afford not to take into account the human capital of the new generations formed in higher education and the innovation activities of

We believe that both issues could take place simultaneously. The pressure of the need to publish ("publish or perish") in prestigious journals is becoming more intense among faculty in these fields, but at the same time requires that books and book chapters appear in the databases, in order to highlight the outstanding formats of output knowledge in Social Sciences and Humanities. Once achieved, a more realistic picture of knowledge and scientific and artistic developments that arise in the universities, it is necessary that the indicators are constructed to evaluate these outputs, to minimize the bias currently observed among different fields of knowledge. This latter aspect, more technical, is essential for legitimating of the rankings, to guarantee the neutrality of the assessments among fields (and universities). In addition, the future global rankings have to be able to catch up the diversification and differentiation among universities (and countries) designing individual institutional profiles,

We are grateful to Dr. Luciano Galan and Dra. María Jesús Mancebón Torrubia, who provided useful comments on earlier drafts of this chapter. Obviously, all errors and

Adler, N. J. and Harzing, A. (2009). When Knowledge Wins: Transcending the Sense and

Aguillo, F., Bar-Ilan, J., Levene, M. and Ortega, J.L. (2010). Comparing university rankings.

Altbach, P.G. (2006). The Dilemmas of Ranking, *International Higher Education*, Vol.42, ISSN

Carey, K. (2006). Collage rankings reformed: the case for a new order in Higher Education. *Education Sector Reports*, No. 19, (September), Washington, DC, Education Sector

*Scientometrics*, Vol.85, No.1, (October), pp.12-16, ISSN 0138-9130

Nonsense of Academic Rankings. *Academy of Management Learning & Education*,


Table 5. Percentage of type of publication or format grouped by field in Scopus

in the Journal Citation Reports, are included the popular Impact Factor, the Index and Cited Immediacy Half Life and recently added the 5-year Impact Factor Eigen factor Score and Article Influence Score. Scopus delegates to various specialists the preparation of the Source Normalized Impact per Paper (SNIP), created by Professor Moed (leader team of CTWS at the University of Leiden) and SCImago Journal Rank (SJR), created by Professor Moya (leader team of Scimago CSIC group).
