**3. Inconsistencies in examining the impacts of ESG investments**

Although the subject of ESG investing has been examined and researched extensively, we have yet reached the definitive implications of this investment concept. Some studies find that ESG investing often leads to higher returns, while others view this issue differently and suggest that the benefits of ESG investing are being exaggerated. In this section, we will use the framework developed by Kotsantonis and Serafeim [59], which explores four dimensions of inconsistencies in ESG reporting to explain the contradictory findings of ESG investing discussed in the previous sections. Furthermore, we will use this framework to provide future research direction, which might help to reduce theses contradictions.

#### **3.1 Data inconsistency**

Kotsantonis and Serafeim [59] argue that there is a question of whether ESG data accurately capture a firm's performance and suggest that there are evidence of the relationship between economic outcomes and ESG information, even though the data are of poor quality. Generally, researchers examine the relationship between ESG information and firm performance assuming explicitly or implicitly that the firm performance relative to the ESG metric is normally distributed. Companies that perform well will be located toward the upper tail of the normal distribution, whereas the underperforming firms could be found toward the lower tail of the distribution. The first plausible inconsistency lies in the fact that the relationships between company performance and ESG matrices might not be normally distributed. This inconsistency often arises from the ways that companies report their ESG data, and generally these companies report their ESG data in various different ways. This, inevitably, creates problems for researchers when they examine the effect of ESG investment and corporate performance. Kotsantonis and Serafeim [59] demonstrate this problem by using examples of 50 large Fortune 500 companies from various sectors. To report the data on Employee Health and Safety in their sustainability report, these companies measured the data in more than 20 different ways, with different terminology and different units of measure. For example, while some companies use "Accident Rate" to describe Employee Health and Safety, others use "Accidents Requiring Time Off," "Injury Rate," "Lost Day rate," or "Financial Lost Due to Injury," among others, to measure Employee Health and Safety. Besides using different metrics to measure an ESG data (such as Employee Health and Safety), the units of measure are, many times, different. For example, some companies use the unit of measurement in term of "Number," while other companies use "Ratio" or "Percentage" instead. Kotsantonis and Serafeim [59] suggest that these inconsistencies make comparisons among companies to be rather challenging since it is unclear whether these metrics provide measurements of the same thing. The performance data on these ESG metrics might lie on different forms of distribution, rather than being normal, thus making comparison of these ESG data problematic.

To see this issue in a more concrete way, we will illustrate this issue using some studies that have been reviewed in the previous sections. While Kumar et al. [8] find that investors should benefit from ESG investing due to the evidence of lower risk and higher return for high ESG factors from their study; however, study by Landi and Sciarelli [25] documented otherwise. They find no significant relationship between ESG and abnormal returns. If we look closer in each of these two studies, one dimension that we can observe in regard to data consistency or the lack of it is the time periods used in these two studies. In Kumar et al. [8], the data cover a 2-year period, from January 2014 to December 2015. On the other hand, Landi and Sciarelli [25] examine the impact of ESG rating on abnormal return by using data from 2007 to 2015. Obviously, the data examined by Landi and Sciarelli [25] cover a much longer period. Looking closer, we can see that the economies were very much different between the 2007–2013 period and the 2014–2015 period. For instant, starting from the second half of 2007 to the middle of 2009, the world economies experienced the Great Recession observing significant declines in national economies worldwide.

Not only that, in the post Great Recession period, the speed of recoveries among these economies was very much different across regions.

As stated by Kotsantonis and Serafeim [59], another dimension of data inconsistency is the ways the data were measured. As in the case of ESG measurements, there are several different methods to compile ESG data. Zhao et al. [10] find that there is a positive relationship between financial and ESG performances for listed power generation companies in China. It is also interesting to note that, because of the nature of the energy industry, the ESG evaluation index requires a specific ESG assessment and evaluation system. In Zhao et al. [10], the ESG evaluation index is based on the model proposed by the OECD and the UNEP based on the business activities of the power generation group and social development. In addition, the company's ESG will also be normalized based on the actual and standard values comparison principle, whereas the actual values are reported by the power generation companies themselves, and the standard values are determined by the internationally recognized values and global averages with inputs from the experts in the field. Importantly, certain ESG indicators are also industry specific indicators. For example, "Power plant power consumption rate" and "Power supply standard coal consumption" are two indexes, among others, used to construct the index evaluation system and have "%" and "g/kWh" as the units of measurements. Furthermore, they use the non-dimensionalized model to determine the sustainability and evaluation indexes. The comprehensive evaluation index is then calculated as the weighted sum of the sustainability index of the evaluation index.

Constantinescu [28], on the other hand, finds that there is a negative relationship between firm value and the environmental factor in the energy sector. Constantinescu [28] uses the ESG scores obtained from the Refinitiv Eikon platform. According to Refinitiv, their ESG scoring methodology is based on a five-step process flow as follows: (1) ESG category scores, (2) Materiality matrix, (3) Overall ESG score calculation and pillar score, (4) Controversies scores calculation, and (5) ESG Combined (ESGC) score. Broadly speaking, we can see that the ways ESG scores are constructed in Zhao et al. [10] and in Constantinescu [28] are very much different in terms of how each data point is calculated and how the composite or combined ESG is determined. This represents an example of the data inconsistency issue in studies that investigate the relationship between firm performance and ESG information.

In order to remedy this issue, we recommend some future directions for future studies as follows. First, there should be studies that systematically incorporate these factors into considerations and try to identify findings that are robust to different types of measurements. Second, future studies might consider statistically constructing an integrated index from all different measures so that there could be a single consistent measure to capture an ESG performance. Quantitative approaches such principal component analysis may also help to address the data inconsistency issue.

#### **3.2 Inconsistencies in benchmarking**

Another source of data inconsistencies is the use of benchmarking. Kotsantonis and Serafeim [59] point out that a crucial point when comparing ESG information among companies, i.e., which companies have better or worse ESG performance, is to look at the definition of performance that leads to the benchmark for the ESG score comparison. As such, in order to determine the company's (ESG) performance, studies often need to look at the ESG performance of a peer group or by using a predefined

#### *Perspective Chapter: The Environmental, Social, and Governance (ESG) Investment and Its… DOI: http://dx.doi.org/10.5772/intechopen.108381*

level of performance on ESG metrices. Either one of these benchmarking methods could lead to the results that are subjected to the data inconsistency issues.

To overcome this issue, it is important to note that the choice of the ESG scoring systems or peer groups has influences that affect the interpretation of the assessment [59]. We propose that future studies may consider employing methodologies such as machine learning to construct ESG scores with appropriate peer rating from unstructured corporate data. These methods would allow the analysis to be performed systematically by machine at the raw data level, which can reduce human bias and error. On the other hand, focusing on classification or clustering methods such as K-mean clustering should yield results that are robust and are comparable across market environments. For example, Margot et al. [60] demonstrated that using machine learning to classify ESG funds into appropriate groups plays an important role in affecting the return of the constructed portfolio. They found that the machine learning constructed portfolio outperformed the best-in-class approach and the benchmark approach.

#### **3.3 Methods used to impute the data**

It is possible that the "raw" data used to construct a certain element of ESG are missing. According to Kotsantonis and Serafeim [59], 50% of top 50 companies in the fortune 500 did not provide information about their health and safety policy. For another related category, 85% of top 50 companies in the fortune 500 did not provide information about their "lost time incident rates and workplace fatalities." However, the final information from various ESG data provider seems to be complete after aggregation and rarely mention about the issue associated with missing observation. Kotsantonis and Serafeim [59] argued that "two different imputed figures can deliver significantly different ESG performance ratings," and inappropriately impute missing data can lead to bias in the associated statistical analyses.

Broadly speaking, missing observations can arise from three types. First, values in a data are "missing completely at random" or MCAR. Little [61] defined MCAR as the missing values in the data that are independent of both observable and unobservable factors. One simply way to understand MCAR is when missing values in the data are generated by a random process. Second, values in the data are "missing at random" or MAR. Little [61] defined MAR as the missing values in the data that are dependent on observable factors but are independent on unobservable factors. One simply way to understand MAR is when missing values in the data are generated by a known process. A key difference between MCAR and MAR is that for MCAR, all analyses (e.g., examining the relationship between ESG performance on some measures of firms' risks or returns) can be performed without any adjustment, and the results will be unbiased. On the other hand, for the case of MAR, all analyses must use some statistical procedures to take into an account the "known" missing process that depends on observable factors. Otherwise, the results from the analyses can be biased. Finally, the last type is "missing not at random" or MNAR. For MNAR, the missing values in the data are dependent on both observable and unobservable factors. In the case of MNAR, all analyses will be biased.

Taking the example from Kotsantonis and Serafeim [59], they showed that using different imputation methods that consist of rule-based approaches (e.g., using average value of an industry) and statistical-based approach (e.g., using regression analysis, predictive mean matching, single imputation, or multiple imputation) can lead to a difference from a real value that can be as large as 8.9%. Thus, missing data in ESG reporting can be a serious issue not only to the investors who need to know reliable information about ESG scores but also the researchers who need to examine and uncover the relationship between ESG and various measures of returns and risks. We suggest that future studies examining the relationship between ESG and various measures of returns and risks need to incorporate the missing observation issue and use proper statistical methods to control for the "known" nature of gap filling in the ESG data. We also suggest that the ESG data providers must be transparent in reporting how they actually impute the ESG data. Another possibility is that the ESG providers can report two different sets of the data in which one set of data is the "raw data" that does not involve any imputation, which will be suitable for researchers who need to examine the relationship between ESG and various measure of returns and risks, and the second set of data is the "complete data" that does involve imputation, which will be suitable for investors for making investment decision.
