Preface

"Why Most Published Research Findings Are False"


The recalcitrant and alarming title of Ioannidis' research article from 2005 (quoted above) addressed a growing concern that the current practice of statistics to a significant extent could be flawed. In his book Willful Ignorance: The Mismeasure of Uncertainty (2009), Herbert Weisberg pointed out the nearly abyssal difference in perspectives of quantitative and qualitative studies and reflected on the historical evolution of statistics, displeased by the fact that practical scientists nowadays systematically avoid discussing the meaning of probability, despite its being intensively debated for over two centuries. These claims of doubt were accompanied by Alex Reinhard, who in his book Statistics Done Wrong: The Woefully Complete Guide (2015) discussed a number of questionable ways statistical analyses of today are performed.

The famous statistician R.A. Fisher stated in the early 20th century that: "The tendency of modern scientific teaching is to neglect the great books, to lay far too much stress upon relatively unimportant modern work, and to present masses of detail of doubtful truth and questionable weight in such a way as to obscure principles." In the modern society flooded not only by massive unleashed computational power but also by data-based artificial intelligence and unprecedented means to craft stunning graphics, it is not at all surprising that people at large are struggling to distinguish facts from fake. Fisher's concern from the 1930s suddenly appears more alive than ever before. The need to re-evaluate statistical practices thus steadily seem to grow the more they are utilized.

These observations suggest current statistical methodologies may have gone too far into an engineering practice with minimal questioning. As an example, multiple tests are often launched in scientific studies to enhance the likelihood of finding a significant result, although this should be impossible …

In the modern age pervaded by data in various forms, statistical processing affects almost everyone, whether we like it or not. Internet-based analyses operate on sample sets almost as big as the populations they are drawn from. Behavioral, political, and customer surveys are frequently made. Despite this, the basic principles and practices have remained remarkably intact over the years. In stark contrast stands the rapid evolution of computer programming methodologies and development of convenient, flexible, high-level, and broadly accessible software tools for statistical analysis and presentation, like Python, R, Matlab/Octave, etc. To fully utilize their potential rather than be distracted and paralyzed by their breathtaking performance, good statistical methodologies and practices are of greater value than ever before. This context provided the basic motivation for this book.

A selection of various aspects of statistical methodologies are here presented by independent authors. The chapters are not meant to be exhaustive or representative, even though each contribution is self-contained and complete within the task addressed. The texts are primarily meant to motivate further reading. Being overviews, many details are left out. The reader is therefore advised to make extended use of the list of references presented at the end of each chapter.

The introductory chapter discusses possible consequences of our willful ignorance, or what is often assumed but seldom known. The main part of the book starts with methods to analyze sensitive questions and how reliable unbiased claims can be extracted. Two-phased stratified sampling is then studied for various estimators. To facilitate search of prior art when authoring and examining patents, a methodology based on machine learning for analyzing and structuring patents is discussed thereafter. Density estimation in inventory control systems, a comparison of maximum likelihood and Bayesian estimation for the Erlang distribution follows, before the asymptotic normality of Hill's estimator is addressed. The book concludes with a study of the Jackknife resampling method.

Current practice has indeed been questioned numerous times in recent years, even though mathematical statistics for a very long time has been widely applied in medicine, social studies, science, and technology. Hopefully, this book will provide guidance into some aspects of statistical data collection and processing for the future.

> Jan Peter Hessling Kapernicus AB, Gothenburg, Sweden

Chapter 1

Knowledge

Jan Peter Hessling

1. Background

and proper interpretation.

scope of this book.

2. Ambiguity

1

Introductory Chapter:

"Facts do not cease to exist because they are ignored."

Mathematical statistics has long been widely practiced in many fields of science [1]. Nevertheless, statistical methods have remained remarkably intact ever since the pioneering work [2] of R.A. Fisher and his contemporary scientists early in the twentieth century. Recently however, it has been claimed that most scientific results are wrong [3], due to malpractice of statistical methods. Errors of that kind are not caused by imperfect methodology but rather, reflect lack of understanding

In this introductory chapter, a different cause of errors is addressed—the ubiquitous practice of willful ignorance (WI) [4]. Usually it is applied with intent to remedy lack of knowledge and simplify or merely enable application of established statistical methods. Virtually all statistical approaches require complete statistical knowledge at some stage. In practice though, that can hardly ever be established. For instance, Bayes estimation relies upon prior knowledge. Any equal a priori probability assumption ("uninformed prior") does hardly disguise some facts are not known, which may be grossly deceiving. Uniform distribution is a specific assumption like any other. Willful ignorance of that kind must not be confused with knowledge to which we associate some degree of confidence. It may be better to explore rather than ignore consequences of what is not known at all. That will require novel perspectives on how mathematical statistics is practiced, which is the

Incomplete knowledge implies that obtained results may not be unique. That is, results may be ambiguous. Ambiguity de facto means the uncertainty associated with any estimated quantity itself is uncertain. We may adopt a probabilistic view and classify ambiguity as epistemic uncertainty. Ambiguity will here refer to lack of knowledge typically substituted with willful ignorance. Alternatives propelled by different types of willful ignorance can thus be explored to assess ambiguity.

A most powerful source of ambiguity is dependencies. Independence is perhaps

the most claimed but often the least discussed presumption. Throwing dices or growing crops, as typically studied by the founders of statistics, independence indeed seems plausible. In all the complexity of modern technology of today however, it is anything but evident observations are independent. For instance,

Ramifications of Incomplete

#### Chapter 1
