**Assay Validation in High Throughput Screening – from Concept to Application**

Sergio C. Chai, Asli N. Goktug and Taosheng Chen

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/59765

## **1. Introduction**

High throughput screening (HTS) has evolved as an indispensable aspect of drug discovery, whereby small organic molecules are identified as potential therapeutics or as probes to aid in the understanding of biological processes. Large screening endeavors tend to encompass massive collections of compounds, which may exhaust resources in terms of time, effort and reagent. As such, it is imperative to firmly validate any assay prior to implementation in HTS. The validation process should be able to discern the robustness and reliability of the assay. In this chapter, we intend to discuss statistical and data visualization approaches as well as factors to consider for predicting a successful HTS campaign.

#### **1.1. HTS assay validation**

In the past several decades, HTS has evolved to such an extent that a considerable array of assay types can be successfully implemented [1, 2]. The wide spectrum of biochemical assays includes enzymatic reactions, and interaction studies among proteins, peptides, oligonucleo‐ tides and ligands. With the rapid progression of cellular engineering, cell-based assays have become predominant in HTS facilities, with most of the assays performed to detect cytotoxicity, reporter gene activity, or other phenotypic changes in response to compound treatment [3-5].

Substantial effort and time can go into assay development, which may not be necessarily amenable for HTS. Often times, assays need to be adapted to smaller volumes in a process known as assay miniaturization, whereby experiments are conducted in microtiter plates of high density, typically in a 96-, 384- or 1536-well format [6]. The driving force behind minia‐ turization is the generation of large data sets in a fast and efficient manner, while reducing reagent consumption and space. Moreover, the experimental protocol might have to be modified to accommodate instrument availability, automation capabilities, reagent stability,

© 2015 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

and screening duration among others. Therefore, assays designed in a basic research labora‐ tory may not necessarily perform to the same standards as those in HTS.

Assay validation ensures that assays meet certain criteria for effective execution during HTS campaigns. Even though assay validation is not universally applied, it is a crucial step that should not be overlooked. The advantages and benefits include:


#### **1.2. Instrumentation in HTS**

Specialized instrumentation is needed to run HTS, many of which have direct and significant impact on the quality of the data. The most basic setup includes various liquid handling devices and plate readers, all of which being potential sources for poor assay performance if not appropriately maintained and utilized. In addition, peripheral components such as tempera‐ ture-controlled incubators can have a detrimental effect on assay quality. Therefore, it is imperative to consider these technologies during assay validation.

One of the most commonly used liquid handlers is the bulk liquid dispenser, which is generally utilized to deliver single assay component at a time, quickly and uniformly across the wells on a plate [7]. Compound delivery in smaller volumes is accomplished using transfer devices, which is an umbrella term for instrumentation that encompasses a wide range of technologies and capabilities.

The plate reader is a specialized device capable of acquiring spectroscopic signal relatively fast with minimal user intervention. The raw data can be conveniently created in a matrix format following the layout of the type of microtiter plates utilized, which facilitates subsequent data analysis. Several typical detection modes provide the scientist with vast flexibility in assay development, including absorbance, fluorescence intensity, fluorescence polarization/ anisotropy (FP/FA), fluorescence resonance energy transfer (FRET), time-resolved FRET (TR-FRET) and luminescence. Even though endpoint detection (*i.e*. a single value per well) is commonly employed, wavelength scans (in the case of absorbance and fluorescence intensity) or kinetic analysis are possible. Because of increased sensitivity and robustness, filter-based detectors are preferable to devices employing monochromators, the latter one needed when wavelength scanning is required.

The incubator is an integral part of fully-automated robotic decks, where it is being used to store microtiter plates, pipette tips and other labware. Incubators with capabilities for con‐ trolling temperature, humidity, carbon dioxide and nitrogen content are particularly prone to cause artifacts or affect negatively the assay signal. This is largely observed in assays where the signal increases over time, such as cell-growth, reporter-based and enzymatic assays.

## **2. Protocol of a typical assay validation**

and screening duration among others. Therefore, assays designed in a basic research labora‐

Assay validation ensures that assays meet certain criteria for effective execution during HTS campaigns. Even though assay validation is not universally applied, it is a crucial step that

**•** A priori knowledge of an inadequate assay can reduce the chances of a failed HTS endeavor,

**•** The information generated during assay validation can be valuable for assay improvement. **•** The assay validation report provides confidence in the assay and serves as a standard for further studies, particularly when working with teams of various scientific backgrounds. **•** When dealing with different reagent batches throughout the venture, assay validations

Specialized instrumentation is needed to run HTS, many of which have direct and significant impact on the quality of the data. The most basic setup includes various liquid handling devices and plate readers, all of which being potential sources for poor assay performance if not appropriately maintained and utilized. In addition, peripheral components such as tempera‐ ture-controlled incubators can have a detrimental effect on assay quality. Therefore, it is

One of the most commonly used liquid handlers is the bulk liquid dispenser, which is generally utilized to deliver single assay component at a time, quickly and uniformly across the wells on a plate [7]. Compound delivery in smaller volumes is accomplished using transfer devices, which is an umbrella term for instrumentation that encompasses a wide range of technologies

The plate reader is a specialized device capable of acquiring spectroscopic signal relatively fast with minimal user intervention. The raw data can be conveniently created in a matrix format following the layout of the type of microtiter plates utilized, which facilitates subsequent data analysis. Several typical detection modes provide the scientist with vast flexibility in assay development, including absorbance, fluorescence intensity, fluorescence polarization/ anisotropy (FP/FA), fluorescence resonance energy transfer (FRET), time-resolved FRET (TR-FRET) and luminescence. Even though endpoint detection (*i.e*. a single value per well) is commonly employed, wavelength scans (in the case of absorbance and fluorescence intensity) or kinetic analysis are possible. Because of increased sensitivity and robustness, filter-based detectors are preferable to devices employing monochromators, the latter one needed when

The incubator is an integral part of fully-automated robotic decks, where it is being used to store microtiter plates, pipette tips and other labware. Incubators with capabilities for con‐ trolling temperature, humidity, carbon dioxide and nitrogen content are particularly prone to cause artifacts or affect negatively the assay signal. This is largely observed in assays where the signal increases over time, such as cell-growth, reporter-based and enzymatic assays.

tory may not necessarily perform to the same standards as those in HTS.

which could signify tremendous waste of resources, time and effort.

serve as an important quality control checkpoint to ensure consistency.

imperative to consider these technologies during assay validation.

should not be overlooked. The advantages and benefits include:

222 Drug Discovery and Development - From Molecules to Medicine

**1.2. Instrumentation in HTS**

and capabilities.

wavelength scanning is required.

A typical assay validation process consists of multiple major components such as repeating the assay of interest on multiple days with the proper experimental controls, verifying the optimum assay conditions using the high throughput instruments in subject and exploring the overall assay quality with various statistical metrics and visualization tools. A reference commonly-used by many HTS facilities is the HTS Assay Validation guidelines provided in the Assay Guidance Manual by Eli Lilly & Company and the National Center for Advancing Translational Sciences [8]. Here, we review the contents of a typical assay validation report and highlight the significance of this information for a successful assay performance in high throughput format.

As with any other scientific experiments, maintaining detailed documentation of the assay validation experiments is of high importance to achieve maximal reproducibility in the proceeding screening campaign. For this purpose, generating a validation report for each project in a standard format, such as in a spreadsheet format with multiple tabs, is an adequate way to organize and store project-specific information as well as the numerical data and its analysis in a single file. The report generally consists of the following sections.


The assay validation experiments are conducted on three different days with three individual plates processed on each day (Figure 1). Each plate set contains three layouts of samples that mimic the highest, medium and lowest assay readouts (hereafter defined as "high", "medium" and "low" signals), while ideally retaining biological relevance. The "high" and "low" signal samples are typically chosen to be the positive and negative controls, so that the upper and lower boundaries of the assay readout (*i.e.* signal window) are assessed as part of the validation process. In an inhibition or antagonistic assay ("signal-decreasing" assay), the "low" signal would correspond to the response by the positive control. In contrast, the "high" signal would represent that of the positive control in experiments involving activators or agonists ("signalincreasing" assay).

On the other hand, the "medium" signal sample needs to lie between the controls in the activity plot and possesses a crucial role to determine the capacity of the assay to capture "hit" compounds during the screen. The "medium" signal is often obtained from a sample at a concentration that results in the EC50 of the positive control compound. Additionally, the "high", "medium" and "low" signal samples are distributed within plates in an interleaved fashion, such that each of the three plates processed on each given day contains samples in different column-wise order: "high-medium-low" (plate 1), "low-high-medium" (plate 2) and "medium-low-high" (plate 3). In order to capture the full characteristics of the assay, it is important to prepare fresh set of samples on each day and to avoid introducing any additional variables between experiments.

PLEASE REPLACE FIGURE **1** WITH THE FOLLOWING (Replacement Figure 1):

**Figure 1.** Standard assay validation protocol

With the above-mentioned procedure, three major aspects are aimed to be addressed at the end of the validation process. While the statistical analyses of the "high", "medium" and "low" signals provide substantial information about the overall magnitude and tightness of the assay control data, they also give an idea of the potential plate-to-plate and day-to-day variations. The interleaved plate setup substantially helps to capture any positional effects that may be caused by the incubation conditions such as edge effects, or by other systematic factors such as drift.

To perform the quantitative assessment of the assay quality, several statistical tests are conducted on the validation data. Primarily, Z'-factor is a well-accepted dimensionless parameter used to calculate the signal separation between highest and lowest assay readouts taking the signal means and standard deviations (stdev) into account under the assumption of normality [9]. Likewise, signal window is computed as another metric of the range of

2

controls. While the Z'-factor and signal window are the most common parameters for overall assay quality measurements, the "high", "medium" and "low" signal trends can also be captured by their respective signal averages, standard deviations and the coefficient of variations (CV).

$$\mathbf{Z}^\*\text{-factor} = \mathbf{1} \cdot \frac{\mathbf{3} \times (\text{stdev}\{\mathbf{S}\_{\text{high}}\} \bullet \text{stdev}\{\mathbf{S}\_{\text{low}}\}) \mathsf{of}\sqrt{n}}{\text{mean}\{\mathbf{S}\_{\text{high}}\} \cdot \text{mean}\{\mathbf{S}\_{\text{low}}\}} \tag{1}$$

$$\text{Signal Window} = \frac{(\text{mean}(\text{S}\_{\text{high}}) \cdot \text{mean}(\text{S}\_{\text{low}})) \cdot 3 \times (\text{stdev}(\text{S}\_{\text{high}}) \cdot \text{stdev}(\text{S}\_{\text{low}})) \cdot \sqrt{n}}{\sqrt{n}} \tag{2}$$

$$\text{CV} = \frac{\text{stdev(S)} / \sqrt{n}}{\text{mean(S)}} \tag{3}$$

where, n is the number of replicates of the test compounds in the actual screen.

Besides passing the visual examination for any systematic errors, the data collected from the 3-day validation experiments needs to quantitatively meet the minimum quality requirements, which are evidently set in the HTS Assay Validation guidelines [8]. According to these criteria, the CV values of the raw "high", "medium" and "low" signals are required to be less than 20% in all nine plates. If the "low" signal fails to meet the CV criteria in any of the plates, then the standard deviation of the "low" signal has to be less than the standard deviations of the "high" and "medium" signals within that plate. Also, the standard deviation of the normalized (percent activity) "medium" signal is supposed to be less than 20 in plate-wise calculations. By nature, Z'-factor is a parameter that ranges between 0 and 1, with 1 indicating a perfect assay. For validation purposes, achieving a Z'-factor of greater than 0.4 or a signal window greater than 2 in all plates is considered acceptable.

$$\text{Percent activity} = \frac{\text{S}\_{\text{i}} \cdot \text{mean}(\text{S}\_{\text{low}})}{\text{mean}(\text{S}\_{\text{high}}) \cdot \text{mean}(\text{S}\_{\text{low}})} \times 100\tag{4}$$

#### **3. Plot interpretation**

2

would correspond to the response by the positive control. In contrast, the "high" signal would represent that of the positive control in experiments involving activators or agonists ("signal-

On the other hand, the "medium" signal sample needs to lie between the controls in the activity plot and possesses a crucial role to determine the capacity of the assay to capture "hit" compounds during the screen. The "medium" signal is often obtained from a sample at a concentration that results in the EC50 of the positive control compound. Additionally, the "high", "medium" and "low" signal samples are distributed within plates in an interleaved fashion, such that each of the three plates processed on each given day contains samples in different column-wise order: "high-medium-low" (plate 1), "low-high-medium" (plate 2) and "medium-low-high" (plate 3). In order to capture the full characteristics of the assay, it is important to prepare fresh set of samples on each day and to avoid introducing any additional

PLEASE REPLACE FIGURE **1** WITH THE FOLLOWING (Replacement Figure 1):

With the above-mentioned procedure, three major aspects are aimed to be addressed at the end of the validation process. While the statistical analyses of the "high", "medium" and "low" signals provide substantial information about the overall magnitude and tightness of the assay control data, they also give an idea of the potential plate-to-plate and day-to-day variations. The interleaved plate setup substantially helps to capture any positional effects that may be caused by the incubation conditions such as edge effects, or by other systematic factors such

To perform the quantitative assessment of the assay quality, several statistical tests are conducted on the validation data. Primarily, Z'-factor is a well-accepted dimensionless parameter used to calculate the signal separation between highest and lowest assay readouts taking the signal means and standard deviations (stdev) into account under the assumption of normality [9]. Likewise, signal window is computed as another metric of the range of

increasing" assay).

variables between experiments.

224 Drug Discovery and Development - From Molecules to Medicine

**Figure 1.** Standard assay validation protocol

as drift.

A very useful and informative way to monitor assay quality is to utilize scatter plots. The left charts in Figures 2 and 3 are such examples (using simulated data), with each plot corre‐ sponding to a single 384-well plate, where the values are extracted in a row-wise fashion to generate the order of the data points in the plot (*i.e*. from well position A1 to A24 followed by B1 to B24, and so on). In this particular case, the "high", "medium" and "low" signals are represented in green, pink and blue, respectively. The charts on the right side are the combined representations of all 9 assay plates in a typical assay validation, grouped in sets of 3 plates in 3 different days (vertical gridlinesdenote data from individual 384-well plates): for each of the 3 types of signals (i.e. "high", "medium" or "low"), the values were extracted in a columnwise manner.

When troubleshooting, these plots can provide valuable information that can lead to the cause of the problem. Patterns created by these data points can be normally ascribed to specific causes. The most typical ones are:


to be screened. In addition, compound build-up at the pins due to inappropriate washing can affect the signal quality of the controls over time (as discussed in greater length below in sections 4. and 5.).

PLEASE REPLACE FIGURE **2** WITH THE FOLLOWING (Replacement Figure 2):

3 types of signals (i.e. "high", "medium" or "low"), the values were extracted in a column-

When troubleshooting, these plots can provide valuable information that can lead to the cause of the problem. Patterns created by these data points can be normally ascribed to specific

**•** Figure 2A: This is one of the most common cases of edge-effect, where the outer rows of the assay plate display the highest signal, with decreasing intensity moving towards the center rows. This is due to temperature gradients, where the biological activity (*e*.*g*. enzymatic activity or rapid growth of bacteria expressing reporter genes, typically in assays with shorter incubation time) is higher at the more elevated temperature of the outer wells. The bowl-shaped effect can be upside down, in which case evaporation in the outer wells due to prolonged incubations lowers the intensity, being commonly seen in assays involving

**•** Figure 2B: Edge-effect with a linear profile, with the highest intensity observed in the last

**•** Figure 2C: Another edge-effect profile, with the intensity values decreasing from the first

**•** Figure 3A: The lack of consistency in reagent preparation for each run leads to variations in signal intensity for the cumulative data points. In this specific example, there is uniformity in all three assay plates for a given run, but inconsistencies are observed among experiments performed in the three separate days. Batch to batch variation is typical when dealing with cell-based assays, particularly with transient transfection. Enzymatic assays with substrate concentrations at or below the Km are markedly sensitive to small concentration changes. Reagent instability during long-term storage can also lead to signal heterogeneity. These issues might not be of great concern if the signal window is adequate and the signal is normalized to internal controls ("in-plate controls"). It is therefore important to include such

**•** Figure 3B: Pintool devices are commonly found in screening facilities, which are utilized to transfer compounds from a compound stock plate to the assay plate. It is a contact-based liquid handler, requiring direct physical contact with the fluid to be transferred. As such, it is susceptible to cross-contamination if compounds adsorb avidly at the pins, or if the washing steps between transfers are not adequate. The assay validation protocol described in this chapter makes use of compounds arranged in three different layouts, so pins exposed to compounds in the first plate (corresponding to columns with "low" or "medium" signals in an inhibition/antagonistic assay) can release the chemicals in the second or even third plate, affecting mostly the "high" signal corresponding to columns with negative control (dimethyl sulfoxide, DMSO). The opposite effect would be observed in experiments with agonistic mode. During screening, normally the same well positions for all plates are allocated for control compounds, so carry-over in this instance is not relevant. However, it is an indication that the pin washing protocol may not be appropriate for the compounds

**•** Figure 2D: Linear edge-effect, with the highest signal observed in the last column.

slow-growing cell (typically in assays with extended incubation time).

wise manner.

row.

column to the last.

internal controls in each plate.

causes. The most typical ones are:

226 Drug Discovery and Development - From Molecules to Medicine

**Figure 2.** Contribution of edge-effects to the signal pattern commonly encountered during assay validation.

3

**Figure 3.** Signal profiles potentially encountered during assay validation.

4 **•** Figure 3C: Bulk-reagent dispensers are one of the most widely used liquid handlers for screening. In between dispensing, the reagents are usually kept in the tubings. For long delays between plate dispensing (as in a complex automation protocol), heterogeneous components in the reagent can settle in the tubings, creating gradient zones of material, which is then pushed to the plate in the next dispensing cycle. The pattern observed in the plot might not be evident without considering that in a 384-well plate, an 8-channel liquid handling cassette dispenses fluid every other row to be later offset by a row in the return dispensing steps. In addition, the tubing length and the way it is tangled affect the shape of the pattern. To prevent this, one can use the "empty/prime" mode, whereby the solution is reverted to the reagent source after plate dispensing. An alternative is to use prolonged priming of the solutions between dispensing in order to discard existing reagent in the tubings.

**•** Figure 3D: The background signal increases over time due to degradation of the substrate in an enzymatic assay from the spectroscopically silent species to the active form. This is a typical example of autohydrolysis of a fluorogenic substrate, whereby the quenching moiety in the molecule is released, leading to buildup of the fluorescent label. This issue becomes problematic at larger-scale screening projects that require prolonged screening times. However, modest upward shifts in the background signal have no major impact due to normalization with internal controls, further demonstrating the importance of including such internal controls in each plate.

## **4. Considerations**

PLEASE REPLACE FIGURE **3** WITH THE FOLLOWING (Replacement Figure 3):

228 Drug Discovery and Development - From Molecules to Medicine

**Figure 3.** Signal profiles potentially encountered during assay validation.

**•** Figure 3C: Bulk-reagent dispensers are one of the most widely used liquid handlers for screening. In between dispensing, the reagents are usually kept in the tubings. For long delays between plate dispensing (as in a complex automation protocol), heterogeneous components in the reagent can settle in the tubings, creating gradient zones of material,

There are several important factors to consider during assay validation for a successful screening campaign. The following points can be used to uncover or solve setbacks during the validation process.

#### **4.1. Biological reagents**

#### *4.1.1. Reagent consistency*

4

It is preferable to use the same batch of a particular reagent during assay validation and the entire screening campaign in order to reduce variations that result from (Figure 3A):


#### *4.1.2. Reagent stability*

An extremely important point to consider in validating an assay is the stability of each reagent that comprises the assay, which could alter screening strategies considerably. Besides the inevitable inherent chemical or biological properties that hinder long-term stability (Figure 3D), the most common reasons to consider are contamination (*e.g*. mycoplasma in mammalian cells, proteases in purified proteins) and freeze/thaw effects.

#### *4.1.3. Sequence of reagent addition*

The sequence in which reagents are combined in the assay plate could influence significantly the degree of inhibition or activation by compounds to be screened or used as positive controls, which may be reflected during the validation procedure. Taking as an example a biochemical binding assay such as TR-FRET or FP (Table 1), the outcome for each of the five cases could be different if there is no sufficient equilibration (incubation) time. In such conditions, cases 1 and 4 would display a more pronounced inhibition/activation value. In some cases, there could be differences between dispensing of the compound into an empty plate (cases 4 and 5) and dispensing the compound to an aqueous solution (cases 1-3), especially when the compound is being transferred from a DMSO stock.

The choice of the order of reagent addition is often limited by the availability of instrumenta‐ tion or convenience in automation. Some devices allow for accurate compound transfer to dry surfaces (*e.g*. acoustic-based devices), while others require a wet surface (pintool systems). When using automation decks with only a single bulk-dispenser, it is normal to dispense a pre-mixed solution of protein and probe (cases 2, 4 and 5).


**Table 1.** Possible ways of combining reagents in a biochemical binding assay for HTS.

#### **4.2. Consumable composition**

Wide ranges of consumables are used in almost every step of an HTS assay. The selection of these consumable materials is highly dependent on the instrument types that will be employed to perform the assay and other budgetary constraints. Microtiter plates, disposable tips, dispenser tubing cartridges and cell culture tools are some examples of consumables that come in direct contact with the assay reagents. Therefore, these products need to be selected with caution and kept consistent throughout the course of the assay validation and the screening endeavor to achieve maximum data reproducibility.

#### *4.2.1. Consumable batch/lot variations*

*4.1.2. Reagent stability*

*4.1.3. Sequence of reagent addition*

is being transferred from a DMSO stock.

**4.2. Consumable composition**

An extremely important point to consider in validating an assay is the stability of each reagent that comprises the assay, which could alter screening strategies considerably. Besides the inevitable inherent chemical or biological properties that hinder long-term stability (Figure 3D), the most common reasons to consider are contamination (*e.g*. mycoplasma in mammalian

The sequence in which reagents are combined in the assay plate could influence significantly the degree of inhibition or activation by compounds to be screened or used as positive controls, which may be reflected during the validation procedure. Taking as an example a biochemical binding assay such as TR-FRET or FP (Table 1), the outcome for each of the five cases could be different if there is no sufficient equilibration (incubation) time. In such conditions, cases 1 and 4 would display a more pronounced inhibition/activation value. In some cases, there could be differences between dispensing of the compound into an empty plate (cases 4 and 5) and dispensing the compound to an aqueous solution (cases 1-3), especially when the compound

The choice of the order of reagent addition is often limited by the availability of instrumenta‐ tion or convenience in automation. Some devices allow for accurate compound transfer to dry surfaces (*e.g*. acoustic-based devices), while others require a wet surface (pintool systems). When using automation decks with only a single bulk-dispenser, it is normal to dispense a

Wide ranges of consumables are used in almost every step of an HTS assay. The selection of these consumable materials is highly dependent on the instrument types that will be employed to perform the assay and other budgetary constraints. Microtiter plates, disposable tips, dispenser tubing cartridges and cell culture tools are some examples of consumables that come in direct contact with the assay reagents. Therefore, these products need to be selected with caution and kept consistent throughout the course of the assay validation and the screening

**Case Sequence of reagent addition** Protein → compound → probe Protein → probe → compound Probe → compound → protein Compound → protein → probe Compound → probe → protein

cells, proteases in purified proteins) and freeze/thaw effects.

230 Drug Discovery and Development - From Molecules to Medicine

pre-mixed solution of protein and probe (cases 2, 4 and 5).

**Table 1.** Possible ways of combining reagents in a biochemical binding assay for HTS.

endeavor to achieve maximum data reproducibility.

The batch of the consumable items is an important factor that plays a crucial role in the final data quality. Proper records of the batch information of all the reagents and the consumable products used for an assay should be documented as part of the assay validation report. For instance, the screener may experience unexpected intermittent drift effects if varied batches of plates are used on different days of a screening project. Material drift effects may be caused by a manufacturing fault in a particular plate batch. The disruption of the overall assay quality due to drift can be caught during the 3-day validation experiments and be avoided in the proceeding screening runs so long as the single batch of plates, which passed the validation criteria, are consistently used.

#### *4.2.2. Leaching and adsorption behavior of consumables*

One of the main concerns with the plastic consumable materials used in liquid handling tasks is the leaching behavior of the plastic labware. It is known that chemical impurities leaching out of the plastic disposable tips, reagent reservoirs and assay/compound storage plates may interfere with the biological reagents resulting in deceptive assay readouts [10-12]. The leachates are mostly released from the plastic labware due to the exposure of the plastic surfaces to strong solvents, such as DMSO. If this behavior is providentially detected at the assay validation phase, alternative solutions may be conceived to pursue the assay with minimal interruption. On the other hand, cell culture media loaded with proteins and other essential nutrients tend to react with certain plastic reservoir materials leading to formation of contaminants affecting the overall health of the cells in subject. One way to test this phenomenon would be to store the cell suspension in the reservoir for the anticipated screening time, after which the final solution should be visually and quantitatively examined for any unpredicted properties.

Consumable labware is something to be vigilant about when used in HTS assays because of its potential for absorbing the test chemicals or other assay reagents. In a study by Palmgren *et al*., it was shown that negatively charged polystyrene surfaces absorb lipophilic and positively charged basic drug compounds leading to unreliable final drug concentrations, although this effect may be negligible at high drug concentrations and proper buffer conditions [13]. In another study, it was demonstrated that the polypropylene tips used by automated liquid handling instruments adsorb certain compounds, therefore, increased contaminant concentrations are observed with lower transfer volumes [14]. It was also mentioned that the discrepancy in the transfer volume is enhanced if the polypropylene tips are used for serial dilution experiments with aqueous diluents.

In biochemical assays, non-specific binding of the enzyme to the plastic components (namely dispense tubing cartridges) of the high throughput bulk dispensing instruments is a concern‐ ing phenomenon (Figure 3C). Especially because the consequences of the binding effect may only become obvious when dispensing large number of plates; the issue may be easily overlooked in the assay validation. One can avoid sticking of the enzyme to the plastic tubing elements by coating their surfaces with blocking agents, such as protein blockers or non-ionic detergents [7]. Alternatively, the blocking agents may be added to the assay buffer as long as they do not interfere with other assay components.

#### **4.3. Plate type selection**

The selection of the appropriate assay plate type is important and mainly depends on the assay detection method. The light reflecting properties of the assay plate surfaces profoundly affect the final signal intensities, background noise levels and well-to-well crosstalk. Black, solid bottom, opaque-walled plates are recommended for fluorescence-based reading technologies to achieve lower background signal and minimal crosstalk, while white plates are good for luminescence signal detection to enhance light output. On the other hand, clear-bottom plates are needed for colorimetric assays, as well as for cell-based assays, where the cells need to be monitored by microscopy throughout the course of the experiment.

Despite these general selection guidelines, a suitable assay plate type should be carefully chosen in compliant with the overall project goals. For instance, in a luminescence assay with low signal window and relatively high assay volume/well, where the "hit" compound is defined as the test sample that causes a drop in the signal intensity in comparison to the negative control, detection of the "hits" may be impaired if white plates are used. That is because a well containing the active compound with low signal intensity would be surrounded by several inactive wells with high signal intensities, and the crosstalk from the surrounding wells would greatly alter the original signal magnitude in the active well leading to increased false negative rates. In such luminescence assays, where the scientist is aiming to detect a signal decrease, black plates would be more appropriate to conduct the experiment.

High content assays commonly require specially designed microtiter plates to attain the maximal scan performances, when high content imagers are used. These plates are generally intended to have optically clear, very thin and uniform well bottoms to ensure high quality images. Additionally, the assays that require fixation and staining processes, and involve multiple washing steps may necessitate plates that enhance cell retention. For this purpose, plates with poly-D-lysine, poly-L-lysine and collagen-coated surfaces are available to promote cell adhesion and growth.

Many more types of plates are offered for different assay methodologies, such as low attach‐ ment plates for cell-based assays using cells in suspension, and non-specific binding surface plates for protein-binding experiments. Besides, selection of the correct plate type for multiplex assays may require extra effort and testing process, especially if luminescence and fluorescence signals are being measured within the same plate. Performing a detailed search of the available plate options for the assay of interest is a time worthy practice, which would eventually save the scientist from developing and validating the assay repeatedly.

#### **4.4. Incubation conditions**

In high throughput practices, a large number of plates are processed on a daily basis, and the environment in which the plates are stored during the screening runs exerts high impact on the assay data quality. Automation compatible incubators are manufactured to optimize storage conditions for high-density plates while maximizing space utilization. Incubators with wide range of storage capacities and temperature settings are available in the market. Obvi‐ ously, optimum cell culture conditions need to be maintained particularly for the assays conducted with living organisms. Therefore, choosing a good quality incubator that can maintain sufficient air circulation throughout the storage chamber is crucial to achieve a uniform temperature gradient inside the unit and to assure consistent data quality plate- and experiment-wise.

Failure to sustain even distribution of temperature and gasses (*i.e.* carbon dioxide, nitrogen and oxygen) inside the incubator often results in significant edge effects (Figure 2). In some instances, the wells facing the center of the storage unit are exposed to higher temperatures, or *vice versa*, resulting in higher evaporation rates in those wells, which subsequently lead to varied effective drug concentrations in different regions within the plates. In such cases, it may be necessary to normalize the data for row-wise, column-wise or bowl-shape edge effects. Various statistical algorithms, such as median polishing, B-score and BZ-score [15, 16], may be applied to reduce these patterns. However, it may sometimes be inevitable to repeat the screening experiment. Non-uniform incubation conditions may also result in spatial patterns. As an example, plates located at the top of the stackers may be exposed to lower temperatures with respect to the plates at the bottom. Since these patterns are observed in an experimentwise fashion, they may be less of a concern for the researcher who always includes in-plate controls for "hit" picking purposes, so long as the biological activity is not compromised by the perturbed temperature distribution. Some temporal or spatial patterns may also be caused due to the over-crowdedness of the plates inside the incubation chamber. To avoid this pitfall, one can choose to allocate the plates more sparsely on the stackers allowing improved air circulation around each plate, although the capabilities of the automation software and the robotics may be a limiting factor. Such decision needs to be made on a case-by-case basis, taking into consideration the impact of the variations caused by the incubators, and the capacity of other instruments used.

#### **4.5. Proper use of internal assay controls**

detergents [7]. Alternatively, the blocking agents may be added to the assay buffer as long as

The selection of the appropriate assay plate type is important and mainly depends on the assay detection method. The light reflecting properties of the assay plate surfaces profoundly affect the final signal intensities, background noise levels and well-to-well crosstalk. Black, solid bottom, opaque-walled plates are recommended for fluorescence-based reading technologies to achieve lower background signal and minimal crosstalk, while white plates are good for luminescence signal detection to enhance light output. On the other hand, clear-bottom plates are needed for colorimetric assays, as well as for cell-based assays, where the cells need to be

Despite these general selection guidelines, a suitable assay plate type should be carefully chosen in compliant with the overall project goals. For instance, in a luminescence assay with low signal window and relatively high assay volume/well, where the "hit" compound is defined as the test sample that causes a drop in the signal intensity in comparison to the negative control, detection of the "hits" may be impaired if white plates are used. That is because a well containing the active compound with low signal intensity would be surrounded by several inactive wells with high signal intensities, and the crosstalk from the surrounding wells would greatly alter the original signal magnitude in the active well leading to increased false negative rates. In such luminescence assays, where the scientist is aiming to detect a signal

High content assays commonly require specially designed microtiter plates to attain the maximal scan performances, when high content imagers are used. These plates are generally intended to have optically clear, very thin and uniform well bottoms to ensure high quality images. Additionally, the assays that require fixation and staining processes, and involve multiple washing steps may necessitate plates that enhance cell retention. For this purpose, plates with poly-D-lysine, poly-L-lysine and collagen-coated surfaces are available to promote

Many more types of plates are offered for different assay methodologies, such as low attach‐ ment plates for cell-based assays using cells in suspension, and non-specific binding surface plates for protein-binding experiments. Besides, selection of the correct plate type for multiplex assays may require extra effort and testing process, especially if luminescence and fluorescence signals are being measured within the same plate. Performing a detailed search of the available plate options for the assay of interest is a time worthy practice, which would eventually save

In high throughput practices, a large number of plates are processed on a daily basis, and the environment in which the plates are stored during the screening runs exerts high impact on the assay data quality. Automation compatible incubators are manufactured to optimize

they do not interfere with other assay components.

232 Drug Discovery and Development - From Molecules to Medicine

monitored by microscopy throughout the course of the experiment.

decrease, black plates would be more appropriate to conduct the experiment.

the scientist from developing and validating the assay repeatedly.

**4.3. Plate type selection**

cell adhesion and growth.

**4.4. Incubation conditions**

#### *4.5.1. Non-biologically vs. biologically-relevant controls*

In an HTS assay, selection of the experimental controls is a major factor in screen quality valuation and "hit" picking process. It is very important to use well-defined controls that suit the assay objectives the best. When choosing positive and negative controls for an assay, the screening scientist should always be aware of the biological question that is aimed to be answered by conducting the screening campaign. For instance, one can prefer to use a known inhibitor compound as a positive control in a biochemical assay to identify "hit" compounds that display higher inhibitory potency than the control compound. Instead, in the same assay, one can choose to simply omit the enzyme and use the "no enzyme" condition as the positive control (to mimic "100% inhibition") to follow a different normalization approach for "hit" selection, especially when there is no known inhibitor, or the known inhibitor is prohibitory expensive. Screening of the same sample library with two different control setups, with and without biological relevance, could result in varied "hit" lists, despite the same "hit" cut-off. Thus, it is an important concept to consider whether to choose a non-biological control over a biologically relevant control.

Preferably, biological/chemical relevance is sought while choosing controls in any experimen‐ tal design, meaning that the mechanism of action of the controls should be pertinent to the biology that is being studied. As an example, staurosporine is a popular control compound used in cytotoxicity assays to induce cell death. Studies have shown that staurosporine can mediate cell death through apoptosis or necroptosis pathways in caspase-dependent or independent manners [17, 18]. Depending on the mechanism of interest, its caspase depend‐ ency and the timing of cell death, staurosporine may not be a suitable biologically relevant positive control for all cell viability assays.

Under certain circumstances, the use of artificial controls is inevitable, especially due to the lack of suitable biological control agents for some assay designs. Omitting essential experi‐ mental components; such as probes or cells, from the biological system to mimic 100% inhibition might artificially lead to lower standard deviations than when biologically relevant controls are used. Although not ideal, using artificial controls has become a common practice in some HTS campaigns due to various limitations. Hence, it is critical to know what have been used as assay controls when comparing the performance of similar screens performed by different groups.

#### *4.5.2. Interaction of positive controls with reagent*

The control agents used in the assay validation process play a significant role in foreseeing the potential issues in the proceeding large-scale screening runs. While some of these problems may be very obvious from the statistical analyses and can be easily fixed, some less obvious issues may be simply overlooked. For instance, control compounds containing certain chemical structures may modulate the activity of the reporter luciferase enzyme in the assay detection kit. If the insidious reaction mechanism of the control compound of interest is not well established, the output signal may be misinterpreted and confused with real biological response. Similar phenomenon may be experienced with autofluorescent or fluorescencequenching compounds in fluorescence-based assays. Although one needs to be vigilant about using these compounds as controls, they may still be utilized in the absence of "real" biological controls.

#### *4.5.3. Hill slope steepness and "medium" signal stability*

The compound concentration used to generate the "medium" signal in the assay validation is typically around the EC50 value for a potent compound. However, the reproducibility of the "medium" signal can be severely affected if the Hill slope of the corresponding dose-response curve is very steep (*i.e*. high coefficient values), as small disparities in the concentration of the control amplifies signal variation. To avoid this, a moderately potent compound, if available, can be used at higher concentration (*e.g*. EC90) to generate the "medium" assay signal.

#### *4.5.4. Stability and solubility of positive controls*

It is important to gather as much information as possible regarding the stability of positive controls, some of which may become evident during assay validation. Factors to consider include:


without biological relevance, could result in varied "hit" lists, despite the same "hit" cut-off. Thus, it is an important concept to consider whether to choose a non-biological control over a

Preferably, biological/chemical relevance is sought while choosing controls in any experimen‐ tal design, meaning that the mechanism of action of the controls should be pertinent to the biology that is being studied. As an example, staurosporine is a popular control compound used in cytotoxicity assays to induce cell death. Studies have shown that staurosporine can mediate cell death through apoptosis or necroptosis pathways in caspase-dependent or independent manners [17, 18]. Depending on the mechanism of interest, its caspase depend‐ ency and the timing of cell death, staurosporine may not be a suitable biologically relevant

Under certain circumstances, the use of artificial controls is inevitable, especially due to the lack of suitable biological control agents for some assay designs. Omitting essential experi‐ mental components; such as probes or cells, from the biological system to mimic 100% inhibition might artificially lead to lower standard deviations than when biologically relevant controls are used. Although not ideal, using artificial controls has become a common practice in some HTS campaigns due to various limitations. Hence, it is critical to know what have been used as assay controls when comparing the performance of similar screens performed by

The control agents used in the assay validation process play a significant role in foreseeing the potential issues in the proceeding large-scale screening runs. While some of these problems may be very obvious from the statistical analyses and can be easily fixed, some less obvious issues may be simply overlooked. For instance, control compounds containing certain chemical structures may modulate the activity of the reporter luciferase enzyme in the assay detection kit. If the insidious reaction mechanism of the control compound of interest is not well established, the output signal may be misinterpreted and confused with real biological response. Similar phenomenon may be experienced with autofluorescent or fluorescencequenching compounds in fluorescence-based assays. Although one needs to be vigilant about using these compounds as controls, they may still be utilized in the absence of "real" biological

The compound concentration used to generate the "medium" signal in the assay validation is typically around the EC50 value for a potent compound. However, the reproducibility of the "medium" signal can be severely affected if the Hill slope of the corresponding dose-response curve is very steep (*i.e*. high coefficient values), as small disparities in the concentration of the control amplifies signal variation. To avoid this, a moderately potent compound, if available,

can be used at higher concentration (*e.g*. EC90) to generate the "medium" assay signal.

biologically relevant control.

different groups.

controls.

positive control for all cell viability assays.

234 Drug Discovery and Development - From Molecules to Medicine

*4.5.2. Interaction of positive controls with reagent*

*4.5.3. Hill slope steepness and "medium" signal stability*

#### *4.5.5. Appropriate washing protocols for liquid transfer components when dealing with "persistentbinding" positive controls*

Liquid handling devices with permanent or reusable components that are in contact with the compounds are predisposed to contamination (Figure 3B). Such is the case when dealing with pintool systems or pipettes with fixed tips. Persistent binding of these chemicals can be prevented by extensive washing or using a more adequate washing solution. For instance, the standard DMSO/isopropanol washing cycles for pintool devices are not suitable for inorganic salts used as positive controls. Instead, acidic aqueous solutions are needed as first step to remove the salt precipitates at the pins.

#### *4.5.6. Negative control selection*

Typically, the negative control in the screening of small molecules is the solvent in which the test compounds are dissolved. DMSO is one of the standard solvents for dissolving chemical compound libraries in drug discovery. All biological assays are sensitive to certain DMSO levels, which should be determined beforehand. This information is crucial to finalize the screening workflow, such as whether intermediate-dilution plates with lower DMSO content would be needed.

## **5. Potential limitations**

Unforeseen problems can still arise during screening even after a thorough validation of an assay. These complications become evident when handling much larger numbers of plates than tested in the assay validation process. Therefore, it is imperative to scrutinize the data after each run during screening, particularly at the initial stages. Figure 4A illustrates a scenario (using simulated data) within a single screening run, where the first few plates behave as expected, but the positive control precipitates over time at the pins of the pintool device due to inappropriate selection of pin washing protocols, causing significant reduction in com‐ pound transferred with concomitant increase in signal variation.

Even after the first few successful runs, data quality can be significantly compromised at a later period. The case depicted in Figure 4B resembles a large screening campaign, where the reagent decomposes slowly (in a time-frame of weeks or months), resulting in a reduction of Z'-factor.

**Figure 4.** Examples of actual screening problems not revealed during initial assay validation.

#### **6. Conclusion**

HTS plays a major role in early drug discovery, but the massive amount of data generated in these endeavors can come with a high price tag in terms of effort, time and cost. Assurance of a successful screening campaign depends largely on the quality and reliability of the assay, which should be scrutinized using standard validation protocols that examine every possible scenario for potential setbacks. If possible, the assay validation process should include as closely as possible all the experimental conditions, mimicking an actual screening run.

#### **Abbreviation**

HTS: high throughput screening FP: fluorescence polarization FA: fluorescence anisotropy FRET: fluorescence resonance energy transfer TR-FRET: time-resolved FRET stdev: standard deviation CV: coefficient of variation DMSO: dimethyl sulfoxide

## **Acknowledgements**

Even after the first few successful runs, data quality can be significantly compromised at a later period. The case depicted in Figure 4B resembles a large screening campaign, where the reagent decomposes slowly (in a time-frame of weeks or months), resulting in a reduction of Z'-factor.

HTS plays a major role in early drug discovery, but the massive amount of data generated in these endeavors can come with a high price tag in terms of effort, time and cost. Assurance of a successful screening campaign depends largely on the quality and reliability of the assay, which should be scrutinized using standard validation protocols that examine every possible scenario for potential setbacks. If possible, the assay validation process should include as closely as possible all the experimental conditions, mimicking an actual screening run.

**Figure 4.** Examples of actual screening problems not revealed during initial assay validation.

236 Drug Discovery and Development - From Molecules to Medicine

**6. Conclusion**

**Abbreviation**

HTS: high throughput screening

FRET: fluorescence resonance energy transfer

FP: fluorescence polarization

FA: fluorescence anisotropy

TR-FRET: time-resolved FRET

stdev: standard deviation

CV: coefficient of variation

DMSO: dimethyl sulfoxide

This work was supported by the American Lebanese Syrian Associated Charities (ALSAC),

St. Jude Children's Research Hospital (SJCRH), National Institutes of Health National Institute of General Medical Sciences [Grants GM086415 & GM110034], and National Cancer Institute [Grant P30-CA21765]. The authors thank members of the Chen group, colleagues at the department of Chemical Biology and Therapeutics, and many collaborators for their valuable discussions.

## **Author details**

Sergio C. Chai, Asli N. Goktug and Taosheng Chen\*

\*Address all correspondence to: taosheng.chen@stjude.org

High Throughput Screening Center, Department of Chemical Biology and Therapeutics, St. Jude Children's Research Hospital, Memphis, TN, USA

Authors Sergio C. Chai and Asli N. Goktug contributed equally to this work.

### **References**


[17] Belmokhtar CA, Hillion J, Segal-Bendirdjian E. Staurosporine induces apoptosis through both caspase-dependent and caspase-independent mechanisms. Oncogene. 2001;20(26) 3354-62.

[6] Burbaum JJ. Miniaturization technologies in HTS: how fast, how small, how soon?

[7] Chai SC, Goktug AN, Cui J, Low J, Chen T. Practical Considerations of Liquid Han‐ dling Devices in Drug Discovery. In: El-Shemy H (ed.). Drug Discovery. Rijeka: In‐

[8] Iversen PW, Beck B, Chen Y-F, Dere W, Devanarayan V, Eastwood BJ, Farmen MW, Iturria SJ, Montrose C, Moore RA, Weidner JR, Sittampalam GS. HTS Assay Valida‐ tion. In: Sittampalam GS, Gal-Edd N, Arkin M, Auld D, Austin C, Bejcek B, Glicks‐ man M, Inglese J, Lemmon V, Li Z, McGee J, McManus O, Minor L, Napper A, Riss T, Trask Jr OJ, Weidner J (ed.). Assay Guidance Manual [Internet]. Bethesda (MD): Eli Lilly & Company and the National Center for Advancing Translational Sciences; 2004. Available from: http://www.ncbi.nlm.nih.gov/books/NBK83783/ (accessed 29

[9] Zhang JH, Chung TDY, Oldenburg KR. A simple statistical parameter for use in eval‐ uation and validation of high throughput screening assays. Journal of Biomolecular

[10] McDonald GR, Hudson AL, Dunn SMJ, You HT, Baker GB, Whittal RM, Martin JW, Jha A, Edmondson DE, Holt A. Bioactive Contaminants Leach from Disposable Labo‐

[11] Watson J, Greenough EB, Leet JE, Ford MJ, Drexler DM, Belcastro JV, Herbst JJ, Chat‐ terjee M, Banks M. Extraction, Identification, and Functional Characterization of a Bi‐ oactive Substance From Automated Compound-Handling Plastic Tips. Journal of

[12] Niles WD, Coassin PJ. Cyclic olefin polymers: Innovative materials for high-density multiwell plates. Assay and Drug Development Technologies. 2008;6(4) 577-90. [13] Palmgren JJ, Monkkonen J, Korjamo T, Hassinen A, Auriola S. Drug adsorption to plastic containers and retention of drugs in cultured cells under in vitro conditions.

European Journal of Pharmaceutics and Biopharmaceutics. 2006;64(3) 369-78.

[14] Harris D, Olechno J, Datwani S, Ellson R. Gradient, Contact-Free Volume Transfers Minimize Compound Loss in Dose-Response Experiments. Journal of Biomolecular

[15] Brideau C, Gunter B, Pikounis B, Liaw A. Improved statistical methods for hit selec‐ tion in high-throughput screening. Journal of Biomolecular Screening. 2003;8(6)

[16] Wu ZJ, Liu DM, Sui YX. Quantitative assessment of hit detection and confirmation in single and duplicate high-throughput screenings. Journal of Biomolecular Screening.

Drug Discovery Today. 1998;3(7) 313-22.

238 Drug Discovery and Development - From Molecules to Medicine

Tech; 2013. p177-200.

September 2014).

Screening. 1999;4(2) 67-73.

Screening. 2010;15(1) 86-94.

634-47.

2008;13(2) 159-67.

ratory Plasticware. Science. 2008;322(5903) 917.

Biomolecular Screening. 2009;14(5) 566-72.

[18] Dunai ZA, Imre G, Barna G, Korcsmaros T, Petak I, Bauer PI, Mihalik R. Staurospor‐ ine Induces Necroptotic Cell Death under Caspase-Compromised Conditions in U937 Cells. Plos One. 2012;7(7) e41945.
