**4.2 Methodological Issues**

A number of methodological issues of child psychiatric prevalence studies will be considered for better understanding the results of these studies. Issues that pertain to general epidemiological methodology, such as sampling and data analysis, will not be discussed here. The focus will be on issues that are specific to child psychopathology, such as assessment, diagnostic principles, and morbidity criteria.

The Epidemiology of Child Psychopathology: Basic Principles and Research Data 143

mother; teacher and teacher aide) was 0.60. These findings were confirmed by more recent studies (Duhig et al. 2000; Grietens et al. 2004). There are several possible explanations for cross-informant discrepancies in parent and teacher reports of child behaviour problems, including issues related to informant bias, the demands of the context in which the child's behaviour is being assessed and poor measurement reliability (De Los Reyes & Kazdin, 2005). For reasons of comparability, it is recommended that prevalence studies report prevalence rates based on specific informants separately, and that procedures for combining information from different informants will be well documented in ways that can be easily

Disagreement among informants can be valuable (Jensen et al., 1999). As example, Ferdinand et al. (2003) studied problem behaviour in adolescents from the Dutch general population, aged 15 to 18 years, across a 4-year period. Initially, parent information was obtained with the CBCL and self-reports with the Youth Self-Report. Signs of poor outcome, including police contacts and drug use, were assessed four years later. Discrepancies between information from parents and adolescents added significantly to the prediction of poor outcome based on information from each informant separately. For instance, scores on the Delinquent Behavior syndrome based on parent information or on adolescents´ selfreports separately did not predict future police contacts. However, if parents reported scores in the deviant range on the Delinquent Behavior scale, while adolescents reported scores in the normal range on this scale, adolescents were at increased risk for later police or judicial

*Morbidity criteria:* Most problem behaviours in children can best be regarded as quantitative variations rather than present/absent categories. This approach allows for inter-individual differences that are normal. Abnormality can be regarded as the quantitative extreme of the normal distribution. This quantitative approach makes it possible to assess the degree to which an individual child's problems deviate from those that are typical of the individual's age and sex. In order to make such comparisons, we need data on large, representative samples of boys and girls of different ages from the general population. Despite the fact that many psychopathological phenomena in children can best be regarded as quantitative variations, for identifying individuals in the general population with mental health problems, we need to dichotomise quantitative information into categories that are defined by cut points for distinguishing between cases and non-cases. There is as yet little basis for perfect categorical distinctions between psychopathology and normality. For most problem rating scales this is done by comparing the distribution of scores for non-cases with the distribution of scores for cases. In the absence of an ultimate criterion for caseness, the most frequently used morbidity criterion is whether a child has been referred for specialist mental health services. However, caution is needed because this approach is fallible; some children who are not referred may have significant problems, while not all children who are referred

DSM diagnostic criteria can also be used for deciding who is disordered and who is not. These criteria are the result of negotiations among expert panels and often lack firm empirical evidence. Prevalence studies that use DSM diagnostic criteria to define caseness run into the problem that DSM criteria are overinclusive, often resulting in extremely high prevalence rates. As example, Bird et al. (1988) found that 49.5% of children in Puerto Rico met criteria for DSM-III disorders. As a result studies using DSM criteria often combine

replicated.

contacts (Verhulst & Koot, 1995).

really need professional help.

*Assessment:* All assessment procedures are subject to error due to variations in the phenomena being assessed and in the procedures themselves. To reduce variations in the data obtained and to improve precision is the use of standardized assessment procedures. Epidemiological researchers in child psychiatry were among the first to use standardized assessment procedures (Rutter et al., 1970). Rating scales were developed because they could easily be applied in a cost effective way in large-scale epidemiological studies, and standardized psychiatric interviews were developed for more in-depth assessments of the prevalence of psychiatric diagnoses. Conversely, epidemiological data are indispensable for obtaining norms and for testing the validity of these instruments (Shaffer et al., 1999). Epidemiological comparisons of normal and disordered children are needed to determine how childhood disorders are actually distributed and for identifying optimal cut-offs for distinguishing between children who will most likely benefit from particular interventions versus those who will not (Fombonne, 2002; Verhulst, 1995).

There are two main approaches, the *empirical* and the *a priori* approach, to determine the level of psychopathology in individuals. The empirical approach employs multivariate statistical techniques, such as factor analysis and principal components analysis that are used to identify sets of problems that tend to occur together. These co-occurring items constitute empirical syndromes. This approach starts with empirical data derived from informants who describe the behaviour of children, without any assumptions about whether these syndromes reflect predetermined diagnostic categories. The empirical-quantitative approach forms the basis of the empirical syndromes of rating scales such as the Child Behavior Checklist (CBCL; Achenbach, 2009) or the Conners' Rating Scales (Conners, 1997). Prevalence studies using the empirical approach generate quantitative scores reflecting the level of problems of a child. Imposing cut points to the quantitative scores can make categorical distinctions between disordered and normal individuals. The second approach refers to the diagnostic categories employed by of one of the two international nosological systems, the fourth edition of the American Psychiatric Association's *Diagnostic and Statistical Manual of Mental Disorders (DSM-IV)* or the World Health Organization's *International Classification of Diseases (ICD)*. This approach starts with assumptions about which disorders exist and about which symptoms define them. Some prevalence studies generating DSM diagnoses for general population samples as discussed above used combinations of both approaches with rating scales for screening the total sample and psychiatric interviews used for assessing a selected subsample of children scoring in the problem range of the rating scales.

*Multiple informants:* To obtain a comprehensive picture of a child's functioning, information from different informants is needed. Many prevalence studies used information from usually parents, teachers, and the child. The reason for this is because agreement among informants is far from perfect, and because no one informant can substitute for all others. Different informants having different relations to the child and seeing the child under different conditions, often vary in their response to the child's behaviour. In a first metaanalytic study, Achenbach et al. (1987) computed the average correlation between different informants' ratings of problem behaviours in a large number of published samples. The mean correlation between pairs of adult informants who played different roles with respect to the children was 0.28 (e.g., parents versus teachers). The mean correlation between selfreports and reports by parents, teachers, and mental health workers was even lower (0.22). In contrast, the mean correlation between pairs of similar informants (e.g., father and

*Assessment:* All assessment procedures are subject to error due to variations in the phenomena being assessed and in the procedures themselves. To reduce variations in the data obtained and to improve precision is the use of standardized assessment procedures. Epidemiological researchers in child psychiatry were among the first to use standardized assessment procedures (Rutter et al., 1970). Rating scales were developed because they could easily be applied in a cost effective way in large-scale epidemiological studies, and standardized psychiatric interviews were developed for more in-depth assessments of the prevalence of psychiatric diagnoses. Conversely, epidemiological data are indispensable for obtaining norms and for testing the validity of these instruments (Shaffer et al., 1999). Epidemiological comparisons of normal and disordered children are needed to determine how childhood disorders are actually distributed and for identifying optimal cut-offs for distinguishing between children who will most likely benefit from particular interventions

There are two main approaches, the *empirical* and the *a priori* approach, to determine the level of psychopathology in individuals. The empirical approach employs multivariate statistical techniques, such as factor analysis and principal components analysis that are used to identify sets of problems that tend to occur together. These co-occurring items constitute empirical syndromes. This approach starts with empirical data derived from informants who describe the behaviour of children, without any assumptions about whether these syndromes reflect predetermined diagnostic categories. The empirical-quantitative approach forms the basis of the empirical syndromes of rating scales such as the Child Behavior Checklist (CBCL; Achenbach, 2009) or the Conners' Rating Scales (Conners, 1997). Prevalence studies using the empirical approach generate quantitative scores reflecting the level of problems of a child. Imposing cut points to the quantitative scores can make categorical distinctions between disordered and normal individuals. The second approach refers to the diagnostic categories employed by of one of the two international nosological systems, the fourth edition of the American Psychiatric Association's *Diagnostic and Statistical Manual of Mental Disorders (DSM-IV)* or the World Health Organization's *International Classification of Diseases (ICD)*. This approach starts with assumptions about which disorders exist and about which symptoms define them. Some prevalence studies generating DSM diagnoses for general population samples as discussed above used combinations of both approaches with rating scales for screening the total sample and psychiatric interviews used for assessing a selected subsample of children scoring in the

*Multiple informants:* To obtain a comprehensive picture of a child's functioning, information from different informants is needed. Many prevalence studies used information from usually parents, teachers, and the child. The reason for this is because agreement among informants is far from perfect, and because no one informant can substitute for all others. Different informants having different relations to the child and seeing the child under different conditions, often vary in their response to the child's behaviour. In a first metaanalytic study, Achenbach et al. (1987) computed the average correlation between different informants' ratings of problem behaviours in a large number of published samples. The mean correlation between pairs of adult informants who played different roles with respect to the children was 0.28 (e.g., parents versus teachers). The mean correlation between selfreports and reports by parents, teachers, and mental health workers was even lower (0.22). In contrast, the mean correlation between pairs of similar informants (e.g., father and

versus those who will not (Fombonne, 2002; Verhulst, 1995).

problem range of the rating scales.

mother; teacher and teacher aide) was 0.60. These findings were confirmed by more recent studies (Duhig et al. 2000; Grietens et al. 2004). There are several possible explanations for cross-informant discrepancies in parent and teacher reports of child behaviour problems, including issues related to informant bias, the demands of the context in which the child's behaviour is being assessed and poor measurement reliability (De Los Reyes & Kazdin, 2005). For reasons of comparability, it is recommended that prevalence studies report prevalence rates based on specific informants separately, and that procedures for combining information from different informants will be well documented in ways that can be easily replicated.

Disagreement among informants can be valuable (Jensen et al., 1999). As example, Ferdinand et al. (2003) studied problem behaviour in adolescents from the Dutch general population, aged 15 to 18 years, across a 4-year period. Initially, parent information was obtained with the CBCL and self-reports with the Youth Self-Report. Signs of poor outcome, including police contacts and drug use, were assessed four years later. Discrepancies between information from parents and adolescents added significantly to the prediction of poor outcome based on information from each informant separately. For instance, scores on the Delinquent Behavior syndrome based on parent information or on adolescents´ selfreports separately did not predict future police contacts. However, if parents reported scores in the deviant range on the Delinquent Behavior scale, while adolescents reported scores in the normal range on this scale, adolescents were at increased risk for later police or judicial contacts (Verhulst & Koot, 1995).

*Morbidity criteria:* Most problem behaviours in children can best be regarded as quantitative variations rather than present/absent categories. This approach allows for inter-individual differences that are normal. Abnormality can be regarded as the quantitative extreme of the normal distribution. This quantitative approach makes it possible to assess the degree to which an individual child's problems deviate from those that are typical of the individual's age and sex. In order to make such comparisons, we need data on large, representative samples of boys and girls of different ages from the general population. Despite the fact that many psychopathological phenomena in children can best be regarded as quantitative variations, for identifying individuals in the general population with mental health problems, we need to dichotomise quantitative information into categories that are defined by cut points for distinguishing between cases and non-cases. There is as yet little basis for perfect categorical distinctions between psychopathology and normality. For most problem rating scales this is done by comparing the distribution of scores for non-cases with the distribution of scores for cases. In the absence of an ultimate criterion for caseness, the most frequently used morbidity criterion is whether a child has been referred for specialist mental health services. However, caution is needed because this approach is fallible; some children who are not referred may have significant problems, while not all children who are referred really need professional help.

DSM diagnostic criteria can also be used for deciding who is disordered and who is not. These criteria are the result of negotiations among expert panels and often lack firm empirical evidence. Prevalence studies that use DSM diagnostic criteria to define caseness run into the problem that DSM criteria are overinclusive, often resulting in extremely high prevalence rates. As example, Bird et al. (1988) found that 49.5% of children in Puerto Rico met criteria for DSM-III disorders. As a result studies using DSM criteria often combine

The Epidemiology of Child Psychopathology: Basic Principles and Research Data 145

*Socio-economic status:* Previous studies have shown that rates of psychopathology are higher among individuals with lower socioeconomic status (SES) than those with higher SES (e.g. Schonberg & Shaw 2007). Published findings regarding associations between parents' marital status, immigration, and child behaviour problems are rare, and fewer studies still have reported on these associations in early childhood (Javo et al., 2004). Achenbach and Rescorla (2007) summarize studies from 15 cultures that tested associations between scores on empirically based scales and measures of socio-economic status (SES) in large population samples. Measures of SES varied across studies, but most used the occupation and/or education of the child's parents and grouped participants into low-, medium-, and high-SES groups. A few studies also used measures of family income. Although the studies varied in statistical details, they were consistent in reporting higher problem scores for children from lower-SES than from higher SES (Verhulst, 1995; Waddell, 2002). Although this finding was consistent across studies, the effects were rather small. There are a number of reasons that may be responsible for the finding that children from lower SES are somewhat

*Degree of urbanization:* Most studies investigating differences in prevalence rates between urban and rural populations did not find significant differences (Waddell et al., 2002). Achenbach and Rescorla (2007) conducted a detailed comparison of varying degrees of urbanization while controlling for sex, age, referral status, SES, region and ethnicity in a US national sample. Children from the most urban areas showed a slight tendency to obtain higher parent reported problem scores than children from the most rural areas. However, unexpectedly, the greatest contrast in problem scores was found between children in the intermediate categories versus those in the most rural areas, with highest scores for children

Behavioural and emotional disturbance are very common among children and adolescents. Approximately 20 % of children in Western, industrialized countries experience the signs and symptoms that constitute internalizing (e.g. anxiety/depression, withdrawal) or externalizing (e.g. oppositional defiance, aggression) DSM-IV disorders (Tolan & Dodge, 2005). Recently, epidemiological research has begun to focus on children younger than six and to consider the clinical significance of behavioural and emotional problems of this period of the life span (Angold & Egger, 2007). A review on the epidemiology of emotional and behavioural disorders in preschool children estimated the overall prevalence of 'problematic' behaviour as lying at somewhere between 7 % and 25 % (Egger & Angold, 2006). Empirical findings illustrate a first peak in multi modal distribution of mental health service utilization in childhood in 6-9-year-old children (e.g. Campbell, 2006). This raises the question of whether child psychiatric disturbance pre-exists school attendance, but remains

In a study in the US, Wadsworth and Achenbach (2005) reported differential incidence by SES for elevated scores on internalizing and externalizing disorders. It is not yet clear whether these findings can be generalized outside of the US, and to preschool age and other informants. From a multicultural perspective, Achenbach and Rescorla (2007) specified in their comprehensive review the need for comparable data on preschool-aged instruments. The aims

**5. Behavioural and emotional problems of kindergarten children** 

disadvantaged.

**5.1 Background** 

undetected.

in the intermediate categories.

DSM diagnostic criteria with impairment measure, for example the Children's Global Assessment Scale (CGAS; Shaffer et al., 1999). The newest versions of some psychiatric interviews such as the Diagnostic Interview Schedule for Children (DISC; Shaffer et al., 2000) and the Child and Adolescent Psychiatric Assessment (CAPA; Angold & Costello, 2000) have included impairment criteria. Because many children who meet criteria for DSM disorders are not greatly impaired in their everyday functioning, the addition of impairment measures results in a decrease of prevalence rates. In a Dutch prevalence study, for example, the prevalence of 21.8 % of children who met criteria for any DSM-III-R disorder based on parent interview information dropped to 5.9 % when combined with a CGAS score indicating definite impairment (Verhulst et al., 1997). Conversely there are also many children who can be regarded functionally impaired but do not meet criteria for DSM diagnoses. Some 50% of children attending clinics in the Great Smoky Mountains Study do not reach DSM or ICD criteria for a diagnosis and yet half of these are significantly impaired in their social functioning (Angold et al., 1999).
