Constructs, methods, confounding, and external validity

Whether we can make generalisations from our sample to the wider population, or across treatments, will also depend on how we operationalized our study; that is, how we defined the constructs and variables we wanted to measure, what treatments (i.e., interventions) we made, and so forth. In this section, we focus on threats to external validity that can arise from using (a) single measures and (b) single methods of constructs. Each of these is discussed in turn:

Generalisation and constructs

In quantitative research designs, we have to narrow down broad concepts we may be interested in into constructs that can be measured [see the article: Constructs in quantitative research].

For example, we may be interested in a broad concept such as intelligence. However, the concept, intelligence, can be viewed from a wide range of perspectives. If we were to try and examine such a broad concept, we would not be confident that our study was construct valid, which would lead to considerable criticism [see the article: Construct validity]. Therefore, rather than tackling broad concepts such as intelligence, we focus on more specific constructs. In terms of the concept, intelligence, we may choose to look at a common construct such as IQ. However, there are others ways to perceive and measure intelligence, including EQ (or emotional intelligence).

Since experiments cannot take into account all of the different ways that a concept can be perceived and measured, it will only be possible to make generalisations from your results within the confines of the operational definition you provided for your concept. By operational definition, we mean the construct you decided to study from some broader concept (i.e., how you perceived a concept), and the specific way you decided to measure that construct (i.e., the variables you used to represent the construct) [see the article: Constructs in quantitative research]. This points to two types of generalisation we need to be careful about when it comes to constructs: generalising across constructs and generalising across measures for a given construct. Each is discussed in turn:

1 2 3 4 5 6