Just as there are threats to construct validity from a single measure, as discussed in the previous section, construct validity can also be threatened when using a single method to measure a given construct (irrespective of whether the construct is acting as the dependent or independent variable). This is because the method used may introduce bias, changing the scores on the independent or dependent variable. It is known as mono-method bias.
Before we reflect back on the threat to construct validity from using a single method, let's look at the problems that can arise from using a single method, based on our example in Study #1:
Study #1
The relationship between background music and task performance amongst employees at a packing facility
To briefly recap, our study examined the relationship between background music (i.e., construct A) and task performance (i.e., construct B) amongst employees at a packing facility (e.g., Amazon, Wal-Mart, Tesco, etc.). The independent variable was background music (i.e., construct A), whilst the dependent variable was task performance (i.e., construct B). One group of employees listened to background music whilst working (i.e., the treatment group), whereas the other group were not provided with any background music (i.e., the control group). The task performance of employees was measured in terms of the number of tasks employees perform correctly per hour. The method used to listen to background music was a loud speaker (i.e., stereo system), whilst task performance was measured using an e-packing system, which automatically collected data on the number of tasks correctly performed by employees.
Let's imagine some of the multiple methods that could be used to measure these two constructs (i.e., construct A, background music, and construct B, task performance):
▪ Independent variable
Method #1: Listening to music through the loud speaker (i.e. stereo system)
Method #2: Listening to music using a personal iPod and headphones
▪ Dependent variable
Method #1: Data automatically collected through the e-packing system
Method #2: Supervisor rating the speed of the packer
Note that sometimes a mono-method (i.e., a single method) is appropriate. For example, Method #1 for the dependent variable (i.e., data being automatically collected through the e-packing system, recording task performance accurately) may be the most accurate measure of task performance in this piece of research. After all, Method #1, where the supervisor rates the speed of the packer is more likely to result from experimenter bias or instrumental bias than an automated system that does not suffer from such bias [see the article, Internal validity, for more information on experimenter bias and instrumental bias]. However, this is often not the case, and the use of multiple methods reduces the threat to construct validity.
When considering mono-method bias in your dissertation, you need to ask yourself:
Would the same results have been recorded if the independent variable, background noise, had been operationalized using a different method; in this case, using Method #2 (i.e., listing to music using a personal iPod and headphones) rather than Method #2 (i.e., listening to music through the loud speaker/stereo system)?
Would the use of multiple methods have provided greater insight into the construct than just a single method; that is, would have multiple methods reduced the potential for method bias to affect the scores on the dependent variable?
Therefore, if the measurement procedure consists of a single method to assess the independent and/or dependent variables, this can act as a threat to construct validity. In order to reduce the threat from mono-method bias, it is useful to use more than one method when measuring a given construct. You can then assess the convergent validity of the two methods to check that they are measuring the same construct, which helps to strengthen the construct validity of your study [see the article: Convergent and divergent validity].