STEP THREE
Consider the potential research quality of your research strategy

At this point of STAGE SIX: Setting your research strategy, you should have a clear idea of the research design, research methods and sampling strategy that you plan to use in your dissertation. As a result, you now need to assess the potential research quality of these three components of your research strategy (i.e., the research design, research methods and sampling strategy) in terms of internal validity, external validity, construct validity and reliability. You should also assess the objectivity of the dissertation process in terms of (a) the way that you arrived at such a research strategy, and (b) the way that you plan to carry out your research strategy. Whilst you should assess the internal validity, external validity, construct validity and reliability of your dissertation, irrespective of the route that you adopted, or the approach within that route, there are additional factors to consider depending on your route and approach. Route A: Duplication, Route B: Generalisation and Route C: Extension are discussed in turn:

Route A: Duplication

When you are following Route A: Duplication, your main goal is to found out if the results from the main journal article can be replicated (i.e., can your study, using the same or an extremely similar research strategy, produce the same results).

If the results from your dissertation do confirm those results from the main journal article, this will inevitable: (a) strengthen the external validity of the main journal article because it will help to show that the hypotheses tested can be confirmed using a second sample (i.e., the sample that you used), showing that the findings from your dissertation and the main journal article can be generalised to the population that was being studied; (b) suggest that the measurement procedure you used (e.g., a survey, structured interview, structured observation, etc.) is reliable, assuming that the tests of reliability that you perform show this to be the case (i.e., testing for the reliability of a measurement procedure is something that we show you how to do in the Data Analysis part of Lærd Dissertation); and (c) strengthen the construct validity of the measurement procedure that you used because it will provide more evidence that the measurement procedure you used to measure the constructs you were interested in (e.g., sexism, obesity, famine, outsourcing, etc.) is valid (i.e., construct validity is something that develops gradually over time as each additional study shows that a measurement procedure is a valid measure of a given construct or set of constructs). However, external validity, reliability and construct validity should not be your main focus when it comes to considering the potential research quality of your research strategy because these are things that are demonstrated (or not) after you have collected and analysed your data, and can say whether your dissertation confirms the results from the main journal article. Since you are not changing the research design, research methods (i.e., the measurement procedure and its measures) or the sampling strategy used in the main journal article, these are not aspects of research quality you need to concern yourself about at this stage. Instead, you will reflect on them during the write up, typically in the Results chapter (i.e., usually Chapter Four: Results) and the Research Limitations section of your Discussion/Conclusions chapter (i.e., usually Chapter Five: Discussion/Conclusions).

When considering the potential research quality of your research strategy, you need to focus on eliminating potential threats to the internal validity of your dissertation that are, to some extent at least, in your control, and which you can minimise in the way that you plan your data collection phase and carry it out. We are not talking about threats to internal validity that may result from the research design or research methods that were used in the main journal article (e.g., threats such as testing effects, instrumentation, causal time order, etc.). If you feel that such threats to internal validity in the main journal article could have significantly affected the results (i.e., that different results would have been found had these threats not been present), it is typically not enough to simply duplicate the main journal article (i.e., as you are doing in Route A: Duplication). Instead, you would find yourself extending the main journal article in some way to reduce such threats (i.e., probably by taking on a design-based extension, or a measurement or methods-based extension within Route C: Extension). However, assuming that this is not the case, and the threats to the internal validity of your dissertation are somewhat controllable and capable of being reduced (e.g., threats such a selection biases, diffusion of treatments, compensation, etc.), these are the things that you need to focus on. Minimising such threats will help you to argue that your results accurately reflect what you were studying, reducing potential criticism that other factors (e.g., selection biases) explain your results.

Route B: Generalisation

When you are following Route B: Generalisation, you main goal is to found out if the results from the main journal article hold across different populations, settings/contexts, treatments, or time, depending on the approach that you take towards this route. The extent to which the results from your dissertation are similar to those of the main journal article will tell you how far the results can be generalised to other populations, settings/contexts, treatments, or time; that is, how externally valid the hypotheses that were tested are.

If the results from your dissertation do confirm those results from the main journal article, this will inevitable: (a) strengthen the external validity of the main journal article because it will help to show that the hypotheses tested can be confirmed using a second sample (i.e., the sample that you used), showing that the findings from your dissertation and the main journal article can be generalised to the population that was being studied; (b) suggest that the measurement procedure you used (e.g., a survey, structured interview, structured observation, etc.) is reliable, assuming that the tests of reliability that you perform show this to be the case (i.e., testing for the reliability of a measurement procedure is something that we show you how to do in the Data Analysis part of Lærd Dissertation); and (c) strengthen the construct validity of the measurement procedure that you used because it will provide more evidence that the measurement procedure you used to measure the constructs you were interested in (e.g., sexism, obesity, famine, outsourcing, etc.) is valid (i.e., construct validity is something that develops gradually over time as each additional study shows that a measurement procedure is a valid measure of a given construct or set of constructs). However, reliability and construct validity should not be your main focus when it comes to considering the potential research quality of your research strategy because these are things that are demonstrated (or not) after you have collected and analysed your data, and can say whether your dissertation confirms the results from the main journal article. Since you are not changing the research design or research methods (i.e., the measurement procedure and its measures) used in the main journal article, these are not aspects of research quality you need to concern yourself about at this stage. Instead, you will reflect on them during the write up, typically in the Results chapter (i.e., usually Chapter Four: Results) and the Research Limitations section of your Discussion/Conclusions chapter (i.e., usually Chapter Five: Discussion/Conclusions).

When considering the potential research quality of your research strategy, you need to focus on eliminating potential threats to the internal validity and external validity of your dissertation that are, to some extent at least, in your control, and which you can minimise in the way that you plan your data collection phase and carry it out. We are not talking about threats to internal and external validity that may result from the research design or research methods that were used in the main journal article (e.g., threats to internal validity such as testing effects, instrumentation, causal time order, etc.), or threats to external validity (e.g., those relating to constructs, methods, and confounding, or "real world" versus "experimental world" differences, etc.). Rather, we are talking about threats that you do have the potential to control.

• In terms of internal validity, if you feel that such threats in the main journal article could have significantly affected the results (i.e., that different results would have been found had these threats not been present), it is typically not enough to simply test the generalizability of the findings in the main journal article (i.e., as you are doing in Route B: Generalisation). Instead, you would find yourself extending the main journal article in some way to reduce such threats (i.e., probably by taking on a population or context/setting-based extension, or perhaps a design-based extension, or a method or measurement-based extension within Route C: Extension). However, assuming that this is not the case, and the threats to the internal validity of your dissertation are somewhat controllable and capable of being reduced (e.g., threats such a selection biases, diffusion of treatments, compensation, etc.), these are the things that you need to focus on. Minimising such threats will help you to argue that your results accurately reflect what you were studying, reducing potential criticism that other factors (e.g., selection biases) explain your results.

• Similarly, you need to try and reduce threats to external validity during the research strategy phase, especially selection biases when creating your sample, which is one of the few major threat to external validity that you have control over when taking on Route B: Generalisation. Dealing with other threats to external validity, such as those relating to constructs, methods, and confounding, require changes to the research methods used in the main journal article, which would mean you would have to extend the main journal article in some way to reduce such threats (i.e., probably by taking on a method or measurement-based extension within Route C: Extension).

As you might have noticed, reducing selection biases when creating your sample (i.e., sampling biases) is particularly important to improving the research quality of dissertations that follow Route B: Generalisation. As discussed in STEP FOUR: Sampling strategy, this is because the characteristics of the new population or setting/context that are important are likely to be different from those characteristics that were important in the main journal article. As a result, you have to rely less on the sampling strategy used in the main journal article and focus more on the population and setting/context that you are interested in. Therefore, when creating your sample in a dissertation based on Route B: Generalisation, you have to think: (a) what the most important characteristics of your sample are; and (b) make sure that the sample you select is as representative of the population you are interested in as possible (i.e., if you have to use a non-probability sampling technique rather than a probability sampling technique when creating your sample, you already know that the internal validity and external validity of your findings are being reduced because the representativeness of your sample is being threatened).