STEP FIVE

Research quality

All research has limitations, which negatively impact upon the quality of the findings you arrive at from your data analysis. This is the case whether you are an undergraduate or master's level student doing a dissertation, a doctoral student, or a seasoned academic researcher. The main journal article you are interested in will also have a number of limitations, some of which will have inevitably become the justifications for your chosen route, and the approach you selected within that route.

You need to think about research quality at this stage in the dissertation process because many of the problems experienced during the dissertation process can be avoided. The trick is to (a) understand the types of research limitation you may face when doing a dissertation, (b) anticipate what these will be in your dissertation, and (c) avoid them becoming a reality (where possible). Quite simply, the better the research quality of your dissertation, (a) the fewer problems you will experience when carrying out your dissertation research, (b) the less time you will need to write up the Research Limitations section of your Discussion/Conclusions chapter (i.e., Chapter Five: Discussion/Conclusions), and (c) the greater the likelihood of a high mark.

To improve the research quality of your dissertation, you need to follow four steps: (a) understand the five factors through which research quality is assessed - internal validity, external validity, construct validity, reliability and objectivity; (b) assess the research quality of the main journal article; (c) consider the potential research quality of your research strategy; and (d) determine how you will overcome such weaknesses in your dissertation, considering the practical aspects of your dissertation, and the implications that these may have on the quality of your findings.

STEP ONE
Understand the five factors through which research quality is assessed

In quantitative dissertations, research quality is assessed based on the internal validity, external validity, construct validity, reliability and objectivity of the research. Irrespective of the route that you are following, or the approach within that route, it is important that (a) your dissertation is as internally and externally validity as possible, (b) the measurement procedure you used (i.e., the research method and its measures) are construct valid and reliable, and (c) your research was carried out in an objective way. If you are already confident that you understand these five means through which the quality of quantitative research is assessed, jump to STEP TWO: Assess the research quality of the main journal article. If not, we would suggest that you learn about these terms in the Research Quality section of the Fundamentals part of Lærd Dissertation before reading on. After all, in STEP TWO below, you will need to assess the research quality of the main journal article, before being able to consider the potential weaknesses in research quality in your dissertation, and how you will overcome these weaknesses. To do this, you first need to understand these five main factors through which research quality is assessed.

STEP TWO
Assess the research quality of the main journal article

Irrespective of the route that you are following, the person marking your work will expect that you have critically analysed the research strategy used in the main journal article. Even if limitations in the research strategy do not act as the main justification for your choice of route, or the approach within that route (i.e., as is the case in method and measurement-based extensions, or design-based extensions within Route C: Extension), being able to critically analyse the research strategy used is typically a very important part of the marking scheme for dissertations. Just as you were expected to critically analyse the literature in STAGE FIVE: Building the theoretical case, you have to demonstrate an equally good knowledge of the weaknesses (and strengths) of the research strategy of the main journal article.

You can critically analyse the research strategy used by assessing the research quality of the research strategy used in the main journal article in terms of (a) the internal and external validity of the research strategy, and (b) the construct validity and reliability of the measurement procedure that was used (i.e., the research method and its measures). In most cases, since you did not witness the way that the research in the main journal article was carried out in practice, it will be difficult to assess the objectivity of the research.

Therefore, in order to assess the research quality of the main journal article, you should read up about internal validity, external validity, construct validity, reliability, and even the objectivity of research in the Research Quality section of the Fundamentals part of Lærd Dissertation. However, it is worth mentioning that:

• To assess the internal and external validity of the research strategy, assess the threats to such internal and external validity in the main journal article. For example, in the article, Internal validity, we discuss 14 potential threats to internal validity, which include (a) history effects, (b) maturation, (c) testing effects, (d) instrumentation, (e) statistical regression, (f) selection biases, (g) experimental mortality, (h) causal time order, (i) diffusion (or imitation) of treatments, (j) compensation, (k) compensatory rivalry, (l) demoralization, (m) experimenter effects and (n) subject effects. Since any of these 14 threats could have affected the internal validity of the main journal article, you should briefly read up about each one, and then assess whether you think these threats were present in the main journal article. You should note that it will not always be possible to tell whether such a threat was a problem because whilst some are more evident (e.g., the authors of the main journal article should have specified how they selected individuals to be included in their sample, which could expose potential selection biases), many are not so obvious (e.g., experimenter effects could have occurred as a result of the personal characteristics of the researchers in the main journal article, or some non-verbal cues that they gave off, which influenced the choices participants made when they were being studied, but this would be extremely difficult to spot, especially if the authors did not explicitly try to assess such bias, which is uncommon). Again, you can learn about internal validity and external validity in the Research Quality section of the Fundamentals part of Lærd Dissertation.

• Construct validity and reliability are two different ways of assessing the measurement procedure used in the main journal article. Construct validity is important because we want to make sure that the measurement procedure (e.g., a survey, structured interview, structured observation, etc.) that was used to measure the constructs we are interested in (e.g., sexism, obesity, famine, outsourcing, etc.) are valid. By construct valid, we mean that there is (a) a clear link between the constructs you are interested in and the measures and interventions that are used to operationalize them (i.e., measure them), and (b) a clear distinction between different constructs. Construct validity is an overarching term used to refer to the process of assessing the validity of the measurement procedure that was used, and you will need to read up about other types of validity that you will need to consider (i.e., content validity, convergent and divergent validity, criterion validity), especially if you are taking on a method or measurement-based extension, or design-based extension within Route C: Extension. You can learn more about construct validity in the article: Construct validity. Reliability is important because in order for the results from a study to be considered valid, the measurement procedure must first be reliable. There are a number of types of reliability that you may need to consider when assessing the main journal article, depending on whether the measurement procedure involved (a) successive measurements; (b) simultaneous measurements by more than one researcher; and/or (c) multi-measure procedures. You can learn more about these types of reliability in the article: Reliability in research. When reading up about construct validity and reliability in these articles, you will learn how to assess a piece of research (i.e., your main journal article) in terms of its construct validity and reliability.

When you understand the five factors through which research quality is assess (i.e., STEP ONE), and have assessed the research quality of your main journal article (i.e., STEP TWO), you will be well-equipped to consider the potential research quality of your research strategy, based on the route you adopted, and the approach within that route (i.e., STEP THREE next).