🕒 25 min
What is a clinical trial and how (why) does it work?
A clinical trial is a study conducted on human volunteers to investigate a variety of questions on a treatment/intervention tested:
- is the treatment/intervention safe?
- does the treatment/intervention work?
- does the treatment/intervention work better than what is already available (if there is a similar treatment/intervention)?
Clinical trials must be conducted in accordance with the International Conference on Harmonisation (ICH) of technical requirements for registration of pharmaceuticals for human use and Good Clinical Practice (GCP) and must be approved before starting by a number of Regulatory or Competent Authorities (CAs) and Ethics Committees (ECs), depending on the country where the study is conducted. Such CAs include the Food and Drug Administration; FDA in the US, the European Medical Agency; EMA for the EU (with a number of European country-specific CA), the Medicines and Healthcare products Regulatory Agency; MHRA in the UK and the Pharmaceuticals and Medical Devices Agency; PMDA in Japan. The examples of Ethics Committees are the Institutional Review Board (IRB) in the US and the Medical Research Ethics Committee (MREC) in the Netherlands. These bodies scrutinize the plan to conduct the study and without their approval and agreement, the trial cannot commence.
Elements of clinical trials
To conduct a clinical trial, a precise and thorough plan needs to be established. In addition to that, an array of relevant stakeholders is identified and careful and meticulous operational and scientific study design and conduct aspects are developed.
Protocol and Informed Consent
A primary document of importance is the study protocol, which describes the study design, the objectives and clinical endpoints, methods and assessment types, schedule of activities, and statistical considerations for data analysis. In addition, the protocol states steps to protect participants’ rights and provides aspects on participants’ overall safety including all study inclusion and exclusion criteria to which each participant needs to adhere 100%.
Another document of importance, intended for a potential participant, is the Informed Consent Form (ICF) which describes the study, safety aspects, participant’s rights, and what happens with data/samples in the study in lay wording. Without being eligible, thoroughly understanding the trial and what it means to be a study participant, and ultimately providing consent, an individual cannot enter a study. As an example, if an inclusion criterion states that a participant must be 18 years old or older, if they are 17 years and 11 months old, they cannot be included in the study. Non-conformance with the inclusion or exclusion criteria can lead to serious consequences – not just endangering the study and the new treatment, but also jeopardizing the reputation of all stakeholders included and in the worst case, leading to serious health outcomes.
Randomized, controlled trial
When discussing clinical trials that investigate drugs or vaccines, although there are other types of research trials, mostly we refer to randomized controlled trials (RCTs), as a gold standard in clinical research. This essentially means that from a given eligible population:
a) the population sample that will be exposed to a drug/vaccine is chosen at random and
b) that the sample that will not be exposed or (to be precise) will be given a placebo/comparator will be also chosen at random.
In such a trial, where we have a group exposed to an intervention (receiving an intervention) and a group exposed to a placebo/comparator, we talk about a controlled trial. By using such a design, we want to ensure that the effect we are seeing is true and not observed by chance.
Placebo (also called a “sugar pill”) is usually a drug or vaccine that looks the same and is administered at the same frequency as the active drug/vaccine. Sometimes using a placebo in clinical research is not the optimal approach, and another active comparator that provides established benefit to study participants can be used. The comparison between placebo/comparator and active intervention groups or trial arms is often conducted in a blinded fashion i.e. participants or even study researchers do not know who received which treatment until the results are analyzed and the “code is broken”.
The individuals in the study should all exhibit similar baseline characteristics so that the only variation is the exposure to the intervention (drug, vaccine, medical device, procedure) being investigated. This is to minimize any bias when interpreting the results. Randomization procedures (i.e. distributing each individual to a study group/arm at random) will ensure that participants are allocated to study arms in a random fashion and thereby ensuring the balance between the study arms according to the participants’ baseline characteristics. These procedures serve to reduce selection bias. There is a number of important baseline characteristics, but the usual ones are sex, age, race, and any study-relevant feature (e.g. presence/absence of certain biomarkers important for the disease). Randomization strategies often employed are stratification and minimization. Stratified randomization or stratification can be used when splitting the arms into additional blocks (strata) by a factor other than the intervention (e.g. sex = male vs. female in the groups) and making sure these blocks are balanced/homogeneous. Stratification is often used if there is a plan for specific subgroup analyses (e.g. by sex). Minimization, on the other hand, is more flexible and allows dynamic/adaptive randomization to minimize the imbalance between arms taking several confounding factors into account. A confounding factor is a variable that influences both independent and dependent variables. For example, age can be a confounding factor:
- For the development/worsening of certain diseases, age can be relevant as it can contribute to worsening, in which: development/worsening of a disease represents a dependent variable = outcome we measure in the study (how development/worsening of disease is related to age).
- Age can also have influence on how well the treatment for this disease will work, in which treatment represents an independent variable = treatment that is varied between groups in the study (a particular treatment may work better in the younger population, for example).
While stratification is usually predetermined, minimization adjusts to the participants’ recruitment and works to ensure minimal imbalance, therefore useful when the sample size is small. Both have advantages and disadvantages and different uses in the trials depending on the trial goals.
Clinical trial objectives/endpoints and statistical (hypothesis) testing
The endpoints (or outcomes), determined for each study participant, are the quantitative measurements required by the objectives. Such an endpoint can be, as an example:
- the number of adverse events in the treatment group compared to the placebo over a year, or
- the survival after 5 years using a new drug in an oncology study in patients with terminal carcinoma.
Clinical trials typically have a primary objective and a corresponding endpoint, based on which the number of participants for the study (= sample size) will be calculated. Additional objectives and endpoints are secondary and they need to be tailored according to the sample size determined for the primary objective. There can be also exploratory or tertiary objectives and endpoints that do not drive the sample size calculation but help support primary and secondary objectives.
The endpoints used in a clinical trial must correspond to the scientific objectives of the study and the methods of outcome assessment should be as free of bias and as accurate as possible. Some examples of endpoints are as follows:
- Continuous measurements: blood pressures, weight – over a period of time;
- Event times: time to recurrence of cancer, survival time – until the occurrence is present.
- Ordered categories: absent, mild, moderate, severe pain.
To establish clinical outcomes and endpoints, it is important to define what hypotheses we want to investigate. In a typical clinical trial, that will be the superiority of one arm (typically a new drug) versus the control/comparator (can be placebo or another active treatment) and essentially what will be investigated is whether one arm is better than the other in a similar group of people for one or more predefined endpoints.
To evaluate whether a new drug is better than a comparator drug or placebo in a non-biased, statistically inferred way, hypothesis testing is applied at the analysis stage, for which we define a null and alternative hypothesis before the study starts:
The null hypothesis (H0) of clinical (superiority) trials states there is no true difference between the two interventions (control and active arms).
The alternative hypothesis (H1) is opposite from the H0 and states that there is a difference between the interventions for the primary endpoint.
For a statistically significant outcome indicating that the new drug is superior, we need to be able to reject the null hypothesis. This is usually done by calculating a p-value, which can be obtained by various statistical testing, depending on the type of trial.
Under the assumption that the null hypothesis is true, the p-value is a number describing how likely it is that the data would have occurred by random chance. A p-value is a number between 0 and 1 and the smaller it is, the higher the indication to reject the null hypothesis. If a p-value is lower than our significance level, we reject the null hypothesis. Typically, the significance level is predetermined to 0.05, meaning that a p-value less than 0.05 indicates strong evidence against the null hypothesis, as there is less than a 5% probability the H0 is correct (and the results are random). Therefore, we reject the null hypothesis and accept the alternative hypothesis that the two treatments are different.
When analyzing the clinical trial data and performing statistical testing, there is a risk of drawing wrong conclusions, jeopardizing the conclusion of the study. These can be minimized by careful trial planning, especially by carefully estimating the correct sample size to address the questions and choosing relevant primary endpoints. By using appropriate statistical methods to calculate an adequate sample size to answer the research questions, errors can be minimized.
The two most common types of error are Type I error = a false positive conclusion indicating that there is a treatment difference when in reality there is none, and a Type II error = a false negative conclusion indicating that there is no treatment difference when in reality there is one. Type I errors are serious and can jeopardize not just one study but can also bring reputational risks and create hesitancy towards pharmaceutical products. This would mean that we see the effect of a new drug or that a new drug is better than the old one when in reality it is not.
|Measured or perceived||True||Correct decision (true positive)||Type I error (false positive)|
|False||Type II error (false negative)||Correct decision (true negative)|
During the analysis stage, when the trial is over, the researchers need to choose how to look at the data – take all the data and analyze them or take a “cleaner” subset. Collected data can include side effects or adverse effects, blood samples, blood pressure, X-ray etc. before and after the new and control treatments are given. The Intent-To-Treat (ITT) approach (all included in the study with the intention to be treated), which takes all participants that were randomized in the study, without any exclusion, is commonly used in RCT. Such analysis does not exclude protocol deviations, i.e. any data that clearly violates what is predefined in the protocol, but the advantage is that it resembles real-world data. It does not take participant withdrawals/rejection of intervention and non-compliance into consideration and analyses. For example, if we have a participant that was supposed to have a blood sample taken 30 days post-vaccination per protocol, and their blood sample was collected only at 60 days post-vaccination, the ITT approach would include this participant in the analysis, even though the protocol interval was not respected. In contrast, the Per-Protocol Set (PPS) analysis only derives conclusions based on data that is strictly adhering to the predefined protocol standards and is more controlled. In our previous example, the participant with a blood sample collected at 60 days post-vaccination would be removed from the PPS analysis. In general, it is recommended to perform both ITT and non-ITT (PPS) analysis and if the same conclusions can be drawn on two populations, there is greater confidence in the study.
Finally, a clinical trial can have multiple analyses during the study. Sometimes, especially if the study is long and/or there are complex study designs where one part may depend on results from another part, interim analyses are performed to evaluate any current data emerging in the study before a final analysis, which encompasses the entire duration of the study, takes place.
Clinical trial blinding
Blinding or masking is an important element in clinical trials by which some or all stakeholders involved are not aware of which drug/vaccine each participant is receiving. In blinded trials, the treatments are masked so that they look the same (e.g. vaccine and placebo can be “packaged” in identical-looking syringes so no one would know the difference).
A double-blind clinical trial is one in which both the recipient and the person that gives the drug/vaccine (administrator) do not know what drug/vaccine the recipient is receiving. A single-blind clinical trial, where either only the study investigators or only the participant is blind to the drug/vaccine allocation, is also often conducted, as well as open-label studies, in which there is no blinding.
Blinding is used to prevent possible biases of those working at the clinical sites when examining participants, as well as preventing the biases of the participants themselves. Not implementing the blinding may create bias in the reporting of adverse events, especially for participants that have a preference towards any of the regimes. Moreover, if one of the treatments is placebo, blinding ensures there is no placebo effect, e.g. if one participant knows they are receiving an active vaccine and the other one is receiving a placebo vaccine, there may be bias in the way they are reporting their side effects – more severe pain at the administration site with active vaccine vs. less severe pain with placebo, whereas in reality, it may be the same.
Finally, sometimes use of a placebo and blinding is not ethical. When there is demonstrated benefit of a new drug with an acceptable safety profile/minimal side effects, and no alternative drug, all future trials must be open-label because the use of placebo is not appropriate and there is no other drug that can be used as a comparator for a disease. Similarly, if the participants are patients depending on the treatment, it is not ethical to give them a placebo – in these cases often one arm continues receiving their standard available treatment and the other gets a new drug for the same disease.
Relevant roles in clinical trials
A clinical trial is an extremely complex platform and requires the participation of many relevant stakeholders. Beyond this, it is also heavily regulated and there are commitments to continuous reporting of data to regulators, ethics committees, journals etc. Essentially, for new treatments and new clinical care to be developed, there are many partnerships established which can be defined by a contract, depending on the relationship.
The initiator of a trial is usually the sponsor, e.g. a pharmaceutical company or a research hospital/institute interested in investigating a new treatment/drug/vaccine/device (= essentially a new product or a new procedure). Beyond the development of the product itself, the sponsor usually comes up with a design of the study, protocol, and associated objectives and study requirements (such as the number of participants, their obligations, and benefits of participating in the study), data collections/analysis plan etc. They also come up with the countries in which the study should be performed (for example, the choice between the northern and southern hemisphere if relevant for the timing of seasonal vaccines: normally a vaccine, such as the influenza vaccine, would be given in winter months, which differ in the north and south) and evaluate the possible clinics/sites to participate. The sponsor can be also the main funder, but the funder can also be a non-profit organization, such as The Bill & Melinda Gates Foundation.
The staff conducting the study, having a direct touch base with the participants, and collecting the data are usually hospital clinics with experience, ready to perform such research. They usually include study nurses, coordinators, sub-investigators, and principal study investigators. Such study site teams are commonly trained in clinical study conduct and ready to be activated at a given point in time. The choice of site will depend on whether a certain country needs to be included for a specific reason but will also depend on the population being investigated: children, older adults, pregnant women, or patients. Hence, the site/team may need to be specialized in a specific branch of medicine, depending on the condition/drug being researched.
One of the most important clinical trial stakeholders is the participant, a healthy volunteer or a patient that can and is willing to participate in concrete research. Without participants, there would be no clinical trials, and ultimately, no improvement of clinical care and treatment. Hence, it is of utmost importance to create and maintain trust, transparency, and respect before, during, and after the trial with the participant. One relevant aspect of clinical trial participation is data/sample collection, analysis, and sharing and naturally, there are ever-growing concerns about data privacy. Today, data (including samples) is being anonymized and coded and is being additionally safeguarded so that the highest level of protection is applied. In addition, the participant is normally able to request withdrawal from the study, as well as the destruction of all data/samples collected, at any given point in time.
Ethics Committees (ECs), such as the IRB in the US, play an important role in the clinical trial setting. Their primary task is to ensure that trials are ethical and that the study participants are treated respectfully and are being protected. Their role is to review research documents, revise where needed, and approve or reject the study while keeping the interests of the study participant at heart. One of the most relevant documents for the ECs to revise (and equally important to participants) is the Informed Consent Form (ICF).
Even with all the abovementioned stakeholders approving the trial, the trial cannot commence without being approved by Regulatory or Competent Authorities (CAs). The Food and Drug Administration; FDA in the US, or the European Medical Agency; EMA for the EU are examples of such CAs. Each country as well has its health agency or authority and associated policies (regulations and guidance) and standards to which a trial, for which a sponsor must seek the approval, must adhere. Similar to ECs, they review, revise and approve/reject the trial, not focusing exclusively on the participant but on the entirety of the study, both on the scientific aspects and on operational conduct. Clinical trials are being regulated to support the development of medical products while at the same time assuring that the results that are generated serve the purpose of providing robust proof of safety and efficacy.
The Data Safety Monitoring Board (DSMB) and/or Data Monitoring Committee (DMC) usually serve as an external body to provide continuous oversight of the safety data collected and advise on whether the trial can continue or should be stopped at any given point in time if risks are identified. They are a group of experts in a given field of research and/or population being researched – usually physicians, scientists, and statisticians responsible to review the data in an unblinded way and monitor the quality and evaluate risk/benefit for each participant and overall for all study participants. If a study has holding rules (set of stopping rules that determine when the study has to be paused due to a possible risk) the DSMB or DMC can stop the trial.
Since clinical trials are complex and prone to ever-evolving regulations, sponsors usually seek external collaborators and services to support the trial conduct. Several vendors can be engaged to help with laboratory testing and processing, shipments, participants’ diaries (paper & electronic diaries & devices to collect adverse events), advertising and recruitment etc.
Finally, once the trial is over and the data analyzed, the results need to be published and shared publicly in predefined timelines from the last participants’ visit in the study. This is a commitment not only to regulators, study participants, and sites but also to the general public. Apart from results being shared as a commitment, usually, they are also presented in peer-reviewed journals and at congresses and conferences.
The results of several studies are also used to prepare regulatory submission packages for the product’s market authorization so that the product can be made commercially available or licensed, for example:
- the FDA’s Biologics License Application (BLA) for biological products – a request for permission to introduce, or deliver for introduction, a biologic product into interstate commerce; and the FDA’s New Drug Application (NDA) for commercialization of small molecules.
- the EMA’s Marketing Authorization Applications (MAA) – once granted by the European Commission, the centralized marketing authorization is valid in all European Union (EU) Member States, Iceland, Norway and Liechtenstein.
For reporting of clinical trial results in the medical journals, Consolidated Standards of Reporting Trials (CONSORT) are usually followed, so that the reporting in the article is standardized and appropriate for its purpose.
Phases of clinical trials
There are three main phases of clinical trials: Phases I-III. Phase IV are studies conducted after the product is authorized in a given market/country and is widely available. These studies serve to further explore the products’ safety/efficacy in real-life settings and provide additional information coming from a very large sample size (Figure 1).
Phase I: is it safe?
Phase I trials are the smallest in scope and encompass a handful of participants. They are conducted in a healthy population and therefore serve to provide an indication of the most common adverse events (AEs) in a small population. They are also called First-Time-in-Human (FTiH) studies given that the intervention is given to humans for the first time ever. The main goal of such studies is to establish the safety of the product used. Such studies usually last for several months up to 1 year.
Phase II: does it work?
Phase II trials serve to further investigate the safety of the products (risks, adverse events) in a target population (e.g. patients with a disease/condition or pregnant women) on a smaller, short-term scale compared to Phase III. The sample size is usually bigger than in Phase I studies and these studies can also serve as further dose-finding studies (to find the most appropriate dose of a product) and different treatment regimens (which frequency of treatment administered is the most appropriate). These studies usually provide a glimpse of the efficacy of the product but that is not their main goal. Phase II usually lasts for ~2 years.
Phase III: is it effective?
Phase II studies contrast Phase III trials, which are longer, large-scale trials that evaluate drug effectiveness/efficacy against a given disease in a target population. They can also evaluate if a new drug is better than what is already available. Phase III can last several years and usually comes with a high investment in terms of money and time.
It is not uncommon to also have Phase I/II or II/III design in one study usually split into stages: e.g. Part 1 of one study being Phase I and Part 2 being Phase II. These studies are usually called seamless and adaptive since they combine multiple phases under one study. The final analysis of such a study would comprise results from all participants receiving the product and comparator/placebo (if applicable) in both stages (e.g. Phase I and Phase II). The adaptive approach means that the study can be adapted in a planned fashion during the course of time, depending on the results of the planned unblinded interim analysis.
Clinical trials designs
Apart from different phases, there are also different ways clinical studies can be designed. The simplest clinical study design is a parallel design. This implies several study arms corresponding to one or more different doses of a new product and a placebo/comparator if applicable, all starting at the same time. The parallel design assumes having a large sample size to be able to assess the observed effect between several distinctive arms with statistical significance. It may be the quickest way to select the most appropriate dose as all treatments would be given at the same frequency for the full duration of the study. Parallel design can also be designed sequentially, starting with one arm with the lowest dose followed by other arms with increasing doses once the safety of the former dose is established (dose escalation). Such a study would take longer compared to non-sequential parallel design, given that each step is followed by evaluation and then subsequent step continuation.
Factorial design is appropriate for a single-dose study including two or more treatments given simultaneously. The simplest factorial design is a 2×2 design, shown here in Table 2 on an example of how plant growth is affected depending on the amount of sunlight and water the plants are receiving simultaneously. This can be extrapolated to clinical trials. Provided that groups are balanced in terms of numbers and baseline characteristics, a composite effect of multiple drugs can be investigated.
|Sunlight Exposure||Low||Plant growth||Plant growth|
|High||Plant growth||Plant growth|
Another frequently used design when testing new drugs is a crossover design, where different combinations of drug doses can be administered for a given duration, followed by an appropriate washout period (period without any exposure to the drug) and another dose/drug can be given afterward with the same duration and frequency as the first dose/drug, to the same patient. Therefore, variability between responses is minimized given that the same person is exposed to both treatments at different times. This approach cannot be used in vaccine trials because there is no appropriate washout period.
There are other designs that can be implemented in studies (for example, cluster-randomized, basket, umbrella and stepped wedge trials) but in most cases, the three types of clinical trial designs described above and represented in Figure 2, are used.
No matter which design is chosen, the study can look at the superiority of a new drug over another or placebo, non-inferiority vs. another drug, or equivalence in terms of effectiveness:
Superiority studies aim to demonstrate that a new treatment is better than an available product or placebo. When there is no alternative, only a new product, then by default this study aims to show the superiority of this new product.
Non-inferiority studies serve to demonstrate that the new treatment is at least as good as the one available. For example, it can be that a new drug is cheaper or less toxic and we want to establish that it is still as effective as the old one. Non-inferiority can also be used when combining vaccines: co-administration of two vaccines should be as good as the two vaccines given separately.
(Bio)equivalence studies look at whether the new treatment is equivalent to an old one. These types of studies are usually used when generics are registered because they need to do the same job (not better and not worse) as the drug that is branded.
Complexity of conducting clinical trials
On average, drug development takes 12 years until approval for wider use, and the lengthiest process in the development, as well as the most expensive part, are the clinical trials. Of the entire costs of product development, clinical trials account for 60% of costs. The time and investments vary according to the difficulty of the development of a given product and the urgency to bring the product to the market, but a clinical trial is overall a tremendously complex, lengthy, and laborious process. According to different sources, only around 10-15% of drugs/vaccines being investigated in clinical trials are being licensed – approved – by regulators for wider public use.
Nevertheless, science is moving forward every day and the experience and learnings from previous trials using the same or similar technology help us with achieving shorter timelines. The greatest example of this is the development of the COVID-19 vaccines, demonstrating an unprecedented product development in unprecedented times, paving the way for future clinical trials.
If you want to learn more about the specificities of vaccine clinical trials and why scientists were able to bring the COVID-19 vaccine to the market fast, stay tuned for “Clinical Trials – Part Three: Vaccines in Clinical Trials)”.
Committee on Strategies for Responsible Sharing of Clinical Trial Data; Board on Health Sciences Policy; Institute of Medicine.Washington (DC): “Sharing Clinical Trial Data: Maximizing Benefits, Minimizing Risk.” National Academies Press (US); 2015 Apr 20.
Day S. J. and Altman D. G. Blinding in clinical trials and other studies, BMJ 2000; 321:504.
AB Tasty. 2022. “Statistics: What are Type 1 and Type 2 Errors?.” Last edited on 22 September 2018. https://www.abtasty.com/blog/type-1-and-type-2-errors/
American Cancer Society. 2022. “Types and Phases of Clinical Trials.” Last edited on 18 August 2020. https://www.cancer.org/treatment/treatments-and-side-effects/clinical-trials/what-you-need-to-know/phases-of-clinical-trials.html
Basicmedical Key. 2022. “Trial Design: Overview of Study Designs (Phase I, II, III, IV, Factorial Design).” Last edited on 26 September 2017. https://basicmedicalkey.com/trial-design-overview-of-study-designs-phase-i-ii-iii-iv-factorial-design/
Certara (by Nathan Teuscher). 2022. “Trial Designs—Non-inferiority vs. Superiority vs. Equivalence.” Last edited on 1 January 2011. https://www.certara.com/knowledge-base/trial-designs-non-inferiority-vs-superiority-vs-equivalence/
European Federation of Pharmaceutical Industries and Associations (EFPIA). 2022. “Clinical Trials.” https://www.efpia.eu/about-medicines/development-of-medicines/regulations-safety-supply/clinical-trials/
Lupus Research Alliance. 2022. “On Biostatistics and Clinical Trials.” Last edited on 1 August 2019. https://lupustrials.org/about-trials/phases-of-a-trial/
PennState Eberly College of Science. 2022. “Stat 509 – Design and Analysis of Clinical Trials.” https://online.stat.psu.edu/stat509/lesson/5
Pubrica Academy. 2022. “Phases of a Clinical Trial.” Last edited on 4 February 2019. https://pubrica.com/academy/statistical/on-biostatistics-and-clinical-trials/
Hoffmann-La Roche Ltd. 2022. “What is a clinical trial and how does a trial work?.” https://www.roche.com/research_and_development/who_we_are_how_we_work/research_and_clinical_trials/what_is_a_clinical_trial.htm
Scribbr (by Lauren Thomas). 2022. “Independent and Dependent Variables | Uses & Examples.” Last edited on 27 August 2021. https://www.scribbr.com/methodology/independent-and-dependent-variables/
Statology. 2022. “A Complete Guide: The 2×2 Factorial Design.” Last edited on 13 May 2021. https://www.statology.org/2×2-factorial-design/
Surveillance, Epidemiology and End Results (SEER) Program Training,. 2022. “Blinded Clinical Trial.” https://training.seer.cancer.gov/treatment/other/trial.html
Wikipedia. 2022 “Confounding.” Last edited on 2 January 2022. https://en.wikipedia.org/wiki/Confounding
Wikipedia. 2022 “Minimization (clinical trials).” Last edited on 4 April 2020. https://en.wikipedia.org/wiki/Minimisation_(clinical_trials)
Wikipedia. 2022 “Null hypothesis.” Last edited on 5 January 2022. https://en.wikipedia.org/wiki/Null_hypothesis