Your Easy Guide to Clinical Studies

Type I error and power

In a hypothesis testing framework, you need to define the type I error and the power. The definition of these 2 parameters is usually done together with the statistician.

 

Definition Type I error: also known as a false positive, as it entails the rejection of the null hypothesis when it is actually true. It represents a situation where the researcher concludes that there is a significant effect when, in reality, there is no such effect (e.g. a cholesterol-lowering drug has an effect on cholesterol level when in reality there is no such effect). Type I error is often denoted by the symbol α (alpha). Type I error is classically set at 5%.

 

Defintion of Power: it is the probability of correctly rejecting the null hypothesis (e.g. a cholesterol-lowering drug has an effect on cholesterol levels). It is classically set at 80 or 90%.

 

Aspects to consider:

• It is important for the study SP-INV to understand how important it is to do a proper sample size calculation, and the associated risks when sample size is underestimated (i.e. the study might lack the power to answer the study question, or a potential treatment effect might be missed)
• The process to get to the number of participants needed for the study bears the risk of frustration for both, the SP-INV and the statistician. The SP-INV just wants to know the number that must be included in a proposal/protocol/grant application, while the statistician cannot provide a number before having received the required information form the SP-INV needed to perform a SSC. A good interdisciplinary collaboration and understanding between study SP-INV and statistician is crucial
• The availability and ability to recruit the required number of study participants can be very challenging
• Each additionally recruited participant increases work load and study budget

Example of a hypothesis testing framework:

We are interested in a lab value (e.g. cholesterol concentration): we want to compare this value in two groups (e.g. intake of cholesterol-lowering drugs versus intake of placebo), and test the significance of this difference.

Thanks to your medical knowledge and previous work we know that:

1. A difference of 20 (or more) between both groups is clinically relevant
2. The standard deviation for this lab value can be assumed as equal to 12 in each group

 

We want to achieve a power of 90% with a two-sided alpha (type I error) of 5%

• Framework: hypothesis testing
• Comparison: mean comparison between intervention and control
• Outcome: Continuous,
• Effect we want to detect: difference of 20
• SD = 12 in both groups

 

Example of a Precision based framework:

We want to estimate the proportion of deaths within 30 days of a Myocardial Infarction (MI). A previous study estimated that 36% (31% - 40%) of patients died within 30 days of an MI. We want to replicate their study with the same precision.

This short description contains the main information:

• Framework: precision based,
• Outcome: binary (i.e., occurrence of MI),
• Expected value: 36%,
• Precision we want to achieve: CI width = 9%

• Basel, Departement Klinische Forschung, CTU, dkf.unibas.ch

• Lugano, Clinical Trials Unit, CTU-EOC, www.ctueoc.ch

• Bern, Clinical Trials Unit, CTU, www.ctu.unibe.ch

• Geneva, Clinical Research Center, CRC, crc.hug.ch

• Lausanne, Clinical Research Center, CRC, www.chuv.ch

• St. Gallen, Clinical Trials Unit, CTU, www.kssg.ch

• Zürich, Clinical Trials Center, CTC, www.usz.ch