Common mistake series: sample size estimation

Written by: Dian Luthfiana Sufyan

“Not the large, nor the small sample define the scientifically sound research,

it is the proper sample size estimation”

This discussion is fueled by the fact that several common mistakes have frequently emerged in the research protocols submitted for review. Perhaps some of you are still wondering why Research Ethics Committee (REC) bothers with research methodology. Doesn’t REC only need to consider the ethical aspect of the study? If you prompted as so, then kindly refer to the previous article titled An Ethically Justified and Scientifically Sound Research.

Scientific and ethical aspects are intertwined in the matter of research since every decided action of one aspect influences another. Consequently, both are reviewed once the protocols are submitted prior to the study conduct. However, many researchers remain using incorrect approaches to estimate the required sample size, which can result in incorrect statistical inferences and study conclusions.

To be more understandable, author elaborates this issue in a deductive way, starting with a basic concept of why we need sample in the first place, followed with how does study objective would influence sample, what is the main considerations of sample size formula and what are the common mistakes done by researchers.

• The principle - Why do we need sample?

We conduct research to understand population. Thus, ideally, we have to study the overall population to draw exact conclusions of a particular parameter. However, this is almost unachievable as the resources are not always there, whether it is limited time, restricted funding, finite equipment and materials, or inadequate study personnel. It can be also due to vast scattered population that prevents measuring the whole population. Therefore, researchers work with samples. Nevertheless, these samples should accurately and precisely represent the population. Sample validity is highly governed by the selection process of the subject or so-called sampling technique (which will be elaborated in the next series), while sample precision is highly determined by the sample size. So essential, that it should be calculated with a proper formula that matches its main study objective, research design, and type of data.

•  Study objective driven - how this sample are being defined if the study objectives are varied?

Research can be done with various objectives. Be it to estimate proportion, such as the prevalence of certain medical or malnutrition conditions, or estimate mean, for instance, the investigation on hemoglobin concentration of a specific population group. Research also can be carried out to test a hypothesis, such as to establish the difference between groups of interventions or conditions in terms of predefined parameters like risk factors, side effects, or expected beneficial effects of these interventions. Any kind of objectives a study has, a proper sample size estimation is required, otherwise, it will lead to inconclusive results. Smaller sample size sometimes is preferable in the view of researcher convenience, yet, it could lead to insufficient power to detect differences and cause the type II error (failure to reject null hypothesis when it is not true). Small sample size is only justified for rare case reports or qualitative studies when the depth of the data overweighs the quantity. Larger sample size than required is also unnecessary, as it waste of resources and is considered unethical for exposing subject to pointless conduct.

• Factors to consider during sample size calculation

There are several main considerations when calculating sample size such as a-error, β-error, effect size, and standard deviation. a-error also called a Type-1 error defined as failure to accept null hypothesis when it is actually true, conventionally set at 5%. The lower the a-error being expected, the more sample size is needed. β-error or type-2 error is described as failure to reject null hypothesis when it is actually not true. Most research set it at 20%, 10%, or 5%. This type of error influences power of the study. As power is equal to 1-type 2 error, thus most research sets at least 80% of power. The higher the expected power, the larger the sample size required. Effect size, often referred as meaningful differences, involves µ1 minus µ 2 in numerical data or p1 minus p2 in quantitative data, and should be considered when conducting hypothesis testing. Further, standard deviation depicts the variability if the type of the data is numerical. In addition, a 10% of attrition rate mostly added to the final sample size to safeguard the response rate and possibility of loss to follow up.

•  Common mistakes

These are several common mistakes found in submitted protocols:

a. No sample size calculation was performed

Either researcher miss to put the calculation on the protocols or none calculation take place.

b. Assumption-based sample size

Researcher make a reckless assumption solely based on prior studies with a different setting and context.

c. Wrong formula

As aforementioned, sample size calculation should be matched to main study objective and research design. For instance, if the general objective of the study is to estimate population proportion, then the following formula can be implemented:

P = estimated proportion

D = absolute margin of error

Z = z-score for confidence level

 Formula for estimation population mean:

 s = standard deviation

D = absolute margin of error

Z = z-score for confidence level

 Formula for hypothesis testing difference between two proportions:

  P = proportion depend on the research design; differed between cross-sectional/cohort and case-control study design.

 Formula for hypothesis testing difference between two means

 Further consideration of effect size is needed if a complex sample is performed. For additional reading on the use of other sample size formulas in health research, WHO published an e-book that can be accessed here. Likewise, a web-based sample size calculator also found to be useful can be retrieved here.

In summary, sample size calculation is paramount in any study. Understanding the study objective and research design is the entry point for selecting a proper sample size calculation. On top of that, sample size calculation must take into account funding availability, facilities, available data, and ethics of subjecting human to research. Researchers must put detailed information regarding sample size calculation when submitting study protocols, otherwise, having the abovementioned common mistake will only prolong the review time.

Dian Sufyan is a lecturer and researcher at Nutrition Department of UPNVJ, reviewer board member of Research Ethics Committee UPNVJ, 2020 SEAMEO RECFON affiliated researcher, DAAD alumnae and currently a Postgraduate Research Student at Human Nutrition Department, University of Glasgow 

References

Ariawan, I. (2019). Sample size calculation and Sampling procedure. Presented on: Postgraduate Training on Survey Planning, 2 May 2019. SEAMEO RECFON, Faculty of Medicine, University of Indonesia: Jakarta.

Gogtay, N. (2010). Principles of sample size calculation. Indian Journal of Ophthalmology, 58(6), 517–518. https://doi.org/10.4103/0301-4738.71692

Lemeshow, S., Hosmer, D. W., Klar, J., & Lwanga, S. K. (1991). Adequacy of Sample Size in Health Studies. In World Health Organization (Vol. 47, Issue 1). John Wiley & Sons Ltd. https://doi.org/10.2307/2532527

Nayak, B. K. (2010). Understanding the relevance of sample size calculation. Indian Journal of Ophthalmology, 58(6), 469–470. https://doi.org/10.4103/0301-4738.71673

Survey Monkey. (2023). Sample size calculator. https://www.surveymonkey.com/mp/sample-size-calculator/