Sunday, November 13, 2011

All Studies Are Not Created Equal


Before discussing specific studies related to precocious puberty, it is important to understand that not all studies are created equal.


Often people see results from a study reported in the media and they conclude, "Research has shown ..." without any assessment of the design, strengths and weaknesses of that particular analysis.

Conclusions drawn from a well-designed study have more credibility than a poorly designed one --, but yet often the media don't give any indication of the quality of the studies on which they report. This can be a serious disservice to the public who often take what they read at face value and make health-related decisions based on that.

One aspect of the public health field of epidemiology is the study of risk factors, things that increase the chances of something happening, such as disease. For example, smoking is a risk factor for lung cancer. There are numerous study designs to test the relationship between risk factors and disease. The following observational study types are listed according to the hierarchy from least dependable strength of evidence to most dependable. 

Least Dependable: Case Report
A case report describes the symptoms, signs, diagnosis, treatment, and follow-up of an individual patient with PP (for example). There may be many reasons for that patient's experience. Thus, one case study does not give analytic evidence regarding what is a risk factor for a PP within the broader population.

Case Series
A case series is a descriptive report of a group of patients. Similar to the case report, this is descriptive rather than analytical by nature and is not rigorous, e.g., does not report on enough patients, to be able to scientifically identify risk factors for a disease.

Ecologic Studies
Ecologic studies base their results on data at the population-level (e.g. average childhood obesity rates within the U.S.) rather than individual-level data (presence of obesity in individual girls). The problem with this is that conclusions that are true for averages may not help us understand what is true for the individual. For example, it seems that many girls who develop PP are not in fact obese, so what led to PP for them?

Cross-Sectional Study
Cross-sectional studies look at potential risk factors and outcomes, using data collected just at one point in time. With this approach, there is the question of which comes first, the risk factor or the PP. All you can see is that both are found at a particular point in time.

Case-Control Study
Case-control studies identify cases, people who have the outcome of interest (e.g., girls who have precocious puberty) and controls (girls who have not developed precocious puberty) and looks backwards in time using medical records and other collected data using tools like patient surveys to test which factors or characteristics have a higher association with PP.

This is an improvement from cross-sectional studies, because at least you can look for risk factors that happened before PP was diagnosed.

Most Dependable: Cohort Study
Cohort studies - also called panel studies, use data collected on a group of people over time to determine the effect of different type of risk factors on developing the outcome of interest. To test the associations between risk factors and PP, this would be the ideal study design, i.e. a large sample of girls chosen to participate from the time they are born going forward, looking to see who develops PP and who does not.

Ultimately, to prove causality, an experimental study, rather than an observational ones, gives the best evidence. The gold standard of experimental studies is the randomized controlled trial (RCT), during which one group is exposed in a highly controlled way to a particular exposure and the other group is not. However ethical considerations often make conducting such studies on humans (particularly children) a non-option. Researchers are therefore left to make conclusions based on observational studies or based on experimental studies on animals -- a practice that has its own ethical and methodological limitations.

As I look at specific studies about precocious puberty, I will address some of these study design points, such as:

1. The type of study -- and how much we should rely on its conclusions, based on its design
2. How many people were studied, i.e., the sample size
3. Did the study include only those with precocious puberty (cases), or were those who did not have precocious puberty (controls) also included for scientific comparison?
4. Do the conclusions made in the study match the evidence, taking into account these factors as well as other potential biases in the study?

I would love to see a long-term cohort study that collects data about children beginning with their mother's pregnancies with them and including information related to all of the potential risk factors for precocious puberty (genetics, environmental exposures, nutrition, family environment, etc).  As far as I know this study does not yet exist.   

1 comment:

Anonymous said...

Thanks for authoring this blog. My 8 yo daughter (adopted from China) has CPP. Diagnosed when she was 7 yo. We opted for monthly Lupron injections. Before her symptoms and diagnosis, I had never heard of precocious puberty and since her diagnosis, haven't been able to find information about causes and long-term effects of delaying it. I've bookmarked this blog. Thanks again for what you are doing.