REVIEW ARTICLE |
https://doi.org/10.5005/jp-journals-10009-1800 |

# Best Practices in the Analysis of Ultrasonographic Research Data: *Ancora Imparo*

Department of Obstetrics and Gynecology and Office of Faculty Development, Paul L. Foster School of Medicine, Texas Tech University Health Sciences Center El Paso, El Paso, Texas, USA; Department of Public Health, Texas Tech University Health Sciences Center, Lubbock, Texas, USA

**Corresponding Author:** Zuber D Mulla, Texas Tech University Health Sciences Center El Paso, MSC 21007, 5001 El Paso Drive, El Paso, Texas, USA, Phone: +1-915-215-5075; e-mail: zuber.mulla@ttuhsc.edu

## ABSTRACT

**Aim:** To briefly review several regression models and epidemiological research strategies that are of interest to women healthcare professionals who are engaged in scholarship involving data from ultrasound imaging studies.

**Background:** Advances in statistical methods in epidemiology in the past two decades can aid clinician investigators. However, these recent developments in research methods may not be well-known outside the disciplines of biostatistics and epidemiology.

**Review results:** Several types of regression models are discussed including log-binomial regression and quantile regression. Modern methods for the analysis of repeated measures data including generalized estimating equations are reviewed. Finally, the utility of directed acyclic graphs (DAGs), a type of causal diagram, is introduced. Directed acyclic graphs are useful in identifying confounders and avoiding a variety of biases such as overadjustment bias and collider-stratification bias.

**Conclusion:** Data arising from ultrasound imaging studies provide a wealth of scholarly opportunities for clinicians. The application of sound, modern statistical techniques will ensure the design and conduct of high-quality research investigations.

**Clinical significance:** Physicians using ultrasound may encounter variables with a skewed distribution such as nuchal translucency or a dataset in which the dependent variable, such as an umbilical artery Doppler index, is measured multiple times. Special methods are required to analyze such datasets properly. Clinician researchers, especially early-career faculty, should consider collaborating with biostatisticians and epidemiologists.

**How to cite this article:** Mulla ZD. Best Practices in the Analysis of Ultrasonographic Research Data: Ancora Imparo. Donald School J Ultrasound Obstet Gynecol 2021;15(4):340–346.

**Source of support:** Nil

**Conflict of interest:** None

**Keywords:** Collider-stratification bias, Confounding, Directed acyclic graphs, Generalized linear models, Longitudinal data analysis, Overadjustment bias, Quantile regression, Ultrasound, Women’s health.

## BACKGROUND

Renaissance polymath Michelangelo is believed to have uttered the words, “Ancora imparo,” Italian for, “Still, I am learning,” at the age of 87 years.^{1} These words of wisdom can serve to inspire clinicians and scientists as they expand their knowledge of data analysis throughout their careers.

The primary goal of this article is to introduce several techniques that are commonly used when analyzing ultrasound data as a part of a clinical or epidemiologic investigation. The objective is not to provide an exhaustive review of these methods but to motivate the reader to seek collaboration with a biostatistician or epidemiologist if needed when designing studies.

This article also discusses recent developments in the use of causal diagrams that are of interest to clinicians and scientists working in the area of women’s health. SAS 9.4 computer code is also provided for selected regression models. Following statistical convention, every reference to a log in this article refers to the natural logarithm (the log raised to the base *e*) rather than a log raised to the base 10.

## REVIEW RESULTS

### Popular Regression Methods

*Generalized Linear Models (GLM)*

Generalized linear models (GLMs) are frequently used in the analysis of data in the health sciences. They allow for the regression analysis of independent observations in which the outcome is either continuous or discrete.^{2} The data analyst specifies both a distribution and a link function when fitting a GLM. Table 1 reports the details of several GLMs. The reader is no doubt familiar with two popular GLMs: the linear regression model and the logistic regression model.^{3}

Regression model |
Outcome |
Possible application |
Distribution |
Link function |
Comments |
---|---|---|---|---|---|

Gamma | Continuous | Identifying factors associated with hospital length of stay. | Gamma | Inverse or log | While the inverse is the canonical link function, the data analyst may choose to specify a log link function depending on the requirements of the analysis (see article by Lee et al.^{11}). |

Linear | Continuous | Identify predictors of body mass index. | Normal | Identity | One of the assumptions of the linear regression model is that the residuals are normally distributed. |

Log-binomial | Dichotomous | Calculate prevalence ratios for having diabetes or risk ratios for developing diabetes. | Binomial | Log^{*} |
Less stable (may not converge) than the logistic regression model. |

Logistic (binary) | Dichotomous | Calculate odds ratios for preeclampsia. | Binomial | Logit | Continuous independent variables may need to be categorized as this model assumes the logit varies in a linear fashion with the independent variable. |

Negative binomial | Count | Identify factors associated with the number of antral follicles. | Negative binomial | Log | Usually preferred over the Poisson regression model due to overdispersion. |

Poisson | Count | Identify predictors of the number of times a patient presented to the physician’s office. | Poisson | Log | Frequently plagued by overdispersion and hence the negative binomial regression model may be a better option. |

Zero-inflated Poisson | Count | This model may be indicated when many subjects have a value of 0 for the outcome. | Zero-inflated Poisson | Log | Compare the results of this model with those from the zero-inflated negative binomial regression model (not shown in this table). |

To fit a logistic regression model the researcher or data analyst will specify a binomial distribution and a logit link. Logit (pronounced “low-jit”) is the natural logarithm of the odds of the outcome. Every statistical test or model makes one or more assumptions. For logistic regression, a key assumption is that the logit varies in a linear fashion with the independent variable when that independent variable is continuous.^{4} An example illustrating this concept follows.

Arya et al. reported on the role of three-dimensional pelvic ultrasound in the assessment of factors associated with intrauterine device (IUD) misplacement and dislocation.^{5} In their logistic regression analysis, the binary outcome was IUD displacement: displaced IUD versus IUD not displaced. One of the possible risk factors that these authors evaluated was the uterine cavity transverse diameter. Figure 1 is a previously unpublished plot of the logit of IUD displacement versus the uterine cavity transverse diameter using data from the study by Arya et al.^{5} The reader will note that the association between the uterine cavity transverse diameter and the log odds of IUD displacement is not linear. Given this nonlinear relationship, the authors wisely created a categorical variable using the original continuous variable of uterine cavity transverse diameter. Balise has written an SAS macro which will create logit plots.^{6}

Logistic regression models produce odds ratios when one is analyzing data from clinical trials, cross-sectional prevalence studies, retrospective cohort studies, prospective cohort studies, and traditional case-control studies. While logistic regression remains a popular tool when modeling binary response data in the health sciences, the best modern practice is to avoid using logistic regression unless the odds ratio is a good approximation to the risk ratio or the prevalence ratio.^{7} Experts instead recommend fitting log-binomial regression models to binary outcome data when the risk ratio or the prevalence ratio is the actual parameter of interest.^{7},^{8} While both the logistic and the log-binomial models require the specification of the binomial distribution, their link functions differ. The data analyst will use the log link with a binomial distribution in order to fit a log-binomial regression model (Table 1).

This author prefers to reserve the use of the logistic regression model to the analysis of data arising from a case-control study. However, at times, one may encounter a sparse data situation when analyzing binary outcome data from a design other than a case-control study such as a prospective cohort study with count (rather than person-time) denominators. In these situations, the author will fit either an exact conditional logistic regression model or perform logistic regression using penalized maximum likelihood estimation.^{4},^{9}

Generalized linear models allow for the analysis of a wide range of outcome data including counts. Clinicians specialized in reproductive medicine are interested in counting ovarian antral follicles using ultrasound.^{10} Factors associated with ovarian antral follicle count could be identified using one of the several GLMs including Poisson regression, negative binomial regression, and zero-inflated Poisson regression (Table 1).

Researchers may be interested in identifying predictors of a skewed continuous variable such as hospital length of stay (measured in days) using gamma regression (Table 1). The distribution of length of stay is positively skewed and hence the data analyst would most likely transform the outcome before fitting a linear regression. An alternative to linear regression in this setting would be to specify a gamma distribution with a log link.^{11} The reader is advised that the canonical (natural) link function for the gamma distribution is the inverse^{1},^{2} however, in this example of modeling length of stay, using the logarithmic link function (rather than the inverse link function) will allow the analyst to compare the parameter estimates from the gamma regression model with those from the linear regression model on the same scale.^{11}

### Properly Measuring Interaction

A brief discussion about assessing interaction is warranted at this point. Scientists may be interested in assessing the interaction (effect measure modification) between two variables, say, a family history of preeclampsia (PE) and the patient’s obesity status, while studying the outcome of systolic blood pressure among a cohort of pregnant women. In the setting of a linear regression model, the evaluation of a possible interaction between the main effects of a family history of PE and obesity would involve the creation of a product term: the data analyst would multiply the family history term and the obesity term. Creating product terms is perfectly acceptable when assessing biologic interaction using linear regression models yet the use of product terms in an effort to explore a possible interaction in a logistic regression model is controversial.^{13}

Including a product term in a linear regression model allows for the evaluation of a departure from additivity. However, Rothman advises against creating product interaction terms in regression models that use logarithmic transformations (such as logistic regression) because the inclusion of such terms amounts to an assessment of a departure from a multiplicative model rather than a departure from additivity.^{13} Including a product term in a logistic or log-binomial regression model will not allow the data analyst to properly assess the presence of a biologic interaction.^{13} Rothman’s solution to this dilemma is to create a single composite exposure (predictor) variable.^{13}

To illustrate Rothman’s approach, assume that a group of clinicians is following a cohort of pregnant women all of whom have completed at least 20 weeks’ gestation. The binary outcome is the development of PE. These investigators will calculate risk ratios for PE using a log-binomial regression model. The two main effects are both binary variables: a family history of PE and obesity status. Instead of multiplying these two variables, as is done in a linear regression analysis, the proper approach to evaluating a biologic interaction between these variables is to create a four-level composite variable as follows: Positive family history of PE and obese, Positive family history of PE and not obese, Negative family history of PE and obese, and Negative family history of PE and not obese (referent category). This four-level composite variable would be modeled using three dummy (indicator) variables.

### Quantile Regression

Quantile regression is a valuable tool for the analysis of continuous response data. This model appears to be underutilized by researchers in medicine. While linear regression estimates the average value of the outcome variable for given levels of the independent variables, quantile regression analysis models the association between the set of independent variables and specific percentiles (quantiles) of the outcome variable.^{14} Quantile regression is robust to extreme values of the outcome variable.^{15}

One of the assumptions of linear regression is that the residuals are normally distributed. Clinicians analyzing data from ultrasound imaging studies may encounter variables that do not follow a normal (Gaussian) distribution. For example, the distribution of nuchal translucency (NT) is skewed to the right (positively skewed).^{16} If a team of clinicians and researchers desired to quantify the changes in the median NT value as a function of maternal age in years, then the QUANTREG Procedure in SAS could be used to accomplish this task.^{15},^{17}

The SAS code that is found below will fit a quantile regression model where the outcome is NT and the predictor (independent variable) is maternal age in years (represented by AGE). Note that SAS, by and large, is not case sensitive; however, in this article user-generated variable names will be capitalized to allow the reader to easily substitute their own variables.

The parameter estimate for AGE represents the change in a specific quantile of NT produced by a one-unit change in AGE.^{14} The following program requests the SAS software package to model the relationship between AGE and the 5th, 10th, 50th (median), 90th, and 95th percentiles of NT:

ods graphics on;

proc quantreg ci=resampling;

model NT = AGE / quantile=0.05 0.10 0.50 0.90 0.95 seed=1268

plot=quantplot;

run;

ods graphics off;

The reader is referred elsewhere for details about the SAS syntax.^{17}

### Repeated Measures/Longitudinal Data Analysis

Clinicians may measure an outcome such as the umbilical artery pulsatility index multiple times in each patient.^{18} When a dependent variable is measured multiple times in a longitudinal study, then special techniques are required to accommodate the correlated nature of the outcome data. To clarify, researchers analyzing repeated measures data need to use methods that account for the statistical dependence among the repeated measurements within their study subjects.^{19} Ignoring the dependency (the correlated nature of the data) can lead to erroneous resulting including invalid *p* values.^{20} This section will briefly discuss two modern approaches to analyzing repeated measures data: generalized estimating equations (GEEs), and random-effects regression models. The goal is to motivate the reader to seek assistance, if needed, when designing a longitudinal study. An older technique for analyzing longitudinal data, which has now fallen out of favor is repeated measures analysis of variance. There are several limitations of repeated measures analysis of variance including the requirement that the outcome must be continuous and its inability to accommodate time-varying covariates.^{20} Given these and other disadvantages of repeated measures analysis of variance, it will not be discussed further.

### Repeated Measures Analysis of Binary Outcomes Using GEE

The first modern technique that will be introduced is GEE.^{2},^{19} GEE logistic regression models are marginal models that allow the data analyst to estimate population-averaged effects.^{19} In other words, if your goal is inference about group differences then use GEE rather than fitting a random-effects regression model.^{19} GEE treats the correlated nature of the outcome as a nuisance.^{19} Hu et al. write that the correlation or dependency, “…between repeated measures is taken into account by robust estimation of the variances of the regression coefficients.”^{19} Assume that a team of clinicians and maternal health epidemiologists are following a cohort of pregnant women. The binary outcome is systolic hypertension (SHTN). The definition of SHTN is not important for this example. The outcome variable is coded as follows: 1 = SHTN is present, 0 = SHTN is absent. The outcome will be measured five times during the study. The team’s database will contain five records for each subject in the study. The risk factor of interest is systemic lupus erythematosus (SLE) coded as follows: 1 = present, 0 = absent. This team of investigators will fit the following GEE logistic regression model:

where *μ** _{ij}* is the mean of the response (the expectation),

*x*

*denotes SLE status (1 or 0 as described above) for subject*

_{i}*i*, and

*t*

*is the time corresponding to the*

_{ij}*j*th measurement for subject

*i*. The time variable in this hypothetical study will take on the values of 1, 2, 3, 4, and 5. Additional predictors, including time-varying covariates such as body mass index and smoking status, can be added to Equation 1.

The following SAS code will fit a GEE logistic regression model:

proc genmod descending;

class SUBJECT_ID;

model SHTN = SLE / d=b;

repeated subject = SUBJECT_ID / type=exch corrw;

estimate ’Odds ratio for SHTN: SLE vs. No SLE’ SLE 1/exp;

run;

The descending option in the proc genmod statement above instructs SAS to model the probability that SHTN is equal to 1 (rather than 0). Various working correlation matrix structures can be specified. In the example above, the SAS keyword exch requests the exchangeable working correlation matrix.

If the reader will be collaborating with statistical consultants during the design of a longitudinal study, then an important question to pose to the consultants is what type of standard errors will be used for inference: the sandwich estimator (also known as the empirical or robust standard error), or model-based standard errors. For a study with a large sample size, the empirical (robust) standard error is preferred over the model-based standard error.^{2},^{21}

The GEE approach is ideal in this example if the researchers are interested in the averaged effect of SLE on SHTN regardless of the change over time in the prevalence of SHTN in the individual. But if the researchers are interested in the differences in the increasing or decreasing trends of the frequency of SHTN between the SLE and non-SLE groups, then using the random-effects approach is warranted.

Repeated measures analysis of binary outcomes using random-effects regression models: Another approach to accommodating the statistical dependence that arises in a repeated measures study is to fit random-effects regression models.^{2},^{19} While GEE logistic regression is a population-averaged approach, random-effects logistic regression is a subject-specific approach.^{19} Random-effects models are also referred to as mixed models because they have both fixed and random effects.^{22} The time dependency problem is handled by adding a random intercept term to the model. The data analyst can also include random slopes; that is, random effects for the predictors and the time variable. In the SHTN example that was introduced above if the researchers were interested in the differences in the changing trends in SHTN over time between the group of patients with SLE and the group free of SLE, then fitting a random-effects model estimating the changes in individuals’ SHTN across time would be appropriate. A simple random-effects logistic regression model can be written as:

where *b*_{0}* _{i}* is a random intercept.

^{19},

^{22}This random intercept is allowed to vary with the study subjects.

^{19}The variable

*x*

*denotes SLE status (1 or 0 as described above) for subject*

_{i}*i*, and

*t*

*is the time corresponding to the*

_{ij}*j*th measurement for subject

*i*. Equation 2 does not contain any randomly varying slopes, but it can easily be extended to allow individuals to have both randomly varying intercepts and randomly varying slopes.

^{22}

Regardless of which approach the researcher opts for, GEE or random-effects regression, determining if there is an interaction between the predictor variable (e.g., the treatment variable in a clinical trial) and the time variable in a longitudinal study is an important task. The reader is referred elsewhere for information on this critical task.^{2},^{22} Briefly, if a statistically significant treatment group-by-time interaction is detected, then additional analyses are needed.^{22}

### Causal Diagrams

The assessment of causation has consumed epidemiologists for many years.^{23} Causal inference in the health sciences has been aided by the use of causal diagrams. The popularity of causal diagrams in epidemiologic and clinical research has increased during the past approximately 20 years.^{24},^{26}

Directed acyclic graphs (DAGs), a type of causal diagram, are versatile and should be in the research methods tool kit of every clinical and public health researcher. DAGs have been used to select variables for confounder control,^{25} identify potential biases including M-bias,^{27} and illustrate a phenomenon known as the Table 2 fallacy.^{28},^{29} The following section will demonstrate the use of DAGs in avoiding the triggering of two types of biases: overadjustment bias and collider-stratification bias.

### Overadjustment Bias

Overadjustment bias occurs when one controls for an intermediate variable that is on the causal pathway from the independent variable (the exposure) to the outcome.^{30}Figure 2 is a DAG which demonstrates overadjustment bias. Prepregnancy obesity has been linked to certain birth defects.^{31} Additionally, an association between maternal obesity and GDM has been reported.^{32} Finally, Ramos-Arroyo et al. found that infants of insulin-treated mothers with gestational and chronic diabetes had a higher risk of developing certain birth defects.^{33} Given these results, a team of researchers may create the DAG that is shown in Figure 2 to encode their knowledge and beliefs about this system of variables.

In Figure 2, maternal prepregnancy obesity is the exposure of interest. The outcome is selected birth defects. Gestational diabetes is on the causal pathway between prepregnancy obesity and birth defects. In other words, gestational diabetes is an intermediate variable on the path between the exposure and the outcome.

The total effect of prepregnancy obesity on birth defects can be decomposed into its causal components, that is, the direct effect, indirect effect, or both.^{29} Direct effects are not mediated. The direct effect of maternal obesity on birth defects is the causal effect of maternal obesity on birth defects that is not mediated through gestational diabetes (Fig. 2). The direct effect in Figure 2 is represented by the arc at the top of the DAG leading directly from maternal obesity to birth defects. The indirect effect of maternal obesity on birth defects is the portion of the exposure effect that is mediated by gestational diabetes.

Given the information displayed in Figure 2, gestational diabetes should not be controlled for in this analysis (i.e., adjusted for using traditional regression techniques, stratification, matching, etc.). In this example, the researcher can consistently estimate the total causal effect of prepregnancy obesity on the outcome (birth defects) using standard regression methods by ignoring the intermediate variable which is gestational diabetes.^{30} However, if the researcher controls for gestational diabetes, then the total causal effect of prepregnancy obesity on birth defects cannot be consistently estimated.^{30}

Overadjustment bias can also occurs if one controls for a descending proxy of an intermediate variable. The reader is referred to Schisterman et al. for these details.^{30}

## COLLIDER-STRATIFICATION BIAS

Collider-stratification bias, a type of selection bias, is an important yet underappreciated source of error in some epidemiological studies.^{34} Collider-stratification bias occurs when one controls for (adjusts for) a collider. The adjustment method is irrelevant and could be restriction, stratification, regression modeling, or matching. In the setting of a DAG, a collider is a variable (other than the outcome) that has two arrows pointing into it.^{25} A fork in the road is a point where two paths diverge. In a DAG an inverted fork is a point where two paths converge. A collider is the variable that is found in the middle of an inverted fork.^{27}

Collider-stratification bias will be illustrated using the classic example of the birth-weight paradox. The birth-weight paradox has been studied by epidemiologists for many years.^{35},^{36}Figure 3 shows a DAG in which the exposure is maternal smoking during pregnancy, the intermediate variable is having a low birth weight infant, and the outcome is infant mortality. Variable *U* is an unmeasured variable that impacts both low birth weight and infant mortality such as a birth defect.

Before the details of the birth-weight paradox are revealed, a short discussion of confounding is merited. In Figure 3, maternal age is a confounder of the association between maternal smoking and infant mortality. A confounder is a variable that is related to the exposure and the outcome and is not in the causal pathway between the exposure and the outcome.^{13} We see in Figure 3 that this final requirement of what constitutes a confounder is met since maternal smoking (the exposure) does not affect one’s chronological age. Confounders should be controlled for.

Returning to the paradox at hand, it has been noted that among low birth weight infants, maternal smoking was associated with a reduced risk of infant mortality.^{36} This counterintuitive relationship has been dubbed the birth-weight paradox. This protective effect of maternal smoking is a result of controlling for birth weight (an intermediate) without controlling for the confounding of the intermediate-outcome association by variable *U* (Fig. 3).

In Figure 3 we see that birth defects (unmeasured variable *U*) is a cause of low birth weight and infant mortality. Birth defects were not controlled for in this analysis. Among mothers who smoke during their pregnancy and have low-birth weight infants, low birth weight may be a result of either her smoking or the presence of a birth defect.^{36} But among the mothers who are nonsmokers and have low-birth weight infants, the low birth weight cannot be a consequence of maternal smoking so some other risk factor for infant mortality, such as a birth defect, must be present.

Low birth weight is a collider on the path from maternal smoking to infant mortality. It is in the middle of an inverted fork. Controlling for birth weight (without controlling for birth defects) triggers collider-stratification bias. According to Bandoli et al., this type of bias can, “…move the observed effect in the opposite direction of the true effect.”^{25} In this scenario (Fig. 3) if the researchers are interested in estimating the overall effect of maternal smoking on infant mortality, then they should not control for (condition on) an intermediate.^{36}

Another dramatic example of collider-stratification (Berksonian) bias that is found in the literature of the 1970s and early 1980s concerns the effect of exogenous estrogens on the risk of developing endometrial cancer.^{37} Both the exposure variable (exogenous estrogens) and the outcome (endometrial cancer) can increase the woman’s risk of uterine bleeding, and therefore any investigation studying this possible association that restricted its sample to women with vaginal bleeding or to women who sought treatment for vaginal bleeding could suffer from collider-stratification bias which could reduce the observed relative risk several fold.^{37} Vaginal bleeding in this scenario is a collider: Exogenous estrogens bleeding endometrial cancer. To clarify, if one controls for a variable that is a common effect of the exposure and the outcome, then collider-stratification bias may occur.^{38}

## DISCUSSION

Data arising from ultrasound imaging studies provide a wealth of scholarly opportunities for clinicians. The application of sound, modern statistical techniques will ensure the design and conduct of high-quality research investigations. This article briefly discussed selected methodological tools such as GEE and causal diagrams that can aid the reader when planning and interpreting research investigations that involve the use of ultrasound imaging.

## CONCLUSION

Continually expanding one’s study design and data analysis skills throughout the professional lifespan is highly desirable. Mervyn Susser was a respected preventive medicine physician, public health scientist, and former Chair of the Division of Epidemiology at Columbia University in New York, USA.^{39} Dr Susser once wrote, “Statisticians and epidemiologists are properly professional skeptics.”^{40} As women’s healthcare professionals continue on their journey of lifelong learning, they would be well-served by having such a collaborator (the professional skeptic) as a travel companion.

## CLINICAL SIGNIFICANCE

Physicians using ultrasound may encounter variables with a skewed distribution such as NT or a dataset in which the dependent variable, such as an umbilical artery Doppler index, is measured multiple times. Special methods are required to analyze such datasets properly. Clinician researchers, especially early career faculty, should consider collaborating with biostatisticians and epidemiologists.

## REFERENCES

1. College of Human Sciences, Auburn University. Available from: https://wp.auburn.edu/italynew/2018/ancora-imparo/. Accessed May 12, 2021.

2. Fitzmaurice GM, Laird NM, Ware JH. Applied Longitudinal Analysis, 2nd ed. Hoboken, New Jersey: John Wiley & Sons, Inc.; 2011.

3. Allison PD. Logistic Regression Using the SAS® System: Theory and Application. Cary, North Carolina: SAS Institute, Inc.; 1999.

4. Hosmer DW, Lemeshow S. Applied Logistic Regression, 2nd ed. New York: John Wiley & Sons, Inc.; 2000.

5. Arya S, Mulla ZD, Nguyen TN, et al.Role of three-dimensional pelvic ultrasound in the assessment of risk factors for intrauterine device misplacement and dislocation. Donald Sch J Ultrasound Obstet Gynecol 2019;13(03):103–109. DOI: 10.5005/jp-journals-10009-1598

6. Balise RR. Logit plot macro (for SAS). Available from: https://web.stanford.edu/~kcobb/courses/hrp261. Accessed March 23, 2018.

7. Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol 2005;162(03):199–200. DOI: 10.1016/s1047-2797(01)00278-2

8. Robbins AS, Chao SY, Fonseca VP. What’s the relative risk? A method to directly estimate risk ratios in cohort studies of common outcomes. Ann Epidemiol 2002;12(07):452–454. DOI: 10.1016/s1047-2797(01)00278-2

9. Fernandez NP, Mulla ZD. Avoiding sparse data bias: an example from gynecologic oncology. J Registry Manag 2012;39(04):167–71.

10. Coelho Neto MA, Ludwin A, Borrell A, et al. Counting ovarian antral follicles by ultrasound: a practical guide. Ultrasound Obstet Gynecol 2018;51(01):10–20. DOI: 10.1002/uog.18945

11. Lee AH, Gracey M, Wang K, et al. A robustified modeling approach to analyze pediatric length of stay. Ann Epidemiol 2005;15(09):673–677. DOI: 10.1016/j.annepidem.2004.10.001

12. Fox J. Chapter 15. Generalized linear models. In: Applied Regression Analysis and Generalized Linear Models, 2nd ed. Los Angeles, California: Sage Publications; 2008. pp. 379–424.

13. Rothman KJ. Epidemiology: An Introduction. New York: Oxford University Press; 2002.

14. Despa S. Cornell University, Cornell Statistical Consulting Unit. StatNews #70: Quantile Regression. November 2007, Updated 2012. Available at https://www.cscu.cornell.edu/news/statnews/stnews70.pdf. Accessed May 7, 2019.

15. Chen C. Paper 213-30. An Introduction to Quantile Regression and the QUANTREG Procedure. Available at: https://support.sas.com/resources/papers/proceedings/proceedings/sugi30/213-30.pdf. Accessed May 7, 2019.

16. Vale SH, Huttly WJ, Wald NJ. Antenatal screening for Down’s syndrome: revised nuchal translucency upper truncation limit due to improved precision of measurement. J Med Screen 2021;28(2):88–92. DOI: 10.1177/0969141320937321

17. SAS Institute, Inc. The QUANTREG Procedure. SAS/STAT® 9.3 User’s Guide. Available at https://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#statug_qreg_sect008.htm. Accessed May 12, 2021.

18. Acharya G, Wilsgaard T, Berntsen GK, et al. Reference ranges for serial measurements of umbilical artery Doppler indices in the second half of pregnancy. Am J Obstet Gynecol 2005;192(03):937–944. DOI: 10.1016/j.ajog.2004.09.019

19. Hu FB, Goldberg J, Hedeker D, et al. Comparison of population-averaged and subject-specific approaches for analyzing repeated binary outcomes. Am J Epidemiol 1998;147(07):694–703. DOI: 10.1093/oxfordjournals.aje.a009511

20. Schober P, Vetter TR. Repeated measures designs and analysis of longitudinal data: if at first you do not succeed-try, try again. Anesth Analg 2018;127(02):569–575. DOI: 10.1213/ANE.0000000000003511

21. Hanley JA, Negassa A, Edwardes MD, et al. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol 2003;157(04):364–375. DOI: 10.1093/aje/kwf215

22. Fitzmaurice GM, Ravichandran C. A primer in longitudinal data analysis. Circulation 2008;118(19):2005–2010. DOI: 10.1161/CIRCULATIONAHA.107.714618

23. Karhausen LR. Causation: the elusive grail of epidemiology. Med Health Care Philos 2000;3(01):59–67. DOI: 10.1023/a:1009970730507

24. Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology 1999;10(01):37–48.

25. Bandoli G, Palmsten K, Flores KF, et al. Constructing causal diagrams for common perinatal outcomes: benefits, limitations and motivating examples with maternal antidepressant use in pregnancy. Paediatr Perinat Epidemiol 2016;30(05):521–528. DOI: 10.1111/ppe.12302

26. Mulla ZD, Pathak IS. Sleep apnea and poor COVID-19 outcomes: beware of causal intermediates and colliders. Am J Respir Crit Care Med 2021;203(10):1325–1326. DOI: 10.1164/rccm.202101-0088LE

27. Luque-Fernandez MA, Schomaker M, Redondo-Sanchez D, et al. Educational note: paradoxical collider effect in the analysis of non-communicable disease epidemiological data: a reproducible illustration and web application. Int J Epidemiol Erratum in: Int J Epidemiol 2019 Apr 1;48(02):640–653. DOI: 10.1093/ije/dyy275

28. Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. Am J Epidemiol 2013;177(04):292–298. DOI: 10.1093/aje/kws412

29. Bandoli G, Palmsten K, Chambers CD, et al. Revisiting the Table 2 fallacy: a motivating example examining preeclampsia and preterm birth. Paediatr Perinat Epidemiol 2018;32(04):390–397. DOI: 10.1111/ppe.12474

30. Schisterman EF, Cole SR, Platt RW. Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology 2009;20(04):488–495. DOI: 10.1097/EDE.0b013e3181a819a1

31. Waller DK, Shaw GM, Rasmussen SA, et al. Prepregnancy obesity as a risk factor for structural birth defects. Arch Pediatr Adolesc Med 2007;161(08):745–750. DOI: 10.1001/archpedi.161.8.745

32. Chu SY, Callaghan WM, Kim SY, et al. Maternal obesity and risk of gestational diabetes mellitus. Diabetes Care 2007;30(08):2070–2076. DOI: 10.2337/dc06-2559a

33. Ramos-Arroyo MA, Rodriguez-Pinilla E, Cordero JF. Maternal diabetes: the risk for specific birth defects. Eur J Epidemiol 1992;8(04):503–508. DOI: 10.1007/BF00146367

34. Griffith GJ, Morris TT, Tudball MJ, et al. Collider bias undermines our understanding of COVID-19 disease risk and severity. Nat Commun 202011(01):5749. DOI: 10.1038/s41467-020-19478-2

35. Hernández-Díaz S, Schisterman EF, Hernán MA. The birth weight “paradox” uncovered? Am J Epidemiol 2006;164(11):1115–1120. DOI: 10.1093/aje/kwj275

36. VanderWeele TJ, Mumford SL, Schisterman EF. Conditioning on intermediates in perinatal epidemiology. Epidemiology Erratum in: Epidemiology 2012;23(1):1–9. DOI: 10.1097/EDE.0b013e31823aca5d

37. Rothman KJ, Greenland S, Lash T. Berksonian bias. In: Modern Epidemiology, 3rd ed. Philadelphia, Pennsylvania: Lippincott Williams & Wilkins; 2008, pp. 135–136.

38. Cole SR, Platt RW, Schisterman EF, et al. Illustrating bias due to conditioning on a collider. Int J Epidemiol 2010;39(02):417–420. DOI: 10.1093/ije/dyp334

39. Susser M. A conversation with Mervyn Susser. Interview by Nigel Paneth. Epidemiology 2003;14(06):748–752. DOI: 10.1097/01.ede.0000091648.75674.24

40. Susser M. Judgement and causal inference: criteria in epidemiologic studies Am J Epidemiol 1977;105(01):1–15. DOI: 10.1093/oxfordjournals.aje.a112349

________________________

© The Author(s). 2021 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.