Discussion Questions for the Ed Panacek Research Video Series

Research Basics

1.   What is the validated way to properly perform research? What is it called and how is it done?

  • Answer: the scientific method
    This should be described as starting with a research question or hypothesis, designing an experiment, conducting the experiment and collecting data, analyzing that data and seeing whether it answers the question or supports the hypothesis.

2.  When a study is analyzed, the degree to which the actual study measurements (data) support the conclusions of the study is called which of following:

A.   Reliability
B.   Internal validity
C.   External validity
D.   Interrater reliability
E.   Concordance

3. In performing research measurements, what is the difference between random error and systematic error? Define the two terms.

  • Random errors are simple mistakes or sloppiness that could occur in any direction, i.e. randomly. Systematic error is also referred to as "bias" and is a pattern of errors that are relatively consistent in terms of the direction and/or magnitude of the measurement error.

Research Questions

1.  List the four elements generally required for properly formatted research questions:

  • Answer: PICO
       i.    namely patients, intervention (or independent variable), comparison, outcome (dependent or outcome variable)

2.    What is the null hypothesis? Define it.

  • Answer: That there is no difference between the two study groups.
       i.    Namely that the results from arm A (data set A), when subtracted from the results of arm B (data set B), result in a "null" or empty set of data.

3. What is the alternative hypothesis? Define it.

  • Answer: That there is a difference between the two study groups
       o    namely that data A minus data set B results in some leftover number that is not equal to zero.

4. When reporting data analysis with statistical comparisons, "p" values are routinely reported. What does "p" mean?

  • Answer: P is simply an abbreviation of the word probability.
        o   It’s designed to quantify the effects of random chance on data sets and help the investigator understand whether differences observed could have happened by random chance alone.

Principles of Research Study Design

1.  There are numerous different study designs available. There also are multiple different classification systems to help understand the different designs. List and describe some different classifications systems in common use:

  • Answer: 
    • i.    Descriptive vs analytic
    • ii.   Interventional versus observational
    • iii.  Classification by time frame
            Retrospective, cross-sectional, prospective
    • iv.  Classification by degree of scientific rigor
             True experimental, quasi-experimental, non-experimental

2.  In research questions in study designs, it is important to understand the main research variables. Define and provide an example of an independent variable and a dependent variable:

  • The independent variable (IV) is that parameter that generally varies between the two groups and is the primary interest of the study. It’s also often referred to as the predictive variable or the intervention variable or the exposure variable. In therapy studies, it’s therapy A vs therapy B.
  • The dependent variable (DV)is the outcome of interest and sometimes referred to as the outcome variable. It, generally, is what is used to see whether the independent variable has an effect or makes a difference. Examples would be death rates, hospital admission rates, improvement in pain scores, etc.

3. Study designs vary in terms of their general level of scientific validity or the "strength of evidence" that they provide to support their conclusions. Organize the following study designs from highest level of scientific validity to lowest, in order: cross-sectional, cohort, case control, randomized clinical trial, retrospective case series.

  • Answer:
    1.    RCT,
    2.    prospective cohort study,
    3.    case control,
    4.    cross-sectional,
    5.    retrospective case series 
    (depending upon other study design elements, prospective cross-sectional studies might be considered to have a higher level of validity and an entirely retrospective case control study, in some situations).

True Experimental Study Designs

1.  Which of the following study design elements are not required in a "true experimental" design?

A.   Randomization
B.   Blinding
C.   Manipulation
D.   Control

2. All of the following are true experimental study designs except:

A.   Randomized clinical trials
B.   Cross-over studies
C.   Factorial studies
D.   Prospective cohort studies
E.   Group sequential

3. All of the following are disadvantages of true experimental study designs, except:

A.   Impractical for common clinical conditions
B.   Require patient consent
C.   Require randomization
D.   Are resource expensive

Cohort Studies

1. Define the term cohort and what it means in terms of clinical study design:

  • Answer: It comes from the Latin term cohors and refers to a set group of soldiers who are all of the same type. 
    • In research it means a group of individuals who all share some trait and then are followed together through time making observations looking for the outcome of interest.

2.  All of the following are examples of cohorts, except:

A.   A group of individuals all born during the same time period or generation
B.   Individuals who all share the same outcome, e.g. died from lung cancer
C.   Group of individuals all living in the same city, all working at the same location
D.   Group of individuals sharing a common exposure, e.g. to cigarette smoke or radiation

3.  All of the following are advantages to cohort studies, except:

A.   Do not need to randomize
B.   Can study multiple outcomes for each exposure of interest
C.   Ideal for studying rare outcome
D.   Prospective design allows precise measurement of exposures and outcome
E.   Generally less expensive than randomized clinical trials

Case Control Studies

1.  In terms of research design classification, case control studies fit into which of the following categories based on degree of scientific validity:

A.   True experimental
B.   Quasi-experimental
C.   Non-experimental
D.   None of the above

2.  Which of the following statements best describes the "case-control" study design?

A.   Subjects selected based upon exposure, but studied retrospectively
B.   Subjects selected based upon exposure, but studied prospectively
C.   Subjects selected based upon outcome and studied retrospectively
D.   Subjects selected based upon outcome and studied prospectively

3.  All of the following are advantages of case-control study designs except:

A.   Ideal for studying rare outcomes
B.   Can be performed entirely retrospectively
C.   Ideal for studying rare exposures (cohort studies are best for this)
D.   Less expensive to perform than randomized trials or cohort studies

Cross Sectional Studies

1.  In terms of the time frame perspective, cross sectional studies should classically be considered:

A.   Prospective
B.   Retrospective
C.   Neither
D.   Both

2.  Cross sectional studies are best and most commonly used to:

A.   Establish cause/effect relationships
B.   Establish dose response curves
C.   Establish prevalence rates
D.   Establish incidence rates
E.   Prove associations

3.   Regarding cross sectional studies, which of the following statements is not true:

A.   Each subject should be studied or measured only once
B.   All observations have to occur at the same time
C.   The denominator is directly measured at the same time as the numerator, making for accurate prevalence calculations
D.   Each subject must be studied at the equivalent time or milestone

Other Research Study Designs

1.  All of the following are examples of "descriptive" study designs except:

A.   Case reports
B.   Ecologic studies
C.   Case series
D.   Descriptive epidemiology

2.  Regarding "before-end-after" studies, which of the following statements is not true:

A.   Can be performed entirely retrospectively
B.   Can be performed entirely prospectively
C.   Is a term used to refer to changes in the environment that are under the investigator’s control
D.   Can be performed ambispective, that is both retrospectively and prospectively
E.   Are best performed with multiple measurement periods, both during the before portion, as well as during the after portion

3.  Within the spectrum of study designs, "case series" are most appropriately used to:

A.   Demonstrate cause-effect relationships
B.   Compare differences between groups
C.   Systematically describe new or rare medical experiencesD.   Calculate prevalence or incidence rates

Elements of a Study Protocol

1.  In selecting study subjects, using "convenience sampling" should be considered in which of the following:

A.   True probability sample
B.   Non-probability sample
C.   Proven representative sample
D.   Random sample

2.  Which of the following would be an example of an "effectiveness analysis" as opposed to an efficacy analysis:

A.   Evaluation under "ideal" conditions
B.   An industry sponsored Phase III clinical trial
C.   A case control study
D.   A multi-center study in "real world" conditions

3.  In establishing the "validity" or accuracy of a study measurement, which of the following is not an accepted approach:

A.   Content validity
B.   Interrater validity (this is used for reliability, not validity)
C.   Criterion validity
D.   Predictive validity

Sample Size and Power

1.  Identify (circle) each of the following parameters that are always included in performing sample size calculations:

A.   Alpha level
B.   Beta level
C.   Effect size
D.   95% confidence interval
E.   Type of study outcome data (dependent variable)

2.  In calculating a sample size, in general, if the investigator is looking to be able to detect a 50% smaller effect size (e.g. detecting a 10% instead of a 20% difference in outcome between groups), the change in required sample size would be about:

A.   Twice as many subjects
B.   Four times as many subjects
C.   The same number of subjects
D.   One-half the number of subjects
E.   One-fourth the number of subjects

3.  Which of the following changes will not require a larger sample size:

A.   Decreasing the Beta level from 0.2 to 0.1
B.   Decreasing the Alpha level from 0.05 to 0.25
C.   Studying an outcome (dependent) variable that is more common
D.   Trying to detect a smaller effect size or difference between groups

Click here to
send us feedback