Analysis of Repeated Measures Will G Hopkins, Auckland University

Analysis of Repeated Measures Will G Hopkins, Auckland University of Technology, Auckland, NZ A tutorial lecture presented at the 2003 annual meeting of the American College of Sports Medicine This presentation applies to continuous or ordinal numeric dependent variables, including data from most Likert scales. It does not apply to nominal dependent variables or variables representing counts or frequencies. Make sure you view this presentation as a full slide show, to get the benefit of the build-up of information on each slide.

OVERVIEW Basics What change has occurred in response to a treatment/intervention? Analysis by ANOVA, within-subject modeling, mixed modeling. Fixed and random effects; individual responses and asphericity. Accounting for Individual Responses What is the effect of subject characteristics on the change? Analyzing for Patterns of Responses What is the treatment's effect on trends in repeated sets of trials? Analyzing for Mechanisms How much of the change was due to a change in whatever?

Basics What change has occurred in response to a treatment or intervention?

Basics: Interventions A repeated measure is a variable measured two or more times, usually before, during and/or after an intervention or treatment. Y Dependent variable Repeated measure Period of treatment exptal control Data are means and standard deviations pre mid post Group Trial or Time Between-subjects factor Different subjects on each level Within-subjects factor Same subjects on each level Analysis by ANOVA, t statistics and within-subject modeling, and mixed modeling.

Basics: Analysis by ANOVA Data are in the form of one row per subject: Select columns to define a within-subjects factor. If there is no control group, use a 1-way repeatedmeasures ANOVA The 1 way is Trial: "(How) does Trial affect Y?" Measure "Y" within-subjects factor "Trial" Girl Group Ypre Ymid Ypost Ann exptal 58 62 68 Bev exptal 45 . 57 Lyn control 39 May control 44 42 45 40 42 Missing value means loss of subject. With a control group, use a 2-way repeated-measures ANOVA. The 2 ways are Group and Trial. You investigate the interaction Group Trial: "(How) does Trial affect Y differently in the different groups?"

Basics: Analysis by t Statistics and Within-Subject Modeling If there is no control group, use a paired t statistic to investigate changes between interesting measurements. Girl Ann Bev Lyn May Ypost Group Ypre Ymid Ypost -Ypre exptal 58 62 68 10 exptal 45 . 57 12 control control 39 44 42 45 40 42 1 -2 Missing value does not affect post – pre changes With a control group, calculate change scores and use the unpaired t statistic to investigate the difference in the changes. Use un/paired t statistics for other interesting combinations of repeated measurements. I call it within-subject modeling. Example: time course of an effect

Basics: More Within-Subject Modeling To quantify a time course: Ann fit lines or curves to each subject's points; predict interesting things for each subject; analyze with un/paired t statistic. Method #1. Fit lines Y a b.T At Time 0 and 3, Y a and a 3b. Change in Y b per week. Bev Missing value no problem. Y Lyn Method #2. Fit quadratics Y a b.T c.T2 At Time 0 and 3, Y a and a 3b 9c. Change in Y 3b 9c over 3 weeks. Maximum occurs at Time -b/(2c). Method #3. Fit exponentials Y a b.eT/c Needs non-linear curve fitting to estimate time constant c. May 0 1 2 3 Time (wk)

Basics: Analysis by Mixed Modeling Data are in the form of one row per subject per trial: Girl Ann Analysis is via maximizing Ann likelihood of observed values rather than ANOVA's approach Ann Bev of minimizing error variance. Bev Bev Group exptal exptal exptal exptal exptal exptal Trial pre mid post pre mid post Y 58 62 68 45 . 57 You investigate fixed effects: Trial, if there's only one group. Group Trial, if there's more Lyn control pre 39 than one group. Lyn control mid 42 You also specify and estimate Missing value means loss of random effects. only one trial for the subject. "Mixed" fixed random. Some mixed models are also known as hierarchical models.

Basics: Fixed Effects Fixed effects are differences or changes in the dependent variable that you attribute to a predictor (independent) variable. They are usually the focus of our research. Their value is the same (fixed) for everyone in a group. They have magnitudes represented by differences or changes in means. Example of difference in means: girls' performance 48 boys' performance 56 so effect of sex (maleness) on performance 56 – 48 8. Example of change in a mean: girls' performance in pretest 48 girls' performance after a steroid 56 so effect of the steroid on girls' performance 56 – 48 8.

Basics: Random Effects Random effects have values that vary randomly within and/or between individuals. They provide confidence limits or p values for the fixed effects. They provide other valuable information usually overlooked. They are mostly hidden in ANOVA, are accessible in t tests, and are up front in mixed modeling. They are the key to understanding repeated measures. They have magnitudes represented by standard deviations (SD). Examples of between-subject SD or random effects: Variation in ability: SD of girls' performance (Y) 9.2 Individual responses: SD of effect of a steroid on Y 5.0, so you can say the effect of the steroid is 8.0 5.0 (mean SD). Example of a within-subject SD or random effect: Error of measurement: SD of any girl's Y in repeated tests 2.0

Basics: The "Hats" Metaphor for Random Effects When you measure something, it's like adding together numbers drawn from several hats. Each hat holds a zillion pieces of paper, each with a number. The numbers are normally distributed with mean 0, SD ? Example: measure a girl's performance several times. Suppose true mean performance of all girls 48.3 A girl's true performance (not observed) 48.3 7.4 55.7 SD 9.2 A girl's observed performance in Trial #1 in Trial #2 55.7 2.1 57.8 55.7 -1.3 54.4 The random effects in SAS are Girl and Girl Trial ( the SD 2.0 residuals).

Basics: Hats plus a Fixed Effect Example: give steroid with a fixed effect of 8.0 between Trials #1 and #2, and measure several girls. Performance in Trial #1 Performance in Trial #2 Ann 55.7 2.1 57.8 55.7 -1.3 8.0 62.4 Bev 48.4 -3.1 45.3 48.4 0.7 8.0 57.1 Cas 65.2 -2.8 SD 62.4 2.0 65.2 -1.4 8.0 71.8 40.7 0.5 SD 41.2 2.0 40.7 2.8 8.0 51.5 Deb Subject hat not shown. SD 2.0 SD 2.0 These are all we can observe. The stats program uses them to estimate the fixed and random effects.

Basics: A Hat for Individual Responses Example: different responses to the steroid. Performance in Trial #1 Performance in Trial #2 Ann 55.7 2.1 57.8 55.7 -1.3 8.0 5.2 67.6 Bev 48.4 -3.1 45.3 48.4 0.7 8.0 -0.5 56.6 Cas 65.2 -2.8 SD 62.4 2.0 65.2 -1.4 8.0 6.2 40.7 0.5 SD 41.2 2.0 40.7 2.8 8.0 -2.7 Deb 78.0 SD 5.0 48.8 SD 5.0 SD 2.0 SD 5.0 SD 2.0 SD 5.0 To estimate the SD for individual responses, you need a control group (see later) or an extra trial for the treatment group.

Basics: Individual Responses and Asphericity It's important to quantify individual responses, but More importantly, they are the most frequent reason for the asphericity type of non-uniform error in repeated measures. You must somehow eliminate non-uniformity of error to get trustworthy confidence limits or p values. Here's the deal on asphericity. Conventional ANOVA is based on the assumption that there is only one random-effects hat, error of measurement. We can use ANOVA for repeated measures by turning the subjects random effect into a subjects fixed effect. But it doesn't work properly when there is asphericity: that is, more than one source of error, such as individual responses. There are four approaches to the asphericity problem.

Basics: Dealing with Asphericity in Repeated Measures Four approaches: MANOVA (multivariate ANOVA) (Univariate) ANOVA with adjustment for asphericity Within-subject modeling with the unequal-variances t statistic Mixed modeling I base my assessment of these approaches mainly on my experience with the Statistical Analysis System (SAS). Other stats programs may produce different output.

Basics: MANOVA/adjusted ANOVA for Asphericity (NOT!) Both these approaches involve different assumptions about the relationship between the repeated measurements. They produce an overall p value for each fixed effect. Incredibly, the p value is too small if sample size and individual responses differ between groups. Adjusted ANOVA (Greenhouse-Geisser or Huynh-Feldt) is worse than MANOVA. Subjects with any missing value are first deleted. So there is needless loss of power, if the missing value is for a minor repeated measurement (e.g., post2). In the old-fashioned approach, you are allowed to "test for where the difference is" only if the overall p 0.05. So there is further loss of power, because you could fail to detect an effect on the overall p or the subsequent test.

Basics: More on MANOVA/adjusted ANOVA The overall p value is OK when the extra random effects are the same in both groups, even when sample sizes differ. Example: two repeated-measures factors; for example, several measurements on one day repeated at monthly intervals. The program then does p values for the requested contrasts (differences in the changes; e.g., post – pre for exptal – control). These comparisons are simply equal-variance t tests. So the p values are too small if sample size and individual responses differ between groups. There is no adjustment other than Bonferroni for inflation of Type I error for contrasts involving repeated measures. Good! But researchers still dial up Tukey or other adjustments and think that the resulting p values are adjusted. They're not. In summary: avoid MANOVA and adjusted ANOVA.

Basics: Unequal-Variances t Statistic Deals with Asphericity Example: controlled trial of effect of the steroid on performance. Variance of post–pre change scores: exptal Y SD 2.0 8 33 SD2 4 SD2 4 post Random effects: SD2 4 SD2 4 SD2 25 control pre Big differences in variances. So use unequal-variances t statistic to analyze changes. Bonus: estimate of individual responses as an SD (SDChgExpt2 – SDChgCont2) SD 5.0

Basics: Summary of t Statistic for Repeated Measures Advantages It works! It's robust to gross departures from non-normality, provided sample size is reasonable. 10 in each group is forgiving, 20 is very forgiving. Missing values are not a problem. Because you analyze separately the changes of interest. Students can do most analyses with Excel spreadsheets. Include my spreadsheet for confidence limits and clinical/practical/mechanistic probabilities. You can include covariates by moving to simple ANOVAs or ANCOVAs of the change scores. Example: how does age modify the effect of the steroid on performance? (See later.) But

Basics: More on t Statistic for Repeated Measures Disadvantages ANOVAs or ANCOVAs of the change scores aren't strictly applicable, if variances of the change scores differ markedly. You can't easily get confidence limits for the SD representing individual responses. That is, I don't have a formula or spreadsheet yet. There's always bootstrapping, but it's hard work. The disdain of editors and peer reviewers, most of whom think state of the art is repeated-measures ANOVA with post-hoc tests controlled for inflation of Type I error. In conclusion, I recommend within-subject modeling using unequal-variances t statistic for analysis of straightforward data. Otherwise use mixed modeling

Basics: Mixed Modeling for Asphericity You take account of potential sources of asphericity by including them as random effects. Advantages It works! Impresses editors and peer reviewers. Confidence limits for everything. Complex fixed-effects models are relatively easy: individual responses, patterns of responses, mechanisms Disadvantages Not available in all stats programs. Takes time and effort to understand and use. The documentation is usually impenetrable. Sample size for robustness to non-normality not yet known.

Accounting for Individual Responses What is the effect of subject characteristics on the change?

Individual Responses: and Subject Characteristics Subjects differ in their response to a treatment boys girls Y Data are values for individuals pre mid post Trial pre mid post due to subject characteristics interacting with the treatment. It's important to measure and analyze their effect on the treatment. Using value of Trial as a characteristic needs special approach pre to avoid artifactual regression to the mean. See newstats.org. Use mixed modeling, ANOVA, or within-subject modeling.

Individual Responses: by Mixed Modeling You include subject characteristics as covariates in the fixedeffects model. The SD representing individual responses will diminish and represent individual responses not accounted for by the covariate. The precision of the estimates of the fixed effects usually improves, because you are accounting for otherwise random error. Covariates can be nominal (e.g., sex) or numeric (e.g., age). Example: how does sex affect the outcome? First, you can avoid covariates by analyzing the sexes separately. Effect on females 8.8 units; effect on males 4.7 units. Effect on females – males 8.8 – 4.7 4.1 units. You can generate confidence limits for the 4.1 "manually", by combining confidence limits of the effect for each sex. Include individual responses for each sex: 8.8 5.2; 4.7 2.5.

Individual Responses: More Mixed Modeling The full fixed-effects model is Y Group Trial Sex Group Trial. The term Sex Group Trial yields the female-male difference of 4.1 units (90% confidence limits 1.5 to 6.7, say). The overall effect of the treatment (from Group Trial) is for an average of equal numbers of females and males. Try including random effects for individual responses in males and females. Example: how does age affect the outcome? Either: convert age into age groups and analyze like sex. Or: if the effect of age is linear, use it as a numeric covariate. Age Group Trial provides the outcome as effect per year: 1.3 units.y-1 (90% confidence limits -0.2 to 2.8). Note that the overall effect of the treatment is for subjects with the average age.

Individual Responses: by Repeated-Measures ANOVA It is possible in principle to include a subject characteristic as a covariate in a repeated-measures ANOVA. But SPSS (Version 10) provides only the p value for the interaction. Incredibly, it does not provide magnitudes of the effect. If a covariate accounts for some or all of the individual responses, the problem of asphericity will diminish or disappear. I don't know whether it's possible to extract the SD representing individual responses from a repeated-measures ANOVA, with or without a covariate.

Individual Responses: by Within-Subject Modeling Calculate the most interesting change scores or other withinsubject parameters: Kid Ann Ben Lyn Merv Ypost Sex Age Group Ypre Ymid Ypost -Ypre F 23 exptal 58 62 68 10 4 M 19 exptal 64 67 68 F M 19 control 19 control 39 59 42 60 40 57 1 -1 If no control group, analyze effect of subject characteristics on change score with unpaired t, regression, or 1-way ANOVA. With a control group, analyze with 2-way ANOVA. As before, a characteristic that accounts partially for individual responses will reduce the problem of asphericity.

Analyzing for Patterns of Responses What is the effect of a treatment on trends within repeated sets of trials?

Patterns of Responses: Bouts within Trials Typical example: several bouts for each of several trials. 1 Y exptal 2 3 control pre mid Trial post 4 Bout Standard deviations: Between Subjects within Bout Within Subject between Trials Within Subject within Trial We want to estimate the overall increase in Y in the exptal group in the mid and post trials, and the greater decline in Y in the exptal group within the mid and post trials (representing, for example, increased fatigue). Use mixed modeling, ANOVA, or within-subject modeling.

Patterns of Responses: by Mixed Modeling and ANOVA With mixed modeling, Bout is simply another (withinsubject) fixed effect you add to the model. The model is Y Trial Bout Trial Bout. Bout can be nominal or numeric. If numeric, Bout specifies the slope of a line, and Trial Bout specifies a different slope for each level of Trial. Add Bout Bout( Trial) to the model for quadratic(s). Elegant and easy, when you know how. With ANOVA, you have to specify Bout as a nominal effect and try to take into account within-subject errors using adjustments for asphericity. Specifying a quadratic or higher-order polynomial Bout effect is possible but difficult (for me, anyway). Within-subject modeling is much easier

Patterns of Responses: by Within-Subject Modeling The trick is to convert the multiple Bout measurements into a single value for each subject, then analyze those values. In the example, derive the Subject: JC Bout mean and slope (or any other parameters) Y within each trial for each subject. Derive the change in mean pre mid post and the change in slope Trial between pre and post (or any other Trials) for each subject. For the changes in the mean, do an unpaired t test between the exptal and control groups. Ditto for the changes in the slope. Simple, robust, highly recommended!

Analyzing for Mechanisms How much of the change was due to a change in whatever?

Analyzing for Mechanisms Mechanism variable something in the causal path between the treatment and the dependent variable. Necessary but not sufficient that it "tracks" the dependent. Dependent variable Mechanism variable exptal control control pre mid exptal post Trial pre mid post Important for PhD projects or to publish in high-impact journals. It can put limits on a placebo effect, if it's not placebo affected. Can't use ANOVA; can use graphs and mixed modeling.

Mechanisms: Why not ANOVA? For ANOVA, data have to be one row per subject: Measure "Y" Mechanism variable within-subjects factor "Trial" (within-subjects covariate) Girl Ann Bev Group Ypre Ymid Ypost Xpre Xmid Xpost exptal 58 62 68 8.4 8.7 9.1 exptal 45 . 57 9.0 . 9.7 Lyn May control control 39 44 42 45 40 42 7.9 7.1 7.7 7.1 You can't use ANOVA, because it doesn't allow you to match up trials for the dependent and covariate. 7.8 7.2

Mechanisms: Analysis Using Graphs Choose the most interesting change scores for the dependent and covariate: Change score for dependent Girl Ann Bev Ypost Xpost Group Ypre Ymid Ypost Xpre Xmid Xpost -Ypre -Xpre exptal 58 62 68 8.4 8.7 9.1 10 1.5 exptal 45 . 57 9.0 . 9.7 12 0.7 Lyn May control control 39 44 42 45 40 42 Then plot the change scores 7.9 7.1 7.7 7.1 7.8 7.2 1 -2 -0.1 0.1 Change score for covariate

Mechanisms: More Analysis Using Graphs Three possible outcomes with a real mechanism variable: 1. Large individual responses tracked by mechanism variable even in the control group. exptal Ypost - Ypre 0 control 0 Xpost - Xpre The covariate is an excellent candidate for a mechanism variable.

Mechanisms: More Analysis Using Graphs Three possible outcomes with a real mechanism variable: 2. Apparently poor tracking of individual responses but it could be due to noise in either variable. Ypost - Ypre 0 0 Xpost - Xpre The covariate could still be a mechanism variable.

Mechanisms: More Analysis Using Graphs Three possible outcomes with a real mechanism variable: 3. Little or no individual responses but mechanism variable tracks mean response. Ypost - Ypre 0 0 Xpost - Xpre The covariate is a good candidate for a mechanism variable.

Mechanisms: Graphical Analysis – how NOT to Relationship between change scores is often misinterpreted. "The correlation between change scores for X and Y is trivial. Therefore X is not the mechanism." Ypost – Ypre 0 0 Xpost – Xpre 0 "Overall, changes in X track changes in Y well, but Noise may have obscured tracking of any individual responses. Therefore X could be a mechanism variable."

Mechanisms: Quantitative Analysis by Mixed Modeling - 1 Need to quantify the role of the mechanism variable, with confidence limits. I have devised a method using mixed modeling. Data format is one row per trial: Girl Ann Ann Ann Bev Mechanism variable (within-subjects covariate) Group Trial exptal pre exptal mid exptal post exptal pre Y 58 62 68 39 X 8.4 8.7 9.1 9.0 No problem with aligning trials for the dependent and covariate.

Mechanisms: More Quantitative Analysis by Mixed Modeling Run the usual fixed-effects model to get the effect of the treatment. Example: 4.6 units (90% likely limits, 2.1 to 7.1 units). Then include a putative mechanism variable in the model. The model is then effectively a multiple linear regression, so You get the effect of the treatment with the mechanism variable held constant which means the same as the effect of the treatment not explained by the putative mechanism variable. Example: it drops to 2.5 units (90% likely limits, -1.0 to 7.0 units). So the mechanism accounts for 4.6 - 2.5 2.1 units. If the experiment was not blind, the real effect is 2.1 units and the placebo effect is 2.5 units. provided the mechanism variable itself is not placebo affectible!

Summary Basics Use the unequal-variance t statistic and within-subject modeling for straightforward models. Repeated-measures ANOVA may not cope with non-uniform error. Mixed modeling is best for fixed and random effects. Accounting for Individual Responses Use within-subject modeling or mixed modeling. Analyzing for Patterns of Responses Use within-subject modeling or mixed modeling. Analyzing for Mechanisms Interpret graphs of change scores properly. Use mixed modeling to get estimates of the contribution of a mechanism variable.

This presentation was downloaded from: A New View of Statistics newstats.org SUMMARIZING DATA GENERALIZING TO A POPULATION Simple & Effect Precision of Statistics Measurement Dimension Reduction Confidence Limits Statistical Models Sample-Size Estimation