Skip to contents

Factorial Data Contrasts

Source: vignettes/FactorialDataContrasts.Rmd
FactorialDataContrasts.Rmd

This page examines a two-factor between-subjects (factorial) design using raw data input, focusing on comparisons and contrasts.

Preliminary Tasks

Data Entry

This code inputs the variable names and creates a viewable data frame.

FactorA <- c(rep(1, 20), rep(2, 20), rep(3, 20))
FactorA <- factor(FactorA, levels = c(1, 2, 3), labels = c("A1", "A2", "A3"))
FactorB <- c(rep(1, 10), rep(2, 10), rep(1, 10), rep(2, 10), rep(1, 10), rep(2, 10))
FactorB <- factor(FactorB, levels = c(1, 2), labels = c("B1", "B2"))
Outcome <- c(6, 8, 6, 8, 10, 8, 10, 9, 8, 7, 5, 9, 10, 9, 11, 4, 11, 7, 6, 8, 7, 13, 11, 10, 13, 8, 11, 14, 12, 11, 7, 8, 7, 11, 10, 7, 8, 4, 8, 10, 9, 16, 11, 12, 15, 13, 9, 14, 11, 10, 8, 6, 8, 11, 5, 7, 9, 3, 6, 7)
FactorialData <- construct(FactorA, FactorB, Outcome)

Summary Statistics

This code obtains the descriptive statistics as two separate tables, each corresponding to a simple effect.

(FactorialData) |> focus(FactorB == "B1") |> focus(Outcome~FactorA) |> describeMoments()

Summary Statistics for the Data

         N       M      SD    Skew    Kurt
A1  10.000   8.000   1.414   0.000  -0.738
A2  10.000  11.000   2.211  -0.617  -0.212
A3  10.000  12.000   2.449   0.340  -1.102
(FactorialData) |> focus(FactorB == "B2") |> focus(Outcome~FactorA) |> describeMoments()

Summary Statistics for the Data

         N       M      SD    Skew    Kurt
A1  10.000   8.000   2.449  -0.340  -1.102
A2  10.000   8.000   2.000  -0.417   0.735
A3  10.000   7.000   2.211   0.000   0.665

These statistics can also be obtained using a simpler function call, a procedure that is then paralleled for the subsequent analyses.

(Outcome ~ FactorA) |> describeMoments(by = FactorB)
$B1

Summary Statistics for the Data

         N       M      SD    Skew    Kurt
A1  10.000   8.000   1.414   0.000  -0.738
A2  10.000  11.000   2.211  -0.617  -0.212
A3  10.000  12.000   2.449   0.340  -1.102


$B2

Summary Statistics for the Data

         N       M      SD    Skew    Kurt
A1  10.000   8.000   2.449  -0.340  -1.102
A2  10.000   8.000   2.000  -0.417   0.735
A3  10.000   7.000   2.211   0.000   0.665

Analyses of the Means

This section produces analyses that are equivalent to one-sample analyses separately for each level of a factor.

Confidence Intervals

This code will provide tables of confidence intervals for each level of the factor.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> estimateMeans()
$B1

Confidence Intervals for the Means

       Est      SE      df      LL      UL
A1   8.000   0.447   9.000   6.988   9.012
A2  11.000   0.699   9.000   9.418  12.582
A3  12.000   0.775   9.000  10.248  13.752


$B2

Confidence Intervals for the Means

       Est      SE      df      LL      UL
A1   8.000   0.775   9.000   6.248   9.752
A2   8.000   0.632   9.000   6.569   9.431
A3   7.000   0.699   9.000   5.418   8.582

This code will produce a graph of the confidence intervals for each level of the factor.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> plotMeans()

The code defaults to 95% confidence intervals. This can be changed if desired.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> estimateMeans(conf.level = .99)
$B1

Confidence Intervals for the Means

       Est      SE      df      LL      UL
A1   8.000   0.447   9.000   6.547   9.453
A2  11.000   0.699   9.000   8.728  13.272
A3  12.000   0.775   9.000   9.483  14.517


$B2

Confidence Intervals for the Means

       Est      SE      df      LL      UL
A1   8.000   0.775   9.000   5.483  10.517
A2   8.000   0.632   9.000   5.945  10.055
A3   7.000   0.699   9.000   4.728   9.272

For the graph, it is possible to add a comparison line to represent a population (or test) value and a region of practical equivalence in addition to changing the confidence level.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> plotMeans(conf.level = .99, line = 9, rope = c(8, 10))

Significance Tests

This code will produce a table of NHST separately for each level of the factor. In this case, all the means are tested against a value of zero.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> testMeans()
$B1

Hypothesis Tests for the Means

      Diff      SE      df       t       p
A1   8.000   0.447   9.000  17.889   0.000
A2  11.000   0.699   9.000  15.732   0.000
A3  12.000   0.775   9.000  15.492   0.000


$B2

Hypothesis Tests for the Means

      Diff      SE      df       t       p
A1   8.000   0.775   9.000  10.328   0.000
A2   8.000   0.632   9.000  12.649   0.000
A3   7.000   0.699   9.000  10.011   0.000

Often, the default test value of zero is not meaningful or plausible. This too can be altered (often in conjunction with what is presented in the plot).

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> testMeans(mu = 9)
$B1

Hypothesis Tests for the Means

      Diff      SE      df       t       p
A1  -1.000   0.447   9.000  -2.236   0.052
A2   2.000   0.699   9.000   2.860   0.019
A3   3.000   0.775   9.000   3.873   0.004


$B2

Hypothesis Tests for the Means

      Diff      SE      df       t       p
A1  -1.000   0.775   9.000  -1.291   0.229
A2  -1.000   0.632   9.000  -1.581   0.148
A3  -2.000   0.699   9.000  -2.860   0.019

Standardized Effect Sizes

This code will produce a table of standardized mean differences separately for each level of the factor. In this case, the mean is compared to zero to form the effect size.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> standardizeMeans()
$B1

Confidence Intervals for the Standardized Means

         d      SE      LL      UL
A1   5.657   1.251   3.005   8.295
A2   4.975   1.111   2.622   7.312
A3   4.899   1.096   2.579   7.203


$B2

Confidence Intervals for the Standardized Means

         d      SE      LL      UL
A1   3.266   0.771   1.644   4.863
A2   4.000   0.915   2.068   5.911
A3   3.166   0.752   1.586   4.721

Here too it is possible to alter the width of the confidence intervals and to establish a more plausible comparison value for the mean.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> standardizeMeans(mu = 9, conf.level = .99)
$B1

Confidence Intervals for the Standardized Means

         d      SE      LL      UL
A1  -0.707   0.364  -1.614   0.222
A2   0.905   0.384  -0.083   1.873
A3   1.225   0.422   0.126   2.317


$B2

Confidence Intervals for the Standardized Means

         d      SE      LL      UL
A1  -0.408   0.343  -1.249   0.451
A2  -0.500   0.348  -1.357   0.378
A3  -0.905   0.384  -1.873   0.083

Analyses of a Comparison

This section produces analyses involving comparisons of two levels of a factor.

Confidence Intervals

This code estimates the confidence interval of the difference.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> estimateDifference()
$B1

Confidence Interval for the Mean Difference

               Est      SE      df      LL      UL
Comparison   3.000   0.830  15.308   1.234   4.766


$B2

Confidence Interval for the Mean Difference

               Est      SE      df      LL      UL
Comparison   0.000   1.000  17.308  -2.107   2.107

This code obtains and plots the confidence intervals for the mean difference in the identified comparison.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> plotDifference()

Of course, you can change the confidence level from the default 95% if desired.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> estimateDifference(conf.level = .99)
$B1

Confidence Interval for the Mean Difference

               Est      SE      df      LL      UL
Comparison   3.000   0.830  15.308   0.561   5.439


$B2

Confidence Interval for the Mean Difference

               Est      SE      df      LL      UL
Comparison   0.000   1.000  17.308  -2.892   2.892

Once again, the confidence levels can be changed away from the default and a comparison line to represent a population (or test) value and a region of practical equivalence can be added to the graph.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> plotDifference(conf.level = .99, line = 0, rope = c(-2, 2))

If you wish, you can get the confidence intervals for the means and the mean difference in one command.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> estimateComparison()
$B1

Confidence Intervals for the Mean Comparison

               Est      SE      df      LL      UL
A1           8.000   0.447   9.000   6.988   9.012
A2          11.000   0.699   9.000   9.418  12.582
Comparison   3.000   0.830  15.308   1.234   4.766


$B2

Confidence Intervals for the Mean Comparison

               Est      SE      df      LL      UL
A1           8.000   0.775   9.000   6.248   9.752
A2           8.000   0.632   9.000   6.569   9.431
Comparison   0.000   1.000  17.308  -2.107   2.107

This code produces a difference plot using the confidence intervals for the means and the mean difference.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> plotComparison()

Of course, you can change the confidence level from the default 95% if desired.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> estimateComparison(conf.level = .99)
$B1

Confidence Intervals for the Mean Comparison

               Est      SE      df      LL      UL
A1           8.000   0.447   9.000   6.547   9.453
A2          11.000   0.699   9.000   8.728  13.272
Comparison   3.000   0.830  15.308   0.561   5.439


$B2

Confidence Intervals for the Mean Comparison

               Est      SE      df      LL      UL
A1           8.000   0.775   9.000   5.483  10.517
A2           8.000   0.632   9.000   5.945  10.055
Comparison   0.000   1.000  17.308  -2.892   2.892

Once again, the confidence levels can be changed away from the default and a region of practical equivalence can be added to the graph.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> plotComparison(conf.level = .99, rope = c(-2, 2))

Significance Tests

This code produces NHST for the identified comparison (using a default test value of zero).

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> testDifference()
$B1

Hypothesis Test for the Mean Difference

              Diff      SE      df       t       p
Comparison   3.000   0.830  15.308   3.614   0.002


$B2

Hypothesis Test for the Mean Difference

              Diff      SE      df       t       p
Comparison   0.000   1.000  17.308   0.000   1.000

If the default value of zero is not plausible, it too can be changed.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> testDifference(mu = -2)
$B1

Hypothesis Test for the Mean Difference

              Diff      SE      df       t       p
Comparison   5.000   0.830  15.308   6.024   0.000


$B2

Hypothesis Test for the Mean Difference

              Diff      SE      df       t       p
Comparison   2.000   1.000  17.308   2.000   0.061

Standardized Effect Sizes

This code calculates a standardized mean difference for the comparison and its confidence interval.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> standardizeDifference()
$B1

Confidence Interval for the Standardized Mean Difference

                 d      SE      LL      UL
Comparison   1.616   0.555   0.529   2.703


$B2

Confidence Interval for the Standardized Mean Difference

                 d      SE      LL      UL
Comparison   0.000   0.471  -0.924   0.924

The width of the confidence interval for the effect size can be altered if desired.

(Outcome ~ FactorA) |> focus(A1, A2) |> describeMoments(by = FactorB) |> standardizeDifference(conf.level = .99)
$B1

Confidence Interval for the Standardized Mean Difference

                 d      SE      LL      UL
Comparison   1.616   0.555   0.188   3.045


$B2

Confidence Interval for the Standardized Mean Difference

                 d      SE      LL      UL
Comparison   0.000   0.471  -1.214   1.214

Analyses of a Contrast

This section produces analyses involving multiple levels of a factor.

Confidence Intervals

This code produces a confidence interval for a specified contrast.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> estimateContrast(contrast = c(-1, .5, .5))
$B1

Confidence Interval for the Mean Contrast

             Est      SE      df      LL      UL
Contrast   3.500   0.687  25.917   2.087   4.913


$B2

Confidence Interval for the Mean Contrast

             Est      SE      df      LL      UL
Contrast  -0.500   0.907  15.806  -2.424   1.424

This code obtains and plots the confidence intervals for the groups and the mean difference in the identified contrast.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> plotContrast(contrast = c(-1, .5, .5))

As in all other cases, the default value of the confidence interval can be changed.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> estimateContrast(contrast = c(-1, .5, .5), conf.level = .99)
$B1

Confidence Interval for the Mean Contrast

             Est      SE      df      LL      UL
Contrast   3.500   0.687  25.917   1.590   5.410


$B2

Confidence Interval for the Mean Contrast

             Est      SE      df      LL      UL
Contrast  -0.500   0.907  15.806  -3.153   2.153

The width of the confidence interval for the contrast can be altered and a comparison line to represent a population (or test) value and a region of practical equivalence can be added to the graph.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> plotContrast(contrast = c(-1, .5, .5), conf.level = .99, line = 0, rope = c(-2, 2))

If you wish, you can get the confidence intervals for the mean subsets and the mean contrast in one command.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> estimateSubsets(contrast = c(-1, .5, .5))
$B1

Confidence Intervals for the Mean Subsets

                 Est      SE      df      LL      UL
Neg Weighted   8.000   0.447   9.000   6.988   9.012
Pos Weighted  11.500   0.522  17.815  10.403  12.597
Contrast       3.500   0.687  25.917   2.087   4.913


$B2

Confidence Intervals for the Mean Subsets

                 Est      SE      df      LL      UL
Neg Weighted   8.000   0.775   9.000   6.248   9.752
Pos Weighted   7.500   0.471  17.822   6.509   8.491
Contrast      -0.500   0.907  15.806  -2.424   1.424

This code produces a difference plot using the confidence intervals for the mean subsets and the mean contrast.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> plotSubsets(contrast = c(-1, .5, .5))

Of course, you can change the confidence level from the default 95% if desired.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> estimateSubsets(contrast = c(-1, .5, .5), conf.level = .99)
$B1

Confidence Intervals for the Mean Subsets

                 Est      SE      df      LL      UL
Neg Weighted   8.000   0.447   9.000   6.547   9.453
Pos Weighted  11.500   0.522  17.815   9.996  13.004
Contrast       3.500   0.687  25.917   1.590   5.410


$B2

Confidence Intervals for the Mean Subsets

                 Est      SE      df      LL      UL
Neg Weighted   8.000   0.775   9.000   5.483  10.517
Pos Weighted   7.500   0.471  17.822   6.142   8.858
Contrast      -0.500   0.907  15.806  -3.153   2.153

Once again, the confidence levels can be changed away from the default and a region of practical equivalence can be added to the graph.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> plotSubsets(contrast = c(-1, .5, .5), labels = c("Level1", "Others"), conf.level = .99, rope = c(-2, 2))

Significance Tests

This code produces a NHST for the identified contrast. It tests the contrast against a value of zero by default.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> testContrast(contrast = c(-1, .5, .5))
$B1

Hypothesis Test for the Mean Contrast

             Est      SE      df       t       p
Contrast   3.500   0.687  25.917   5.093   0.000


$B2

Hypothesis Test for the Mean Contrast

             Est      SE      df       t       p
Contrast  -0.500   0.907  15.806  -0.551   0.589

If desired, the contrast can be tested against other values.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> testContrast(contrast = c(-1, .5, .5), mu = 4)
$B1

Hypothesis Test for the Mean Contrast

             Est      SE      df       t       p
Contrast  -0.500   0.687  25.917  -0.728   0.473


$B2

Hypothesis Test for the Mean Contrast

             Est      SE      df       t       p
Contrast  -4.500   0.907  15.806  -4.963   0.000

Standardized Effect Sizes

This code calculates a standardized contrast and its confidence interval.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> standardizeContrast(contrast = c(-1, .5, .5))
$B1

Confidence Interval for the Standardized Mean Contrast

             Est      SE      LL      UL
Contrast   1.689   0.428   0.850   2.527


$B2

Confidence Interval for the Standardized Mean Contrast

             Est      SE      LL      UL
Contrast  -0.224   0.430  -1.068   0.619

The width of the confidence interval for the effect size can be altered if desired.

(Outcome ~ FactorA) |> describeMoments(by = FactorB) |> standardizeContrast(contrast = c(-1, .5, .5), conf.level = .99)
$B1

Confidence Interval for the Standardized Mean Contrast

             Est      SE      LL      UL
Contrast   1.689   0.428   0.586   2.791


$B2

Confidence Interval for the Standardized Mean Contrast

             Est      SE      LL      UL
Contrast  -0.224   0.430  -1.332   0.884

Analyses of 2x2 Interaction Comparisons

This section produces analyses involving an interaction among multiple factors.

Confidence Intervals

Identify a 2 x 2 interaction of interest (in this case, two levels of FactorA and the existing two levels of FactorB). Estimate and plot the interaction contrast (which includes the comparisons within each simple effect).

(Outcome ~ FactorA) |> focus(A1, A2) |> estimateInteraction(by = FactorB)

Confidence Intervals for the Mean Interaction

                        Est      SE      df      LL      UL
Simple Effect at B1   3.000   0.830  15.308   1.234   4.766
Simple Effect at B2   0.000   1.000  17.308  -2.107   2.107
Interaction          -3.000   1.300  32.129  -5.647  -0.353
(Outcome ~ FactorA) |> focus(A1, A2) |> plotInteraction(by = FactorB)

Significance Tests

Test the interaction contrast (which includes the comparisons within each simple effect) for statistical significance.

(Outcome ~ FactorA) |> focus(A1, A2) |> testInteraction(by = FactorB)

Hypothesis Tests for the Mean Interaction

                 Est      SE       t      df       p
Effect at B1   3.000   0.830   3.614  15.308   0.002
Effect at B2   0.000   1.000   0.000  17.308   1.000
Interaction   -3.000   1.300  -2.308  32.129   0.028