An anthology of
experimental designs

Emi Tanaka

Department of Econometrics and Business Statistics

Take-away message 1

  • edibble is an experimental R-package I have been developing to plan, design, and simulate experiments

edibble stands for experimental design table (or tibble)

  • aims to provide the experimental design in table format
  • github.com/emitanaka/edibble
  • commonly misspelled as edible (perhaps because of autocorrect)

Take-away message 2

  • deggust is a work-in-progress R-package I am developing to easily visualise edibble experimental designs
deggust alludes to the design of experiments as ggplot object
  • aims to provide a ggplot object
  • github.com/emitanaka/deggust
  • deggust, a play on the verb “degust”, is to savour the edibble designs to fully appreciate it

Take-away message 3

  • The edibble system and its (future) extensions signficiantly reframe the way that the experimental designs are constructed from standard, often recipe-driven, approach
  • I refer this programming language agnostic system to as
    “The Grammar of Experimental Designs”

A work-in-progress book “The Grammar of Experimental Designs” can be found at

https://emitanaka.org/edibble-book

Experimental design
short stories

Giesbrecht, Francis G. and Gumpertz, Marcia L. (2004) Planning, Construction, and Statistical Analysis of Comparative Experiments.

Bailey, Rosemary (2008) Design of Comparative Experiments.

Lawson, John (2015) Design and Analysis of Experiments with R.

STORY 1 🌱 The folklore of fervent field experiments

STORY 1 🌱 The folklore of fervent field experiments

Consider a field experiment in agriculture where plot are laid in out as 6 rows by 8 columns (each also called strip). There are 4 modes of seedbed preparation and 3 crop varieties that are of the interest to the researcher. The mode of seedbed preparation is assigned randomly to the whole columns, and the crop variety is assigned randomly to the whole row. This experimental design is called a strip-plot design.

library(edibble)
df1 <- design("Strip-plot experiment") %>%
  set_units(row = 6,
            col = 8,
            plot = crossed_by(row, col)) %>%
  set_trts(seedbed = 4,
           variety = 3) %>% 
  allot_trts(seedbed ~ col,
             variety ~ row) %>%
  assign_trts("random", seed = 20220324) %>%
  serve_table()

df1
# Strip-plot experiment 
# An edibble: 48 x 5
         row       col       plot  seedbed  variety
   <unit(6)> <unit(8)> <unit(48)> <trt(4)> <trt(3)>
 1      row1      col1     plot1  seedbed4 variety3
 2      row2      col1     plot2  seedbed4 variety2
 3      row3      col1     plot3  seedbed4 variety2
 4      row4      col1     plot4  seedbed4 variety1
 5      row5      col1     plot5  seedbed4 variety3
 6      row6      col1     plot6  seedbed4 variety1
 7      row1      col2     plot7  seedbed2 variety3
 8      row2      col2     plot8  seedbed2 variety2
 9      row3      col2     plot9  seedbed2 variety2
10      row4      col2     plot10 seedbed2 variety1
# … with 38 more rows

STORY 1 🌱 The folklore of fervent field experiments

library(edibble)
df1 <- design("Strip-plot experiment") %>%
  set_units(row = 6,
            col = 8,
            plot = crossed_by(row, col)) %>%
  set_trts(seedbed = 4,
           variety = 3) %>% 
  allot_trts(seedbed ~ col,
             variety ~ row) %>%
  assign_trts("random", seed = 20220324) %>%
  serve_table()

df1
# Strip-plot experiment 
# An edibble: 48 x 5
         row       col       plot  seedbed  variety
   <unit(6)> <unit(8)> <unit(48)> <trt(4)> <trt(3)>
 1      row1      col1     plot1  seedbed4 variety3
 2      row2      col1     plot2  seedbed4 variety2
 3      row3      col1     plot3  seedbed4 variety2
 4      row4      col1     plot4  seedbed4 variety1
 5      row5      col1     plot5  seedbed4 variety3
 6      row6      col1     plot6  seedbed4 variety1
 7      row1      col2     plot7  seedbed2 variety3
 8      row2      col2     plot8  seedbed2 variety2
 9      row3      col2     plot9  seedbed2 variety2
10      row4      col2     plot10 seedbed2 variety1
# … with 38 more rows
library(deggust)
autoplot(df1) +
  theme(text = element_text(size = 30),
        axis.text.x = element_text(size = 24),
        axis.text.y = element_text(size = 24),
        plot.margin = margin(20, 20, 20, 20))

STORY 1 🌱 The folklore of fervent field experiments

Consider a field experiment in agriculture where plot are laid in out as 6 rows by 8 columns (each also called strip). There are 4 modes of seedbed preparation and 3 crop varieties that are of the interest to the researcher. The mode of seedbed preparation is assigned randomly to the whole columns, and the crop variety is assigned randomly to the whole row. If fertility gradients are suspected then the strips can be grouped into sets (i.e. introduce blocking factors in one or more directions). This variation is called a strip-block design.

library(edibble)
df2 <- design("Strip-block experiment") %>%
  set_units(block = 2,
            row = 6,
            col = nested_in(block, 4),
            plot = crossed_by(row, col)) %>%
  set_trts(seedbed = 4,
           variety = 3) %>% 
  allot_trts(seedbed ~ col,
             variety ~ row) %>%
  assign_trts("random", seed = 20220324) %>%
  serve_table()

df2
# Strip-block experiment 
# An edibble: 48 x 6
       block       row       col       plot  seedbed  variety
   <unit(2)> <unit(6)> <unit(8)> <unit(48)> <trt(4)> <trt(3)>
 1    block1      row1      col1     plot1  seedbed4 variety2
 2    block1      row2      col1     plot2  seedbed4 variety3
 3    block1      row3      col1     plot3  seedbed4 variety2
 4    block1      row4      col1     plot4  seedbed4 variety1
 5    block1      row5      col1     plot5  seedbed4 variety1
 6    block1      row6      col1     plot6  seedbed4 variety3
 7    block1      row1      col2     plot7  seedbed3 variety2
 8    block1      row2      col2     plot8  seedbed3 variety3
 9    block1      row3      col2     plot9  seedbed3 variety2
10    block1      row4      col2     plot10 seedbed3 variety1
# … with 38 more rows

STORY 1 🌱 The folklore of fervent field experiments

library(edibble)
df2 <- design("Strip-block experiment") %>%
  set_units(block = 2,
            row = 6,
            col = nested_in(block, 4),
            plot = crossed_by(row, col)) %>%
  set_trts(seedbed = 4,
           variety = 3) %>% 
  allot_trts(seedbed ~ col,
             variety ~ row) %>%
  assign_trts("random", seed = 20220324) %>%
  serve_table()

df2
# Strip-block experiment 
# An edibble: 48 x 6
       block       row       col       plot  seedbed  variety
   <unit(2)> <unit(6)> <unit(8)> <unit(48)> <trt(4)> <trt(3)>
 1    block1      row1      col1     plot1  seedbed4 variety2
 2    block1      row2      col1     plot2  seedbed4 variety3
 3    block1      row3      col1     plot3  seedbed4 variety2
 4    block1      row4      col1     plot4  seedbed4 variety1
 5    block1      row5      col1     plot5  seedbed4 variety1
 6    block1      row6      col1     plot6  seedbed4 variety3
 7    block1      row1      col2     plot7  seedbed3 variety2
 8    block1      row2      col2     plot8  seedbed3 variety3
 9    block1      row3      col2     plot9  seedbed3 variety2
10    block1      row4      col2     plot10 seedbed3 variety1
# … with 38 more rows
library(deggust)
autoplot(df2) +
  theme(text = element_text(size = 30),
        axis.text.x = element_text(size = 24),
        axis.text.y = element_text(size = 24),
        plot.margin = margin(20, 20, 20, 20))

STORY 1 🌱 The folklore of fervent field experiments

library(edibble)
df3 <- design("Strip-block experiment 2") %>%
  set_units(blockr = 2,
            blockc = 2,
            row = nested_in(blockr, 3),
            col = nested_in(blockc, 4),
            plot = crossed_by(row, col)) %>%
  set_trts(seedbed = 4,
           variety = 3) %>% 
  allot_trts(seedbed ~ col,
             variety ~ row) %>%
  assign_trts("random", seed = 20220324) %>%
  serve_table()

df3
# Strip-block experiment 2 
# An edibble: 48 x 7
      blockr    blockc       row       col       plot  seedbed
   <unit(2)> <unit(2)> <unit(6)> <unit(8)> <unit(48)> <trt(4)>
 1   blockr1   blockc1      row1      col1     plot1  seedbed4
 2   blockr1   blockc1      row2      col1     plot2  seedbed4
 3   blockr1   blockc1      row3      col1     plot3  seedbed4
 4   blockr2   blockc1      row4      col1     plot4  seedbed4
 5   blockr2   blockc1      row5      col1     plot5  seedbed4
 6   blockr2   blockc1      row6      col1     plot6  seedbed4
 7   blockr1   blockc1      row1      col2     plot7  seedbed3
 8   blockr1   blockc1      row2      col2     plot8  seedbed3
 9   blockr1   blockc1      row3      col2     plot9  seedbed3
10   blockr2   blockc1      row4      col2     plot10 seedbed3
# … with 38 more rows, and 1 more variable: variety <trt(3)>
library(deggust)
ggplot(df3, aes(col, row)) + 
  geom_tile(aes(fill = blockr)) +
  scale_fill_manual(values = c("grey10", "grey80")) +
  ggnewscale::new_scale_fill() + 
  geom_unit_node(aes(fill = seedbed)) +
  scale_fill_viridis_d() +
  ggnewscale::new_scale_fill() + 
  geom_unit_node(aes(fill = variety), 
                 shape = "square", 
                 width = 0.5, height = 0.5) +
  theme(text = element_text(size = 30),
        axis.text.x = element_text(size = 24),
        axis.text.y = element_text(size = 24),
        plot.margin = margin(20, 20, 20, 20))

STORY 2 👕 The horrifying tale of dirty laundry

STORY 2 👕 The horrifying tale of dirty laundry

A manufacturer of household appliances wants to find the best combination of wash temperature and drying temperature to produce unwrinkled cotton sheets at the end of the laundry session. He wants to compare four different wash temperatures and three different drying temperatures. He uses eight similar washing machines and six similar dryers. First, 48 cotton sheets are randomly allocated to the washing machines, six per machine. The wash temperatures are randomly allocated to the washing machines so that two machines are run at each temperature. After the wash, the six sheets in each machine are randomly allocated to the dryers, one per dryer. Then the drying temperatures are randomly allocated to the dryers so that two machines are run at each temperature. After the drying, all 48 sheets are scored by experts for how wrinkled they are.

library(edibble)
dat1 <- design("Laundry", seed = 1) %>%
  set_trts(wash_temp = 4,
           dry_temp = 3) %>% 
  set_units(washer = 8,
            dryer = 6,
            sheet = 48) %>% 
  allot_units(washer ~ sheet,
              dryer ~ washer/sheet) %>% 
  assign_units("random") %>% 
  allot_trts(wash_temp ~ washer,
              dry_temp ~ dryer) %>% 
  assign_trts("random") %>% 
  set_rcrds(score = sheet) %>% 
  serve_table()

dat1
# Laundry 
# An edibble: 48 x 6
    wash_temp  dry_temp    washer     dryer      sheet  score
     <trt(4)>  <trt(3)> <unit(8)> <unit(6)> <unit(48)> <rcrd>
 1 wash_temp3 dry_temp2   washer1    dryer5    sheet1       ■
 2 wash_temp2 dry_temp2   washer4    dryer1    sheet2       ■
 3 wash_temp3 dry_temp2   washer8    dryer1    sheet3       ■
 4 wash_temp4 dry_temp1   washer2    dryer4    sheet4       ■
 5 wash_temp2 dry_temp3   washer6    dryer2    sheet5       ■
 6 wash_temp1 dry_temp2   washer3    dryer1    sheet6       ■
 7 wash_temp1 dry_temp2   washer7    dryer1    sheet7       ■
 8 wash_temp4 dry_temp1   washer5    dryer6    sheet8       ■
 9 wash_temp4 dry_temp2   washer2    dryer1    sheet9       ■
10 wash_temp1 dry_temp2   washer3    dryer5    sheet10      ■
# … with 38 more rows

STORY 2 👕 The horrifying tale of dirty laundry

library(edibble)
dat1 <- design("Laundry", seed = 1) %>%
  set_trts(wash_temp = 4,
           dry_temp = 3) %>% 
  set_units(washer = 8,
            dryer = 6,
            sheet = 48) %>% 
  allot_units(washer ~ sheet,
              dryer ~ washer/sheet) %>% 
  assign_units("random") %>% 
  allot_trts(wash_temp ~ washer,
              dry_temp ~ dryer) %>% 
  assign_trts("random") %>% 
  set_rcrds(score = sheet) %>% 
  serve_table()

dat1
# Laundry 
# An edibble: 48 x 6
    wash_temp  dry_temp    washer     dryer      sheet  score
     <trt(4)>  <trt(3)> <unit(8)> <unit(6)> <unit(48)> <rcrd>
 1 wash_temp3 dry_temp2   washer1    dryer5    sheet1       ■
 2 wash_temp2 dry_temp2   washer4    dryer1    sheet2       ■
 3 wash_temp3 dry_temp2   washer8    dryer1    sheet3       ■
 4 wash_temp4 dry_temp1   washer2    dryer4    sheet4       ■
 5 wash_temp2 dry_temp3   washer6    dryer2    sheet5       ■
 6 wash_temp1 dry_temp2   washer3    dryer1    sheet6       ■
 7 wash_temp1 dry_temp2   washer7    dryer1    sheet7       ■
 8 wash_temp4 dry_temp1   washer5    dryer6    sheet8       ■
 9 wash_temp4 dry_temp2   washer2    dryer1    sheet9       ■
10 wash_temp1 dry_temp2   washer3    dryer5    sheet10      ■
# … with 38 more rows
library(deggust)
autoplot(dat1) +
  theme(text = element_text(size = 30),
        axis.text.x = element_text(size = 24),
        axis.text.y = element_text(size = 24),
        plot.margin = margin(20, 20, 20, 20))

This is in fact a strip-plot design!

STORY 2 👕 The horrifying tale of dirty laundry

library(edibble)
dat2 <- design("Laundry", seed = 1) %>%
  set_trts(wash_temp = 4,
           dry_temp = 3) %>% 
  set_units(washer = 8,
            dryer = 6,
            sheet = crossed_by(washer, dryer)) %>% 
  allot_trts(wash_temp ~ washer,
              dry_temp ~ dryer) %>% 
  assign_trts("random") %>% 
  set_rcrds(score = sheet) %>% 
  serve_table()

dat2
# Laundry 
# An edibble: 48 x 6
    wash_temp  dry_temp    washer     dryer      sheet  score
     <trt(4)>  <trt(3)> <unit(8)> <unit(6)> <unit(48)> <rcrd>
 1 wash_temp3 dry_temp3   washer1    dryer1    sheet1       ■
 2 wash_temp4 dry_temp3   washer2    dryer1    sheet2       ■
 3 wash_temp4 dry_temp3   washer3    dryer1    sheet3       ■
 4 wash_temp1 dry_temp3   washer4    dryer1    sheet4       ■
 5 wash_temp1 dry_temp3   washer5    dryer1    sheet5       ■
 6 wash_temp2 dry_temp3   washer6    dryer1    sheet6       ■
 7 wash_temp2 dry_temp3   washer7    dryer1    sheet7       ■
 8 wash_temp3 dry_temp3   washer8    dryer1    sheet8       ■
 9 wash_temp3 dry_temp2   washer1    dryer2    sheet9       ■
10 wash_temp4 dry_temp2   washer2    dryer2    sheet10      ■
# … with 38 more rows
library(deggust)
autoplot(dat2) +
  theme(text = element_text(size = 30),
        axis.text.x = element_text(size = 24),
        axis.text.y = element_text(size = 24),
        plot.margin = margin(20, 20, 20, 20))

Setting and simulating records with expectations

edibble records

  • Records are intended variables, e.g. responses, that will be measured or observed
  • You can set expectations of the record (plausible values) and simulate records, censoring values (default as missing) outside of expectations, or export data with data validation
dat2 %>% 
  expect_rcrds(score >= 1, score <= 10) %>% 
  simulate_rcrds(score = sim_normal(mean = 3, sd = 4))
# Laundry 
# An edibble: 48 x 6
    wash_temp  dry_temp    washer     dryer      sheet score
     <trt(4)>  <trt(3)> <unit(8)> <unit(6)> <unit(48)> <dbl>
 1 wash_temp3 dry_temp3   washer1    dryer1    sheet1   1.35
 2 wash_temp4 dry_temp3   washer2    dryer1    sheet2   4.01
 3 wash_temp4 dry_temp3   washer3    dryer1    sheet3  NA   
 4 wash_temp1 dry_temp3   washer4    dryer1    sheet4   4.74
 5 wash_temp1 dry_temp3   washer5    dryer1    sheet5  NA   
 6 wash_temp2 dry_temp3   washer6    dryer1    sheet6   2.10
 7 wash_temp2 dry_temp3   washer7    dryer1    sheet7   4.51
 8 wash_temp3 dry_temp3   washer8    dryer1    sheet8   3.53
 9 wash_temp3 dry_temp2   washer1    dryer2    sheet9   6.22
10 wash_temp4 dry_temp2   washer2    dryer2    sheet10  2.77
# … with 38 more rows

STORY 3 🍪 The legend of the scrumptious cookie

STORY 3 🍪 The legend of the scrumptious cookie

Recipes for chocolate and orange cookies include exactly the same ingredients up to the point where the syrup was added to the batch. However, after the cookies were baked, the chocolate cookies had an appealing round and plump appearance, while the orange cookies spread during the baking process and became thin, flat, and unappealing. A factorial experiment was devised to determine if there was a way to change the process of making the orange cookies that would reduce the spreading during baking. The factors that were chosen to be varied were A: the amount of shortening in the dough batch (80% of what the recipe called for or 100%), B: the baking temperature (below, at, or above the temperature called for by the recipe), and C: the temperature of the cookie sheet upon which the cookies were placed to be baked (hot out of the oven, or cooled to room temperature). The cookie-making process consists of the two steps: 1. mix cookie dough batch, and 2. bake cookies. Factor A is related to step 1, factors B and C to step 2. The amount of shortening was a hard-to-vary factor because each time it was changed it required making a new batch of cookie dough, while the baking temperature and tray temperature were easy to vary. Once a batch of dough was made, there was enough to make six trays of cookies, and all six combinations of baking temperature and tray temperature could be tested within each dough batch. First, plan to make four batches of cookie dough and randomly assign two batches to use 80% of the recipe recommended amount of shortening and two batches to receive the full amount of shortening recommended by the recipe. Next, bake six trays of cookies from each batch of dough and completely randomize the six combinations of bake temperature and tray temperature to the six trays of cookies within each batch.

STORY 3 🍪 The legend of the scrumptious cookie

library(edibble)
des <- design("Cookie recipe", seed = 2022) %>%
  set_trts(shortening = c("80%", "100%"),
           baking_temp = c("below", "at", "above"),
           tray_temp = c("hot", "room temp")) %>% 
  set_units(batch = 4,
            tray = nested_in(batch, 6)) %>% 
  allot_trts(shortening ~ batch,
             baking_temp:tray_temp ~ tray) %>% 
  assign_trts("random") %>% 
  serve_table()

des
# Cookie recipe 
# An edibble: 24 x 5
   shortening baking_temp tray_temp     batch       tray
     <trt(2)>    <trt(3)>  <trt(2)> <unit(4)> <unit(24)>
 1       100%       at    room temp    batch1     tray1 
 2       100%       below hot          batch1     tray2 
 3       100%       above hot          batch1     tray3 
 4       100%       at    hot          batch1     tray4 
 5       100%       above room temp    batch1     tray5 
 6       100%       below room temp    batch1     tray6 
 7       80%        above hot          batch2     tray7 
 8       80%        below hot          batch2     tray8 
 9       80%        above room temp    batch2     tray9 
10       80%        below room temp    batch2     tray10
# … with 14 more rows
library(deggust)
autoplot(des) +
  theme(text = element_text(size = 30),
        axis.text.x = element_text(size = 24),
        axis.text.y = element_text(size = 24),
        plot.margin = margin(20, 20, 20, 20))


  • Experiments are human endeavours and the edibble system aims to better capture user’s intention or understanding