class: middle center hide-slide-number monash-bg-gray80 .info-box.w-50.bg-white[ These slides are viewed best by Chrome or Firefox and occasionally need to be refreshed if elements did not load properly. See <a href=lecture-11A.pdf>here for the PDF <i class="fas fa-file-pdf"></i></a>. ] <br> .white[Press the **right arrow** to progress to the next slide!] --- class: title-slide count: false background-image: url("images/bg-01.png") # .monash-blue[ETC5521: Exploratory Data Analysis] <h1 class="monash-blue" style="font-size: 30pt!important;"></h1> <br> <h2 style="font-weight:900!important;">Using computational tools to determine whether what is seen in the data can be assumed to apply more broadly</h2> .bottom_abs.width100[ Lecturer: *Emi Tanaka* <i class="fas fa-envelope"></i> ETC5521.Clayton-x@monash.edu <i class="fas fa-calendar-alt"></i> Week 11 - Session 1 <br> ] <style type="text/css"> table { margin: auto; border-top: 0.16em solid #111111; border-bottom: 0.16em solid #111111; } table thead th { border-bottom: 0.1em solid #111111; text-align: left; } tr, th { padding-left: 15px; padding-right: 15px } </style> --- class: transition middle # Revisiting <br> .yellow[hypothesis testing] --- # Testing coin bias .font_small[Part 1/2] * Suppose I have a coin that I'm going to flip <img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"> -- * If the coin is unbiased, what is the probability it will show heads? -- * Yup, the probability should be 0.5. -- * So how would I test if a coin is biased or unbiased? -- * We'll collect some data. -- * **Experiment 1**: I flipped the coin 10 times and this is the result: <center> <img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"> </center> -- * The result is 7 head and 3 tails. So 70% are heads. -- * Do you believe the coin is biased based on this data? --- # Testing coin bias .font_small[Part 2/2] * **Experiment 2**: Suppose now I flip the coin 100 times and this is the outcome: <img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_Fifty_Cents_Obv.jpg" height = "50px" style="vertical-align:middle;"><img src="images/Australian_50c_Coin.png" height = "50px" style="vertical-align:middle;"> -- * We observe 70 heads and 30 tails. So again 70% are heads. -- * Based on this data, do you think the coin is biased? --- # (Frequentist) hypotheses testing framework * Suppose `\(X\)` is the number of heads out of `\(n\)` independent tosses. * Let `\(p\)` be the probability of getting a head for this coin. | | | |-----|-----| |**Hypotheses** | `\(H_0: p = 0.5\)` vs. `\(H_1: p \neq 0.5\)` | -- |**Assumptions** | Each toss is independent with equal chance of getting a head. | -- |**Test statistic** | `\(X \sim B(n, p)\)`. Recall `\(E(X) = np\)`.<br> The observed test statistic is denoted `\(x\)`. | -- |**P-value** <br>.font_small[(or critical value or confidence interval)] | `\(P(\mid X - np\mid \geq \mid x - np\mid )\)` | -- |**<i class="fas fa-gavel"></i> Conclusion** | Reject null hypothesis when the `\(p\)`-value is less than<br> some significance level `\(\alpha\)`. Usually `\(\alpha = 0.05\)`.| -- * The p-value for experiment 1 is `\(P(|X - 5| \geq 2) \approx 0.34\)`. * The p-value for experiment 2 is `\(P(|X - 50| \geq 20) \approx 0.00008\)`. --- # Judicial system .grid[ .item[ <br> <center> <img src="images/omni/judicial-court.png"> </center> ] .item.bg-gray80.pl3[ <br> {{content}} ] ] -- <center> <img src="images/omni/statistical-court.png"> </center> <ul> <li><i class="fas fa-search"></i> Evidence by test statistic</li> <li><i class="fas fa-gavel"></i> Judgement by p-value, critical value or confidence interval</li> </ul> {{content}} -- <span class="red">Does the test statistic have to be a <i>numerical summary statistics</i>?</span> --- class: transition middle # Visual inference --- # Visual inference .grid[ .item[ * Hypothesis testing in visual inference framework is where: * <i class="fas fa-search"></i> the .monash-blue[_test statistic is a plot_] and * <i class="fas fa-gavel"></i> judgement is by human perceptions. {{content}} ] .item.bg-gray80.pa3[ From Exercise 4 in week 9 tutorial: a residual plot after modelling high-density lipoprotein in human blood. <img src="images/week11A/HDL-resplot-1.png" width="432" style="display: block; margin: auto;" /> ] ] -- * You (and many other people) actually do visual inference many times but generally in an informal fashion. {{content}} -- * Here, we are making an inference on whether the residual plot has any patterns based on a single data plot. --- class: transition middle .font_large[ <i class="fas fa-chart-bar"></i> Data plots tend to be over-interpreted <br> <i class="fas fa-chart-bar"></i> Reading data plots require calibration ] --- # Visual inference more formally .flex[ .w-60[ 1. State your null and alternate hypotheses. {{content}} ]] -- 2. Define a <b>visual test statistic</b>, `\(V(.)\)`, i.e. a function of a sample to a plot. {{content}} -- 3. Define a method to generate <b>null data</b>, `\(\boldsymbol{y}_0\)`. {{content}} -- 4. `\(V(\boldsymbol{y})\)` maps the actual data, `\(\boldsymbol{y}\)`, to the plot. We call this the <b class="monash-blue">data plot</b>. {{content}} -- 5. `\(V(\boldsymbol{y}_0)\)` maps a null data to a plot of the same form. We call this the <b class="monash-blue">null plot</b>. We repeat this `\(m - 1\)` times to generate `\(m-1\)` null plots. {{content}} -- 6. A <b class="monash-blue">lineup</b> displays these `\(m\)` plots in a random order. {{content}} -- 7. Ask `\(n\)` human viewers to select a plot in the lineup that looks different to others without any context given. --- count: false # Visual inference more formally .flex[ .w-60[ 1. State your null and alternate hypotheses. 2. Define a **visual test statistic**, `\(V(.)\)`, i.e. a function of a sample to a plot. 3. Define a method to generate **null data**, `\(\boldsymbol{y}_0\)`. 4. `\(V(\boldsymbol{y})\)` maps the actual data, `\(\boldsymbol{y}\)`, to the plot. We call this the .monash-blue[**data plot**]. 5. `\(V(\boldsymbol{y}_0)\)` maps a null data to a plot of the same form. We call this the .monash-blue[**null plot**]. We repeat this `\(m - 1\)` times to generate `\(m-1\)` null plots. 6. A .monash-blue[**lineup**] displays these `\(m\)` plots in a random order. 7. Ask `\(n\)` human viewers to select a plot in the lineup that looks different to others without any context given. ] .w-40.pl3[ .info-box[ Suppose `\(x\)` out of `\(n\)` people detected the data plot from a lineup, then * the .monash-blue[**visual inference p-value**] is given as `$$P(X \geq x)$$` where `\(X \sim B(n, 1/m)\)`, and * the .monash-blue[**power of a lineup**] is estimated as `\(x/n\)`. ] ]] --- # .monash-blue[Lineup] .circle.monash-bg-blue.white[1] In which plot is the pink group higher than the blue group? * Note: there is no correct answer here. <img src="images/week11A/weight-lineup-1.png" width="1008" style="display: block; margin: auto;" /> --- # Visual inference p-value (or "see"-value) * So <span contenteditable="true">&nbsp;&nbsp; `\(x\)` &nbsp;&nbsp;</span> out of <span contenteditable="true">&nbsp;&nbsp; `\(n\)` &nbsp;&nbsp;</span> of you chose the data plot. * So the visual inference p-value is `\(P(X \geq x)\)` where `\(X \sim B(n, 1/10)\)`. * In R, this is ```r 1 - pbinom(x - 1, n, 1/10) # OR nullabor::pvisual(x, n, 10) ``` --- # .orange[Case study] .circle.white.bg-orange[1] Weight loss by diet .flex[ .w-40[ .panelset[ .panel[.panel-name[📊] .f5[ <img src="images/week11A/weight-plot-1.png" width="360" style="display: block; margin: auto;" /> .gray[This is actually Plot 4 in the previous lineup.] ] ] .panel[.panel-name[data] .h200.scroll-sign.f5[ ```r data("WeightLoss", package = "carData") # purposefully make it 2 groups df <- filter(WeightLoss, group!="DietEx") skimr::skim(df) ``` ``` ## ── Data Summary ──────────────────────── ## Values ## Name df ## Number of rows 24 ## Number of columns 7 ## _______________________ ## Column type frequency: ## factor 1 ## numeric 6 ## ________________________ ## Group variables None ## ## ── Variable type: factor ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ## skim_variable n_missing complete_rate ordered n_unique top_counts ## 1 group 0 1 FALSE 2 Con: 12, Die: 12, Die: 0 ## ## ── Variable type: numeric ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ## skim_variable n_missing complete_rate mean sd p0 p25 p50 p75 p100 hist ## 1 wl1 0 1 4.92 1.25 3 4 5 6 7 ▃▇▇▆▃ ## 2 wl2 0 1 3.62 1.24 2 3 3.5 4.25 6 ▆▇▇▅▂ ## 3 wl3 0 1 2.17 1.13 1 1 2 3 4 ▇▅▁▅▃ ## 4 se1 0 1 14.8 2.11 11 13 15 16.2 19 ▃▆▂▇▁ ## 5 se2 0 1 14.0 2.35 11 11.8 14 15.2 19 ▇▇▆▅▂ ## 6 se3 0 1 15.6 2.45 11 14 15 18 19 ▃▃▆▃▇ ``` ]] .panel[.panel-name[R] .f5[ ```r gweight <- ggplot(df, aes(group, wl1, color = group)) + ggbeeswarm::geom_quasirandom() + labs(x = "", y = "Weight loss at 1 month") + theme(text = element_text(size = 22)) + guides(color = "none") + scale_color_manual(values = c("#006DAE", "#ee64a4")) gweight ``` ] ] ] ] .w-60[ * Is weight loss greater with diet after 1 month? .f5[ {{content}} ] ] ] -- <table class=" lightable-classic" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; width: auto !important; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="text-align:left;"> Group </th> <th style="text-align:right;"> N </th> <th style="text-align:right;"> Mean </th> <th style="text-align:right;"> Std. Dev </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Control </td> <td style="text-align:right;"> 12 </td> <td style="text-align:right;"> 4.50 </td> <td style="text-align:right;"> 1.00 </td> </tr> <tr> <td style="text-align:left;"> Diet </td> <td style="text-align:right;"> 12 </td> <td style="text-align:right;"> 5.33 </td> <td style="text-align:right;"> 1.37 </td> </tr> </tbody> </table> {{content}} -- ```r with(df, t.test(wl1[group=="Diet"], wl1[group=="Control"], alternative = "greater")) ``` ``` ## ## Welch Two Sample t-test ## ## data: wl1[group == "Diet"] and wl1[group == "Control"] ## t = 1.7014, df = 20.125, p-value = 0.05213 ## alternative hypothesis: true difference in means is greater than 0 ## 95 percent confidence interval: ## -0.01117097 Inf ## sample estimates: ## mean of x mean of y ## 5.333333 4.500000 ``` --- # Null data generation method .w-70[ * We are testing `\(H_0: \mu_{diet} = \mu_{control}\)` vs. `\(H_1: \mu_{diet} > \mu_{control}\)` where `\(\mu_{diet}\)` and `\(\mu_{control}\)` are the average weight loss for population on diet and no diet, respectively. {{content}} ] -- * There are a number of ways to generate null data under `\(H_0\)`, e.g. {{content}} -- * we could assume a parametric distribution of the data and estimate the parameters from the data, or {{content}} -- * we could permute the labels for the diet and control group. -- .flex[ .w-33[ <img src="images/week11A/weight-null-demo1-1.png" width="288" style="display: block; margin: auto;" /> ] {{content}} ] -- <div class="w-33"> <img src="images/week11A/weight-null-demo2-1.png" width="288" style="display: block; margin: auto;" /> </div> {{content}} -- <div class="w-33"> <img src="images/week11A/weight-null-demo3-1.png" width="288" style="display: block; margin: auto;" /> </div> --- # .monash-blue[Lineup] .circle.monash-bg-blue.white[2] In which plot is the pink group higher than the blue group? <img src="images/week11A/weight-ex-lineup-1.png" width="1008" style="display: block; margin: auto;" /> --- # .orange[Case study] .circle.white.bg-orange[1] Weight loss by diet and exercise .flex[ .w-40[ .panelset[ .panel[.panel-name[📊] .f5[ <img src="images/week11A/weight-ex-plot-1.png" width="360" style="display: block; margin: auto;" /> .white[This is actually Plot 10 in the previous lineup.] ] ] .panel[.panel-name[data] .h200.scroll-sign.f5[ ```r df2 <- filter(WeightLoss, group!="Diet") skimr::skim(df2) ``` ``` ## ── Data Summary ──────────────────────── ## Values ## Name df2 ## Number of rows 22 ## Number of columns 7 ## _______________________ ## Column type frequency: ## factor 1 ## numeric 6 ## ________________________ ## Group variables None ## ## ── Variable type: factor ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ## skim_variable n_missing complete_rate ordered n_unique top_counts ## 1 group 0 1 FALSE 2 Con: 12, Die: 10, Die: 0 ## ## ── Variable type: numeric ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ## skim_variable n_missing complete_rate mean sd p0 p25 p50 p75 p100 hist ## 1 wl1 0 1 5.27 1.86 3 4 5 6 9 ▇▃▃▂▂ ## 2 wl2 0 1 4.59 1.87 2 3 4.5 5.75 9 ▇▅▇▃▁ ## 3 wl3 0 1 2.14 1.17 1 1 2 3 4 ▇▅▁▃▃ ## 4 se1 0 1 15.0 1.65 11 14 15 16 17 ▁▃▂▃▇ ## 5 se2 0 1 13.9 1.93 11 12.2 13.5 15 18 ▇▇▇▃▂ ## 6 se3 0 1 16.2 2.22 11 15 17 18 19 ▂▂▃▇▇ ``` ]] .panel[.panel-name[R] .f5[ ```r gweight %+% df2 + aes(y = wl2) + labs(y = "Weight loss at 2 months") ``` ] ] ] ] .w-60[ * Is weight loss greater with diet *and exercise after 2 months*? .f5[ {{content}} ] ] ] -- <table class=" lightable-classic" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; width: auto !important; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="text-align:left;"> Group </th> <th style="text-align:right;"> N </th> <th style="text-align:right;"> Mean </th> <th style="text-align:right;"> Std. Dev </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Control </td> <td style="text-align:right;"> 12 </td> <td style="text-align:right;"> 3.33 </td> <td style="text-align:right;"> 1.07 </td> </tr> <tr> <td style="text-align:left;"> DietEx </td> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 6.10 </td> <td style="text-align:right;"> 1.45 </td> </tr> </tbody> </table> {{content}} -- ```r with(df2, t.test(wl2[group=="DietEx"], wl2[group=="Control"], alternative = "greater")) ``` ``` ## ## Welch Two Sample t-test ## ## data: wl2[group == "DietEx"] and wl2[group == "Control"] ## t = 5.0018, df = 16.317, p-value = 6.155e-05 ## alternative hypothesis: true difference in means is greater than 0 ## 95 percent confidence interval: ## 1.802104 Inf ## sample estimates: ## mean of x mean of y ## 6.100000 3.333333 ``` --- class: transition # What about if we change the visual test statistic? --- # .monash-blue[Lineup] .circle.monash-bg-blue.white[3] In which plot is the pink group higher than the blue group? `geom_point()` .white[Data plot is Plot 3] <img src="images/week11A/weight-lineup3-1.png" width="936" style="display: block; margin: auto;" /> --- # .monash-blue[Lineup] .circle.monash-bg-blue.white[4] In which plot is the pink group higher than the blue group? `geom_boxplot()` .white[Data plot is Plot 2] <img src="images/week11A/weight-lineup4-1.png" width="936" style="display: block; margin: auto;" /> --- # .monash-blue[Lineup] .circle.monash-bg-blue.white[5] In which plot is the pink group higher than the blue group? `geom_violin()` .white[Data plot is Plot 4] <img src="images/week11A/weight-lineup5-1.png" width="936" style="display: block; margin: auto;" /> --- # .monash-blue[Lineup] .circle.monash-bg-blue.white[6] In which plot is the pink group higher than the blue group? `ggbeeswarm::geom_quasirandom()` .white[Data plot is Plot 10] <img src="images/week11A/weight-lineup6-1.png" width="936" style="display: block; margin: auto;" /> --- # .orange[Case study] .circle.white.bg-orange[1] Weight loss by exercise .flex[ .w-40[ .panelset[ .panel[.panel-name[📊] .f5[ <img src="images/week11A/weight-ex-plot3-1.png" width="360" style="display: block; margin: auto;" /> ] ] .panel[.panel-name[data] .h200.scroll-sign.f5[ ```r df3 <- filter(WeightLoss, group != "Control") ``` ]] .panel[.panel-name[R] .f5[ ```r gweight %+% df3 + aes(y = wl2) + labs(y = "Weight loss at 2 months") ``` ] ] ] ] .w-60[ * Is weight loss greater with *exercise* after 2 months? .f5[ <table class=" lightable-classic" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; width: auto !important; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="text-align:left;"> Group </th> <th style="text-align:right;"> N </th> <th style="text-align:right;"> Mean </th> <th style="text-align:right;"> Std. Dev </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Diet </td> <td style="text-align:right;"> 12 </td> <td style="text-align:right;"> 3.92 </td> <td style="text-align:right;"> 1.38 </td> </tr> <tr> <td style="text-align:left;"> DietEx </td> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 6.10 </td> <td style="text-align:right;"> 1.45 </td> </tr> </tbody> </table> ```r with(df3, t.test(wl2[group=="DietEx"], wl2[group=="Diet"], alternative = "greater")) ``` ``` ## ## Welch Two Sample t-test ## ## data: wl2[group == "DietEx"] and wl2[group == "Diet"] ## t = 3.5969, df = 18.901, p-value = 0.0009675 ## alternative hypothesis: true difference in means is greater than 0 ## 95 percent confidence interval: ## 1.133454 Inf ## sample estimates: ## mean of x mean of y ## 6.100000 3.916667 ``` ] ] ] --- # Power of a lineup .w-70[ * The power of a lineup is calculated as `\(x/n\)` where `\(x\)` is the number of people who detected the data plot out of `\(n\)` people <br> {{content}} ] .footnote.f4[ Hofmann, H., L. Follett, M. Majumder, and D. Cook. 2012. “Graphical Tests for Power Comparison of Competing Designs.” IEEE Transactions on Visualization and Computer Graphics 18 (12): 2441–48. ] -- Plot type | `\(x\)` | `\(n\)` | Power --- | --- | --- | --- `geom_point` | `\(x_1\)` | `\(n_1\)` | `\(x_1 / n_1\)` `geom_boxplot` | `\(x_2\)` | `\(n_2\)` | `\(x_2 / n_2\)` `geom_violin` | `\(x_3\)` | `\(n_3\)` | `\(x_3 / n_3\)` `ggbeeswarm::geom_quasirandom` | `\(x_4\)` | `\(n_4\)` | `\(x_4 / n_4\)` <br> -- * The plot type with a higher power is preferable -- * You can use this framework to find the optimal plot design --- # Some considerations in visual inference * In practice you don't want to bias the judgement of the human viewers so for a proper visual inference: * you should _not_ show the data plot before the lineup * you should _not_ give the context of the data * you should remove labels in plots * You can crowd source these by paying for services like: * [Amazon Mechanical Turk](https://www.mturk.com/), * [Appen (formerly Figure Eight)](https://appen.com/figure-eight-is-now-appen/) and * [LABVANCED](https://www.labvanced.com/). * If the data is for research purposes, then you may need ethics approval for publication. --- # Resources and Acknowledgement .font18[ - Buja, Andreas, Dianne Cook, Heike Hofmann, Michael Lawrence, Eun-Kyung Lee, Deborah F. Swayne, and Hadley Wickham. 2009. “Statistical Inference for Exploratory Data Analysis and Model Diagnostics.” Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences 367 (1906): 4361–83. - Wickham, Hadley, Dianne Cook, Heike Hofmann, and Andreas Buja. 2010. “Graphical Inference for Infovis.” IEEE Transactions on Visualization and Computer Graphics 16 (6): 973–79. - Hofmann, H., L. Follett, M. Majumder, and D. Cook. 2012. “Graphical Tests for Power Comparison of Competing Designs.” IEEE Transactions on Visualization and Computer Graphics 18 (12): 2441–48. - Majumder, M., Heiki Hofmann, and Dianne Cook. 2013. “Validation of Visual Statistical Inference, Applied to Linear Models.” Journal of the American Statistical Association 108 (503): 942–56. - Data coding using [`tidyverse` suite of R packages](https://www.tidyverse.org) - Slides constructed with [`xaringan`](https://github.com/yihui/xaringan), [remark.js](https://remarkjs.com), [`knitr`](http://yihui.name/knitr), and [R Markdown](https://rmarkdown.rstudio.com). ] --- background-size: cover class: title-slide background-image: url("images/bg-01.png") <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-sa/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>. .bottom_abs.width100[ Lecturer: *Emi Tanaka* <i class="fas fa-envelope"></i> ETC5521.Clayton-x@monash.edu <i class="fas fa-calendar-alt"></i> Week 11 - Session 1 <br> ]