3 Choices
3.1 Choices of units
3.2 Choices of treatments
The choice of treatments must be guided by the domain expert, however in some cases this may be guided by statistics. These generally include treatments that can be encoded as a numerical value, such as in dose finding experiments or Latin hyper cube sampling. In dose finding experiment, small-scale screening experiments may be done first to determine the dose levels for the larger trial. For Latin hyper cube sampling, several factors that have a continuous range and the level is sampled in a way that maximises a certain criterion.
The effectiveness of any treatment is only established in relation to another treatment, e.g. the control. As an analogy, consider a student that has been heralded as a mathematical genius because she has been getting perfect scores in all her assessments. But if you find out that all the other students in her class has been getting perfect scores as well, then you may rescind calling her a mathematical genius. The thing is though, all the students could indeed be mathematical geniuses but without comparisons with typical mathematics students, you will not be sure. Wrong comparisons could also be made, e.g. comparing the student with a member of general public, who does not practice mathematics day-to-day, leads to a logical fallacy of student appearing to excel in mathematics, when in fact it would be expected she would be better at mathematics than those who don’t make regular use of it.
The control treatment should be chosen such that the comparison is meaningful. For example, a new cancer treatment shouldn’t be compared to “no treatment” but to the standard approach to treat cancer. Not only may it be unethical to assign a “no treatment” to a cancer patient, it is wasteful of experimental resources to design experiments where there there is little interest in comparing to “no treatment”.