10 Management
Experiments costs time and resources. Some experiments, you may not be able to repeat again. Designing an experiment goes beyond just producing an experimental design layout โ you need to consider holistically the total experimental process. Beyond the statistical and computational concerns, you need to be acquainted with some of the realities that come into play in designing experiments. This chapter aims to illuminate on the management aspects.
10.1 Communication
An experiment generally involves more than one person. For simplicity, let us suppose there are four actors:
- the domain expert who drives the experimental objective and has the intricate knowledge about the subject area,
- the statistician who creates the experimental design layout after taking into account statistical and practical constraints,
- the technician who carries out the experiment and collects the data, and
- the analyst who analyses the data after the data are collected.
The actors are purely illustrative and in practice, multiple people can take on each role, one person can take on multiple roles, and a person is not necessary a specialist in the role assigned (e.g. a statistician role can be carried out by a person whoโs primarily training is not in statistics). All the roles can be acted out by a single individual. Ideally, all parties should have clarity of the final experimental design.
To carry out this experiment, the people involved in the experiment must come to some degree of shared understanding about the experimental motivations, limitations and protocol. This understanding is achieved by communication. People may opt for a series of behavioural cycles that facilitate the process of clarification as they work to reduce uncertainty about their understanding.
Each actor does not need to have a full comprehension of the experiment, but rather focus on the most pertinent part of the experiment that is relevant to their role. For example in the guinea pig experiment [add ref], a statistician does not need to know how the guinea pigs were sourced for the experiment โ just that the subjects are genetically identical โ in fact, a statistician can generate an experimental design without even knowledge that the subjects are guinea pigs. From the view point of the statistician, the experiment can be simplified to its bare essential elements to generate the experimental design. Why then would there be any need to encode the context of the experiment in generating the experimental design?
The minimum required understanding for a statistician to conduct their role does not include understanding the experimental context. This is the reason perhaps that a lot of computational systems to generate the experimental design have often the inputs stripped off the experimental context. In an idealistic world, this approach would be fine but the major problem in this process is that the involved parties most likely do not know what is the most relevant knowledge for a statistician.
- Type of role: collaboration or consultation?
- Documentation
- Responsibility - who does what?
- Expectations - financial feasibility of experiment, execution of experiment etc
- Timeline
- Reward (financial, experience, joint work, social good, etc)
10.2 Project management
- Principles of project management from Project Management Institute (2021)
- Show how the framework relates to designing experiments
10.3 Data management
- Data standards (metrics for measurement, data dictionary, ISO 601 for representing times and dates, etc)
- Data storage, access, and distribution, including meta data
- Data quality indicators
- Data citation, data versioning
- FAIR principles for scientific data
- Planning for data provenance
- Responsible data stewardship
See Section 13.3 for some specific reporting protocol.