Main Effect and interactions}
This is the simple effect of a factor on a dependent variable. It is the effect of the factor alone averaged across the levels of other factors.
An experiment is a process or study that results in the collection of data. The results of experiments are not known in advance. Usually, statistical experiments are conducted in situations in which researchers can manipulate the conditions of the experiment and can control the factors that are irrelevant to the research objectives.
For example, a rental car company compares the tread wear of four brands of tires, while also controlling for the type of car, speed, road surface, weather, and driver.
Experimental design is the process of planning a study to meet specified objectives. Planning an experiment properly is very important in order to ensure that the right type of data and a sufficient sample size and power are available to answer the research questions of interest as clearly and efficiently as possible, with consideration for the amount of resources available to carry out such experiments.
Perform the following steps when designing an experiment: 1. Define the problem and the questions to be addressed. 2. Define the population of interest. 3. Determine the need for sampling. 4. Define the experimental design.
A clear definition of the details of the experiment makes the desired statistical analyses possible, and almost always improves the usefulness of the results. The overall data collection and analysis plan considers how the experimental factors, both controlled and uncontrolled, fit together into a model that will meet the specific objectives of the experiment and satisfy the practical constraints of time and money.
The data collection and analysis plan provides the maximum amount of information that is relevant to a problem by using the available resources most efficiently. Understanding how the relevant variables fit into the design structure indicates whether the appropriate data will be collected in a way that permits an objective analysis that leads to valid inferences with respect to the stated problem.
The desired result is to produce a layout of the design along with an explanation of its structure and the necessary statistical analyses. The data collection protocol documents the details of the experiment such as the data definition, the structure of the design, the method of data collection, and the type of analyses to be applied to the data.
Defining the experimental design consists of the following steps:
Factor is any aspect of the experimental conditions which may affect the result obtained form an experiment.
Treatments are administered to experimental units by level, where level implies amount or magnitude. For example, if the experimental units were given 5mg, 10mg, 15mg of a medication, those amounts would be three levels of the treatment.
Level is also used for categorical variables, such as Drugs A, B, and C, where the three are different kinds of drug, not different amounts of the same thing.
A cholesterol reduction clinic has two diets and one exercise regime. It was found that exercise alone was effective, and diet alone was effective in reducing cholesterol levels (main effect of exercise and main effect of diet).
Also, for those patients who didn’t exercise, the two diets worked equally well (main effect of diet); those who followed diet A and exercised got the benefits of both (main effect of diet A and main effect of exercise). However, it was found that those patients who followed diet B and exercised got the benefits of both plus a bonus, an interaction effect (main effect of diet B, main effect of exercise plus an interaction effect).
If a treatment condition appears more than one time, it is defined to be replicated. True replication refers to responses that are treated in the same way.
Replication is essential for estimating experimental error. The type of replication that’s possible for a data collection plan represents how the error terms should be estimated.
Two or more measurements should be taken from each experimental unit at each combination of conditions, if possible.
In addition, it is desirable to have measurements taken at a later period in order to test for repeatability over time. The first method of replication gives an estimate of pure error, that is, the ability of the experimental units to provide similar results under identical experimental conditions.
Most experimental designs require experimental units to be allocated to treatments either randomly or randomly with constraints, as in blocked designs.
Blocks are groups of experimental units that are formed to be as homogeneous as possible with respect to the block characteristics.
The term block comes from the agricultural heritage of experimental design where a large block of land was selected for the various “treatments”, that had uniform soil, drainage, sunlight, and other important physical characteristics.
Homogeneous clusters improve the comparison of treatments by randomly allocating levels of the treatments within each block.
The design structure consists of those factors that define the blocking of the experimental units into clusters. The types of commonly used design structures are described are as follows.
Subjects are assigned to treatments completely at random. For example, in an clinical trial study, volunteer patients are randomly assigned to one of four treatment groups (three new types of a treatment and the standard).
Suppose the total number of patients in the 4 groups is 96 (i.e. randomly assign 1/4 of them, or 24 patients, to each of the 4 types of treatments).
Subjects are divided into b blocks according to demographic characteristics. Subjects in each block are then randomly assigned to treatments so that all treatment levels appear in each block.
For example, in the clinical study, the patients would presumably have different demographic characteristics. For the sake of simplicity, let’s say there are four main demographic subgroups A,B,C and D.
Patients within each subgroup are randomly assigned to one of the four types of treatments.
There might be significant variability between the subjects in each demographic subgroup, each of which contains 24 patients. Randomly assign 6 patients to each of the three types of tests and the standard.
The demographic subgroup is now the ‘block’. The primary interest is in the main effect of the test.
A researcher is carrying out a study of the effectiveness of four different skin creams for the treatment of a certain skin disease. He has eighty subjects and plans to divide them into 4 treatment groups of twenty subjects each.
Using a randomised blocks design, the subjects are assessed and put in blocks of four according to how severe their skin condition is; the four most severe cases are the first block, the next four most severe cases are the second block, and so on to the twentieth block. The four members of each block are then randomly assigned, one to each of the four treatment groups.
An effect is a change in the response due to a change in a factor level. There are different types of effects. One objective of an experiment is to determine if there are significant differences in the responses across levels of a treatment (a fixed effect) or any interaction between the treatment levels.
If this is always the case, the analysis is usually easily manageable, given that the anomalies in the data are minimal (outliers, missing data, homogeneous variances, unbalanced sample sizes, and so on).
In a medical experiment, the comparison of treatments may be distorted if the patient, the person administering the treatment and those evaluating it know which treatment is being allocated. It is therefore necessary to ensure that the patient and/or the person administering the treatment and/or the trial evaluators are ‘blind to’ (don’t know) which treatment is allocated to whom. Sometimes the experimental set-up of a clinical trial is referred to as double-blind, that is, neither the patient nor those treating and evaluating their condition are aware (they are ‘blind’ as to) which treatment a particular patient is allocated. A double-blind study is the most scientifically acceptable option. Sometimes however, a double-blind study is impossible, for example in surgery. It might still be important though to have a single-blind trial in which the patient only is unaware of the treatment received, or in other instances, it may be important to have blinded evaluation.
Introduction Analysis of variance (ANOVA) is a popular tool that has an applicability and power that we can only start to appreciate in this course. The idea of analysis of variance is to investigate how variation in structured data can be split into pieces associated with components of that structure. We look only at one-way and two-way classifications, providing tests and confidence intervals that are widely used in practice.
Analysis of variance (ANOVA) is a popular tool that has wide applicability in the sciences. The idea of analysis of variance is to investigate how variation in structured data can be split into pieces associated with components of that structure. For the next few classes, We look at one-way and two-way classifications, providing tests and confidence intervals that are widely used in practice.
One-way analysis of variance looks to see how much of the variation in grouped data comes from differences between the groups, and how much is just random observational error. There can be any number of groups, that may be of different sizes (each group with at least two observations). A typical application of one-way analysis of variance would be to investigate whether three different types of growing conditions make any difference to the yield of an agricultural crop, and if so, how great those differences are. The observations would be the yields of many different experimental plots, grouped according to the growing condition that applied to them.
We compute the test statistics F = 62=3 .. 20:7 while the \(95\%\) quantile of F distribution with 3 and 8 degrees of freedom is given as
qf(0.95,3,8)
# 4.066181
We clearly see that the test informs us about a significant difference between the means. But which means are different? The least significant difference method described in Section 3.9: We compute the least significant difference s p 2=n … t, where s 2 is within sample estimate of variance and \(t\) is the \(97.5\%\) quantile of Student-t distribution with h(n 1) degrees of freedom.
sqrt(mean(s))*sqrt(2/3)*qt(0.975,8)
# 3.261182
m=apply(x,1,mean)
m
#[1] 101 102 97 92
The associated degrees of freedom: for within-sample h(n 1) (in our example \(4 \times 2 = 8\)), for between-sample h 1 (in our example 3). Total number of degrees freedom hn 1 and we see \(hn 1 = h(n 1) + h 1:\) But there is more then the relation between degrees of freedom. Namely \[ SST = SSM + SSR \]
where \[ SST = \]
and \[ SSM = \]
x=c(102,100,101,101,101,104,97,95,99,90,92,94)
factors=c(rep("A",3),rep("B",3),rep("C",3),rep("D",3))
res=aov(x~factors)
anova(res)
Analysis of Variance Table
Response: x
Df Sum Sq Mean Sq F value Pr(>F)
factors 3 186 62 20.667 0.0004002 ***
Residuals 8 24 3
---
Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1
An experiment is a process or study that results in the collection of data. The results of experiments are not known in advance. Usually, statistical experiments are conducted in situations in which researchers can manipulate the conditions of the experiment and can control the factors that are irrelevant to the research objectives.
For example, a rental car company compares the tread wear of four brands of tires, while also controlling for the type of car, speed, road surface, weather, and driver.
Experimental design is the process of planning a study to meet specified objectives. Planning an experiment properly is very important in order to ensure that the right type of data and a sufficient sample size and power are available to answer the research questions of interest as clearly and efficiently as possible, with consideration for the amount of resources available to carry out such experiments.
Perform the following steps when designing an experiment: 1. Define the problem and the questions to be addressed. 2. Define the population of interest. 3. Determine the need for sampling. 4. Define the experimental design.
The subject matter for steps 1 to3 were covered in Science Maths 3 (MA4603). We will look at step 4 specifically in this module.
A clear definition of the details of the experiment makes the desired statistical analyses possible, and almost always improves the usefulness of the results. The overall data collection and analysis plan considers how the experimental factors, both controlled and uncontrolled, fit together into a model that will meet the specific objectives of the experiment and satisfy the practical constraints of time and money.
The data collection and analysis plan provides the maximum amount of information that is relevant to a problem by using the available resources most efficiently. Understanding how the relevant variables fit into the design structure indicates whether the appropriate data will be collected in a way that permits an objective analysis that leads to valid inferences with respect to the stated problem.
The desired result is to produce a layout of the design along with an explanation of its structure and the necessary statistical analyses. The data collection protocol documents the details of the experiment such as the data definition, the structure of the design, the method of data collection, and the type of analyses to be applied to the data.
Defining the experimental design consists of the following steps: \begin{enumerate} * Identify the experimental unit. * Identify the types of variables. * Define the treatment structure. * Define the design structure.
Factor is any aspect of the experimental conditions which may affect the result obtained form an experiment.
Controlled Factor is any factor that can be altered by the experimenter at will.
Uncontrolled Factor is any factor that canât be freely altered.
Factor Levels are the discretized values of indicating the degree of presence of a given factor (for example, high and low).
Inconclusive results are likely to result if any of these classifications are not adequately defined. It is important to consider all the relevant variables (even those variables that might, at first, appear to be unnecessary) before the final data collection plan is approved in order to maximize confidence in the final results. * Primary variables are independent variables that are possible sources of variation in the response. These variables comprise the treatment and design structures and are referred to as Primary factors.
Uncontrollable factors are those variables that are known to exist, but conditions prevent them from being manipulated, or it is very difficult (due to cost or physical constraints) to measure them.
The experimental error is due to the influential effects of uncontrollable variables, which will result in less precise evaluations of the effects of the primary and background variables. The design of the experiment should eliminate or control these types of variables as much as possible in order to increase confidence in the final results.
Treatments are administered to experimental units by level, where level implies amount or magnitude. For example, if the experimental units were given 5mg, 10mg, 15mg of a medication, those amounts would be three levels of the treatment.
Level is also used for categorical variables, such as Drugs A, B, and C, where the three are different kinds of drug, not different amounts of the same thing.
This is the simple effect of a factor on a dependent variable. It is the effect of the factor alone averaged across the levels of other factors.
If a treatment condition appears more than one time, it is defined to be replicated. True replication refers to responses that are treated in the same way.
Replication is essential for estimating experimental error. The type of replication thatâs possible for a data collection plan represents how the error terms should be estimated.
Two or more measurements should be taken from each experimental unit at each combination of conditions, if possible.
In addition, it is desirable to have measurements taken at a later period in order to test for repeatability over time. The first method of replication gives an estimate of pure error, that is, the ability of the experimental units to provide similar results under identical experimental conditions.
Most experimental designs require experimental units to be allocated to treatments either randomly or randomly with constraints, as in blocked designs.
Blocks are groups of experimental units that are formed to be as homogeneous as possible with respect to the block characteristics.
The term block comes from the agricultural heritage of experimental design where a large block of land was selected for the various âtreatmentsâ, that had uniform soil, drainage, sunlight, and other important physical characteristics.
Homogeneous clusters improve the comparison of treatments by randomly allocating levels of the treatments within each block.
The design structure consists of those factors that define the blocking of the experimental units into clusters. The types of commonly used design structures are described are as follows.
Subjects are assigned to treatments completely at random. For example, in an clinical trial study, volunteer patients are randomly assigned to one of four treatment groups (three new types of a treatment and the standard).
Suppose the total number of patients in the 4 groups is 96 (i.e. randomly assign 1/4 of them, or 24 patients, to each of the 4 types of treatments).
Test Method Standard New Treatment 1 New Treatment 2 New Treatment 3 24 Patients 24 Patients 24 Patients 24 Patients
Subjects are divided into b blocks according to demographic characteristics. Subjects in each block are then randomly assigned to treatments so that all treatment levels appear in each block.
For example, in the clinical study, the patients would presumably have different demographic characteristics. For the sake of simplicity, letâs say there are four main demographic subgroups A,B,C and D.
Patients within each subgroup are randomly assigned to one of the four types of treatments.
There might be significant variability between the subjects in each demographic subgroup, each of which contains 24 patients. Randomly assign 6 patients to each of the three types of tests and the standard.
The demographic subgroup is now the ‘block’. The primary interest is in the main effect of the test.
Treatment MethodSubgroups Standard Treatment 1 Treatment 2 Treatment 3 Group A 6 Patients 6 Patients 6 Patients 6 Patients Group B 6 Patients 6 Patients 6 Patients 6 Patients Group C 6 Patients 6 Patients 6 Patients 6 Patients Group D 6 Patients 6 Patients 6 Patients 6 Patients
The improvement of this design over a completely randomized design enables you to make comparisons among treatments after removing the effects of a confounding variable, in this case, different subgroups.
A researcher is carrying out a study of the effectiveness of four different skin creams for the treatment of a certain skin disease. He has eighty subjects and plans to divide them into 4 treatment groups of twenty subjects each.
Using a randomised blocks design, the subjects are assessed and put in blocks of four according to how severe their skin condition is; the four most severe cases are the first block, the next four most severe cases are the second block, and so on to the twentieth block. The four members of each block are then randomly assigned, one to each of the four treatment groups.
Treatment Method
Subgroups Cream A Cream B Cream C Cream D
Group 1 Patient 3 Patient 4 Patient 1 Patient 2
Group 2 Patient 8 Patient 7 Patient 6 Patient 5
Group 2 Patient 11 Patient 12 Patient 9 Patient 10
â¦. â¦. â¦. â¦. â¦.
An effect is a change in the response due to a change in a factor level. There are different types of effects. One objective of an experiment is to determine if there are significant differences in the responses across levels of a treatment (a fixed effect) or any interaction between the treatment levels.
If this is always the case, the analysis is usually easily manageable, given that the anomalies in the data are minimal (outliers, missing data, homogeneous variances, unbalanced sample sizes, and so on). ### Blinding} In a medical experiment, the comparison of treatments may be distorted if the patient, the person administering the treatment and those evaluating it know which treatment is being allocated. It is therefore necessary to ensure that the patient and/or the person administering the treatment and/or the trial evaluators are ‘blind to’ (don’t know) which treatment is allocated to whom.
Sometimes the experimental set-up of a clinical trial is referred to as double-blind, that is, neither the patient nor those treating and evaluating their condition are aware (they are ‘blind’ as to) which treatment a particular patient is allocated. A double-blind study is the most scientifically acceptable option.
Sometimes however, a double-blind study is impossible, for example in surgery. It might still be important though to have a single-blind trial in which the patient only is unaware of the treatment received, or in other instances, it may be important to have blinded evaluation.
Equal variances across samples is called homogeneity of variances. Bartlett’s test is used to test if multiple samples have equal variances.
Some statistical tests, such as the analysis of variance, assume that variances are equal across groups or samples. The Bartlett test can be used to verify that assumption.
A two-factor analysis of variance consists of three significance tests: a test of each of the two main effects and a test of the interaction of the variables. An analysis of variance summary table is a convenient way to display the results of the significance tests. A summary table for the hypothetical experiment described in the section on factorial designs and a graph of the means for the experiment are shown below.
Sum of Mean
SOURCE df Squares Square F p
T 1 47125.3333 47125.3333 384.174 0.000
D 2 42.6667 21.3333 0.174 0.841
TD 2 1418.6667 709.3333 5.783 0.006
ERROR 42 5152.0000 122.6667
TOTAL 47 53738.6667
The summary table shows four sources of variation: (1) Task, (2) Drug dosage, (3) the Task x Drug dosage interaction, and (4) Error.
The degrees of freedom total is always equal to the total number of numbers in the analysis minus one. The experiment on task and drug dosage had eight subjects in each of the six groups resulting in a total of 48 subjects. Therefore, df total = 48 - 1 = 47.
The degrees of freedom for the main effect of a factor is always equal to the number of levels of the factor minus one. Therefore, df task = 2 - 1 = 1 since there were two levels of task (simple and complex). Similarly, df dosage = 3 - 1 = 2 since there were three levels of drug dosage (0 mg, 100 mg, and 200 mg).
The degrees of freedom for an interaction is equal to the product of the degrees of freedom of the variables in the interaction. Thus, the degrees of freedom for the Task x Dosage interaction is the product of the degrees of freedom for task (1) and the degrees of freedom for dosage (2). Therefore, df Task x Dosage = 1 x 2 = 2.
The degrees of freedom error is equal to the degrees of freedom total minus the degrees of freedom for all the effects. Therefore, df error = 47 - 1 - 2 - 2 = 42.
As in the case of a one-factor design, each mean square is equal to the sum of squares divided by the degrees of freedom. For instance, Mean square dosage = 42.6667/2 = 21.333 where the sum of squares dosage is 42.6667 and the degrees of freedom dosage is 2.
The F ratio for an effect is computed by dividing the mean square for the effect by the mean square error. For example, the F ratio for the Task x Dosage interaction is computed by dividing the mean square for the interaction ( 709.3333) by the mean square error (122.6667). The resulting F ratio is: F = 709.3333/122.6667 = 5.783
The degrees of freedom denominator is equal to the degrees of freedom error. Therefore, the degrees of freedom for the F ratio for the main effect of task are 1 and 42, the degrees of freedom for the F ratio for the main effect of drug dosage are 2 and 42, and the degrees of freedom for the F for the Task x Dosage interaction are 2 and 42.
An F distribution calculator can be used to find the probability values. For the interaction, the probability value associated with an F of 5.783 with 2 and 42 df is 0.006.
When a main effect is significant, the null hypothesis that there is no main effect in the population can be rejected. In this example, the effect of task was significant. Therefore it can be concluded that, in the population, the mean time to complete the complex task is greater than the mean time to complete the simple task (hardly surprising). The effect of dosage was not significant. Therefore, there is no convincing evidence that the mean time to complete a task (in the population) is different for the three dosage levels
The significant Task x Dosage interaction indicates that the effect of dosage (in the population) differs depending on the level of task. Specifically, increasing the dosage slows down performance on the complex task and speeds up performance on the simple task. The effect of increasing the dosage therefore depends on whether the task is complex of simple.
There will always be some interaction in the sample data. The significance test of the interaction lets you know whether you can infer that there is an interaction in the population.