How to conduct a DOE

A DOE (Design Of Experiments) is structured, planned method, which is used to find the relationship between different factors that affect a tested subject and the different outputs. This is done with multiple factors on various levels combined in one experiment. Instead of testing each factor individually in a DOE multiple factors are variated at once to reduce the amount of test with the possibility to analyze interactions between factors.

What is a Factor

A factor is an input for a experiment that can influence the output when variating. Its like a dimmer (factor) of a lamp when turning the knob the brightness of the lamp changes.

Why use a complex DOE instead of the standard approach?

With the classical approach only one input factor is changed to determine the influence of it on the output with a DOE more than one input is changed to see the influence of multiple factors on the output.

So the answer is to reduce the amount of experiments.
In other words reduce cost and time.

As visible in the below table with a classical approach with 16 tests 1 factor is examined and with a DOE approach 4. With an other matrix this could be even more!

	Classical		DOE
Test			Factor 1	Factor 2	Factor 3	Factor 4
1	High		High	High	High	High
2	High		High	High	High	Low
3	High		High	High	Low	High
4	High		High	High	Low	Low
5	High		High	Low	High	High
6	High		High	Low	High	Low
7	High		High	Low	Low	High
8	High		High	Low	Low	Low
9	Low		Low	High	High	High
10	Low		Low	High	High	Low
11	Low		Low	High	Low	High
12	Low		Low	High	Low	Low
13	Low		Low	Low	High	High
14	Low		Low	Low	High	Low
15	Low		Low	Low	Low	High
16	Low		Low	Low	Low	Low

But this is not all, with a DOE it is possible to investigate interactions and create response surface graphs. About these later more.

Of course it is possible to do the classical test with only 1 High and 1 Low test but then the statistical power is lower as with the DOE approach.

With more than 2 factors it is wise to use a DOE approach.

What is important for a successful DOE?

1. The output can be measured, preferable in continuous scale!!
2. The influencing factors are known
3. Important Factors can be controlled (variated on a desired level or fixed on a constant level)
4. Try to control Noise (uncontrollable factors) or record them (Environment temperature, Air pressure, different operators, ect)
5. Keep the DOE simple as possible
6. DO the Confirmation Run!

The Level of the factor

The level of a factor is the input setting of test. For the lamp dimmer the setting of the dimmer is the level (0%, 25%, 50% ect).
But the bigness of a lamp can have more influences from different factors with it own levels:

Factor	Levels
Shape of lamp	Ball, Cone, Candle
Power	1, 2, 9, 30, 40 60 100Watt
Setting on the dimmer	0%…100%
Input current	0...230V
Color of glass	Clear, White, Silver, Green, Red
Type of lamp	Light bulb, LED, TL
Armature	Silver reflector, White reflector, No reflector

Settings of levels

1. Try to chose realistic values for the levels (not impractical high/low or outside the machine settings)
2. Avoid impossible combinations of the levels with other factors in the experiment

The test Arrays

For a DOE there are various types of Arrays each with it own pros and cons. See table below.

Name	Full factorial	Response surface	Orthogonal
Type	Full Factorial Array	Box-Behnken design	Orthogonal Array
		Central Composite design	Plackett–Burman Array
Amount of tests	High (all combinations)	Medium	Small
			No interactions
Usage	Simple	More complex	Simple no interactions
			Complex with interactions
Interactions	All interactions	First level interactions	Yes possible
Response Surface design	No	Yes	No

When starting for the first time with a DOE start with a simple and small DOE

• Full factorial with a small amount of factors or Orthogonal without interactions.

A proper DOE array is in balance!

Amount of test with different arrays

There are various types of arrays the simple one is the Full Factorial array, this array consisted of all the combinations of the levels. The amount of test with a Full Factorial is high, therefor there are arrays designed to reduce this.

Test	Full Factorial Array				Box-Behnken design BB3				Orthogonal Array L9
	Factor 1	Factor 2	Factor 3		Factor 1	Factor 2	Factor 3		Factor 1	Factor 2	Factor 3	Factor 4
1	-1	-1	-1		0	0	0		-1	-1	-1	-1
2	0	-1	-1		-1	-1	0		-1	0	0	0
3	1	-1	-1		1	-1	0		-1	1	1	1
4	-1	0	-1		-1	1	0		0	-1	0	1
5	0	0	-1		1	1	0		0	0	1	-1
6	1	0	-1		0	-1	-1		0	1	-1	0
7	-1	1	-1		0	0	0		1	-1	1	0
8	0	1	-1		0	-1	1		1	0	-1	1
9	1	1	-1		0	1	-1		1	1	0	-1
10	-1	-1	0		0	1	1
11	0	-1	0		-1	0	-1
12	1	-1	0		-1	0	1
13	-1	0	0		1	0	-1
14	0	0	0		1	0	1
15	1	0	0		0	0	0
16	-1	1	0
17	0	1	0
18	1	1	0
19	-1	-1	1
20	0	-1	1
21	1	-1	1
22	-1	0	1
23	0	0	1
24	1	0	1
25	-1	1	1
26	0	1	1
27	1	1	1

As visible for the orthogonal array the smallest amount of test is needed. With this small amount of test even 4 factors can be tested on 3 levels (-1,0 and 1) for the other arrays types only 3 factors can be tested on 3 levels!
Of course this small amount of test is not for free.... With the L9 orthogonal array interactions and Response Surface can't be calculated and the power is lower. The power can easily increased by adding repetitions.

Array selection

The choice between arrays for a DOE is between Full Factorial and a Fractional array (Orthogonal) if you not want to draw a Response surface. Looking to the table below it is clear that quite fast a Full Factorial array is not economical.

	2 Levels			3 Levels
Factors	Full Factorial	Orthogonal		Full Factorial	Orthogonal	Box-Behnken
	Test Runs	Test Runs		Test Runs	Test Runs	Test Runs
2	4	4		9	9
3	8	4		27	9	16
4	16	8		81	9	26
5	32	8		243	18	45
6	64	8		729	18	54
7	128	8		2187	18
8	256	12		6561	27

Above 4 factors with 2 levels go for a Orthogonal array, and above 3 factors with 3 levels the advise is to go for an Orthogonal array. If interactions are needed a Box-Behnken design or Central Composite design. When building the samples for an Orthogonal array a reduced amount of different samples need to be build. If the power is to low the array can easily be replicated to increase the sample size.

Interactions

A interaction is when the result is not the sum of two factors. With an interaction it can occur that the result of the to factors is lower or higher as the sum of the result.
When using an Orthogonal array interactions are not included. The interactions have to be included in the array this can be calculated with Develve see.
Be careful with adding interactions in a design:

• A interaction is not a common thing
• Adding interactions will increase the array size especially in a 3 level array (2 extra test runs)
• If it is not logic that an interaction between factors do not include it

Interaction with the typical crossing line	No interaction

2 or 3 level design

	2 Levels	3 Levels
Amount of tests	Low	Higher
Non linear response	No	Yes

There are Orthogonal arrays with 2 or 3 levels and even with different amount of levels in one array. With Orthogonal Arrays it is possible to combining columns to get factors with more levels. When generating Full Factorial array for each factor an dedicated amount of levels can be chosen.

Example without interactions

We want to create a DOE with the following factors

Factor	Level 1	Level 2
Size	10mm	20mm
Weight	1kg	10kg
Color	White	Black

There are 3 factors and in a L4 is space for 3 factors.

Example with interactions

We want to create a DOE with the following factors

Factor	Level 1	Level 2
Size	10mm	20mm
Weight	1kg	10kg
Color	White	Black
Interaction 1	Size	Weight
Interaction 2	Size	Color
Interaction 3	Weight	Color

There are 3 factors, in a L4 array is space for 3 Factors on 2 levels. But we want to include interactions so a bigger array is needed in L8 is space for 7 factors.
Open this array.

Now add the first interaction. Column C is now used for the interaction.	The second interaction. Column E is also used for an interaction.
The third interaction. Column F is now also used for a interaction.	The name of the factors and levels are added.

With this array all the factors and first order interaction can be tested. The not used columns can be removed (C, E, F and G).
As you can see the result is the same as a Full factorial array this is because all the interactions between the factors are tested.

Building and testing the samples

Now build the samples according the array.
Fist sample 10mm, 1kg and white etc.

Some important points

• Always build and test the complete array.
• When adding repetitions it is preferred is not to test replicates on a row (samples with the same setting) but first finish the first replicate then the next. This is to randomize the order to prevent drift over time in the result.
• When testing try not to test all factors on the first level than on the second level but try to randomize.
• Do not add an extra test in the array except Center points this will create an unbalanced array, and can lead to wrong results.

Analyzing the result

Put the measurement data in the input table. In this case there is one replicate. See sort data in subgroups for how it is calculated.

Data file
Looking to the data

• Size 20mm gives a higher output this factor is also significant according the Anova
• Weight Small influence not significant. But it looks that it has a big influence on the variation! This can be seen in the Response graph Column D and E.
• Color Small not significant influence
• The interaction between Size and Weight is significant. So if Weight is combined with Size Weight is significant (see Anova)! In the Response graph Column H and I the typical crossing lines that is an indication for an interaction.
  • The interaction is the cause of the bigger variation of Weight at 1 kg!

Anova DOE

To check on significance influence of each factor in the DOE. If a factor is not significant it can bee pooled, and the influence of the non pooled factors is getting bigger.

Weight and color are not significant.

After pooling Color (the least significant factor) and Weight are still not significant.

Confirmation Run

In the Anova Dialog the optimal setting can selected in the column Level confirmation run. To confirm output (14.43) of the DOE build samples according these settings!

When a interaction is significant only set the level of the interaction do not set them individual. If you do both you set the influence of the factor twice!