For a good result out of the test data, proper data collection is essential. This starts with how to measure the data. Is the characteristic measurable or is only a derivation measurable. In the best case the measurement result is also the characteristic we want to know. But in many cases this is not the case due; cost, time or the possibility to measure.
Example of derivations
CW factor (air resistance) of a car: You are actually interested in fuel consumption or noise of the car.
Scales of the measurement
When choosing a measurement method it is important to look at the scale of the measurement data. The characteristic of the output is important for what kind of statistic test can be used.
The goal is to measure data with a continuous scale!
Output is a continuous scale without a absolute zero. Like with Ratio you can use every fraction (or ratio) with a ratio variable. Without a true zero, it is impossible to compute ratios. Multiplying and dividing is not possible (20˚C is not twice as hot as 10˚C).
It happens that you are thinking the data is continuous (Ratio, Interval) but it is actually Ordinal data. This can be caused by a low resolution of your measurement equipment. Due low resolution of the measurement the data is rounded to the nearest digit. This leads to data that the data is grouped in small sets see graph. To solve this try to increase the measurement resolution. Use the histogram or the individual dot plot see if there is a rounding effect in the data.
When you have a Ratio or interval scale the shape is not always linear for instance with sound measurements the dBA is not linear but logarithmic. As result of this behavior the result may not be normally distributed and the data must be normalized or a Non parametric tests must be used.
It is crucial to have a good measurement this makes the statistics more significant with smaller sample sizes.