What is a Durbin, Skillings-Mack test
The Durbin test is an alternative to the Friedman test for the case where a study has been built using a balanced incomplete block design, knowing that the Friedman test requires a complete block design.
Reminder on block designs
A block design is a design in which we study the influence of two factors on one or more phenomena. We know that one factor has an impact that we cannot control, but that is not of interest. So we want to ensure that this factor does not disturb the analysis that we perform once the collected data. For this we make sure that the various levels of other factors are well represented in each block.
The blocking factor can correspond to judges evaluating products, and the factor of interest would then be the products being studied.
A complete block design is a design in which all levels of the factors of interest are present once within each block. For a sensory design, this corresponds to a design where all products are seen once by each judge.
In an incomplete block design, all levels of the factors of interest are not present for all levels of the blocking factor. It is balanced if each level of the factor of interest are present a same number of times r in the design, and if each pair of levels of each factor is present the same number of times λ.
If t is the number of treatments, b the number of blocks, k the number of treatments measured within each block, we show that the following conditions are necessary (but not sufficient) to have a balanced incomplete block design:
- r*(k-1)= λ*(t-1)
Using the Durbin test on a balanced incomplete block design (BIBD)
A study needs to be conducted to evaluate 5 products. 10 experts have been asked to rate the products. However, we know that for this type of evaluation, the experts can give reliable ratings only if they evaluate 3 products or less. We are therefore forced to carry out an incomplete block design. However there exists a balanced incomplete block plan for this case. It is such that each product will be seen by 6 consumers.
Note: The DOE function of XLSTAT-MX can be used to create such a design of experiment with also the possibility to optimize the order in which each expert sees the product, so there is no order effect.
The results of the study are shown in the table below:
We now want to know whether the products P1 to P5 can be considered equivalent or not. If the Friedman test can not handle such a case because of missing data, the Durbin test is ideal because we have here a balanced incomplete block design. Nevertheless, even if the design is not balanced, XLSTAT would adapt and use the Skillings-Mack statistic for the test.
As for the Friedman test, the null and alternative hypotheses used in the test are:
- H0 : The t treatments are not different.
- Ha : At least one of the treatments is different from another :
Dataset for a Durbin test
An Excel sheet with both the data and the results can be downloaded by clicking here.
Setting up a Durbin test in Excel using XLSTAT
Once XLSTAT-Pro is activated, select the XLSTAT / Nonparametric tests / Durbin, Skillings-Mack test command, or click on the corresponding button of the Nonparametric test menu (see below).
Once you've clicked the button, the dialog box appears. You can then select the data on the Excel sheet.
In the Options tab we choose to compute the asymptotic p-value to proceed as most software does.
After you have clicked on the OK button, the results are displayed on a new Excel sheet (because the Sheet option has been selected for outputs).
Interpreting the results of a Durbin test
XLSTAT can also use Monte Carlo simulations to get a better estimate of the p-value. In this particular case that the exact p-value (0.022) is intermediate between the Durbin’s approximation which is too conservative (as for what concerns H0), and the Conover’s approximation which is too pessimistic. However, in our particular case, as shown below, with both approximations we reject the null hypothesis H0.
As the H0 hypothesis has to be rejected, then at least one treatment is different from another. To identify which treatment(s) is/are responsible for rejecting H0, a multiple comparison procedure can be used. XLSTAT allows to use for the Durbin test the procedure suggested by Conover (1999).
Thus we see here that P4 and P1 are rated lower than P2 and P3. These differences (P4 <P3/P2) and (P3> P4/P1), are responsible for the rejection of H0.