This tutorial will help you set up and interpret a Check-all-that-apply (CATA) analysis in Excel using the XLSTAT statistical software.
Dataset for running a CATA analysis in XLSTAT
For this tutorial, we use data provided by Ares et al. (2014). They correspond to the assessment of 6 products (5 regular and 1 ideal) by 119 consumers over 15 attributes. Data is recorded in a binary format (0: attribute not ticked; 1: attribute ticked). Moreover, every product (except the ideal one) is rated overall (0-10) by each consumer.
Data is in a vertical format, which means that we have one row per combination of consumer and product.
An Excel sheet with both the data and the results can be downloaded by clicking here.
Goal of this tutorial
The aim of this tutorial is to conduct a CATA (check-all-that-apply) analysis to characterize products tested by consumers.
Setting up a CATA analysis in XLSTAT
To conduct a CATA analysis, click on XLSTAT-Sensory data analysis / CATA data analysis.
In the General tab, first make sure you select the vertical data format. In the CATA data field, select the attribute table. Then select the Consumer, Sample and liking columns in the In the Assessors, Products and Preference data fields, respectively. Capture the ideal product identifier in the Ideal product field.
In the Options(1) tab, select the Chi-square distance for the Correspondence Analysis.
Click the OK button. A dialog box appears for the user to select and validate the axes to be displayed on the graphical representation of the correspondence analysis.
Interpreting the results of a CATA analysis in XLSTAT, first part
The first table only takes into account data arising from attributes and surveyed products. The first column contains p-values associated to Cochran’s Q tests which compares products independently for each attribute. The remaining of the table contains proportions of 1’s across assessors for each combination of products and attributes.
A high proportion means the attribute is frequently ticked by consumers for the considered product.
For a given attribute, Cochran’s Q test allows to test the effect of an explanatory variable (Products) on whether the consumers feel the attribute or not. A low p-value beyond a significance threshold indicates that products significantly differ from each other. If the p-value is significant, the user may be interested by examining multiple pairwise comparisons, represented by small letters inside table cells: two products sharing the same letter(s) do not differ significantly. Two products having no letter in common differ significantly.
We can see that all of the attributes except two related to smelling (odourless and intense odour) are associated to significant p-values at 0.05. For example, if we consider the crispy attribute, we see that the product 257 is the “crispiest”, but is not significantly crispier than the 548 (check the letters). Products 992 and 366 are the least crispy and do not differ significantly from each other.
The following contingency table is the sum of attribute tables across assessors. It is used to construct a correspondence analysis (CA).
The independence between the rows and columns is tested (this result is currently only available for the classic CA (using the Chi-square distance)). As the p-value is lower than the significance level (0.05) we conclude that it is very likely that real differences exist between the products in terms of their sensory profiles.
The table of the eigenvalues and the corresponding plot allow to verify the quality of the analysis. The quality of the analysis is good (90.39% of explained total inertia on the first two dimensions). According to the map of the analysis, an ideal product should be relatively tasty, juicy, crispy, firm and sweet and have a relatively intense odour and an apple flavour.
On the other hand, it should not be relatively too sour, bitter, astringent, grainy, soft, mealy, tasteless or odourless. Product 548 seems to be the closest to the ideal product whereas product 106 is far away because of its relative bitterness, sourness and astringency. Products 366 and 992 are also relatively far from the ideal product.
More information about correspondence analysis is available here.
Then, a correlation matrix including attributes (tetrachoric correlation) and liking scores (biserial correlation, last row) is displayed. We see some strong correlations. The negative correlation between sweet and sour indicates that when people tick sour, they do not tick sweet, and vice versa. Liking scores seem to be positively –although weakly- correlated to attributes that were linked to the ideal product in the correspondence analysis (juicy, tasty, apple flavour).
Principal Coordinates Analysis (PCoA) is applied to the correlation coefficients and results are visualized in a two dimensional map. The first two dimensions explains 40.78% of the variation and the scree plot indicates that the two first dimensions are sufficient to interpret relationships between attributes. Here again, we see that liking is associated to the attributes juicy, tasty and apple flavour.
More information about principal coordinate analysis is available here.
Interpreting the results of a CATA analysis in XLSTAT, second part
When liking data is available, next results are related to the penalty analysis.
A first analysis based on incongruence in which the attribute is missing in the real but not the ideal product allow to identify the must have attributes. A summary table indicates the frequencies with which P(No)|(Yes) and P(Yes)|(Yes) occurs for each attribute. The graphical representation that follows shows these frequencies as well as the percentage of records for these occurrences.
Mean drops in liking between the two situations are then presented for each attribute and their significances tested. For example, the firm attribute implies an increase of 1.5 Liking points between the tested products and the ideal product. This increase is significant at 0.05 (P < 0.0001).
Note: In the case where there is no ideal product, this analysis is substituted by an analysis of presence and absence of the attributes.
The mean impact chart shows the attributes with a significant mean impact. Mean increases are displayed in blue and are identified as “must haves”, mean decreases are displayed in red.
The mean drops vs % chart also allows to clearly identify “must haves”. The Y axis corresponds to differences between cell [1,1] and cell [0,1] (checked for ideal and product minus checked only for ideal). The X axis represents the percentage of responses including a check of the ideal product without a check of the product for a given attribute, which correspond to situations where the attribute describes well the ideal product but is somehow missing in real products. Therefore, attributes that are associated to high coordinates on both the X and Y axes (tasty, sweet, juicy, apple flavour, crispy, firm, intense odour) appear here again to be “must haves”.
A second analysis allow to identify the “nice to have” attributes. It is similar to the first one but is based on incongruence in which the attribute is missing in the ideal but not the real product.
Note: This analysis is only available when an ideal product is available.
The mean impact chart shows the attributes with a significant mean impact. Mean increases are displayed in blue and are identified as “nice to haves”, mean decreases are displayed in red and are identified as must not haves.
The mean drops vs % chart also allows to clearly identify “must not haves” and “nice to haves”. The Y axis corresponds to differences between cell [0,0] and cell [1,0] (not checked for ideal and product minus checked only for product). The X axis represents the percentage of responses including a check of the product without a check of the ideal product for a given attribute, which correspond to situations where the attribute describes well the real products but is somehow missing in the ideal product. Therefore, attributes that are associated to low coordinate on the Y axis (sour, bitter, soft, mealy, grainy, tasteless and odourless) appear here again to be “must not haves”. Attributes associated to high coordinates on the Y axis are “nice to haves”.
The two previous analyses are finally summarized in one map. Here again, tasty, sweet, apple flavour, firm, crispy and intense odour appear as “must haves“; and sour, bitter astringent, soft, mealy, grainy, tasteless and odourless are “must not haves”.
Then, one 2x2 table is displayed per attribute. On the left of each table, we have the values recorded for the ideal product and at the top, the values obtained for the surveyed products. In the cells of the tables, we can find the average preference (averaged over the assessors and the products) and the % of all records that correspond to this combination of 0s and/or 1s.
For a given attribute, if the attribute is checked for the ideal product (second row), then if the preference for the products that are checked (cell [1,1]) is higher than when it is not checked (cell [1,0]), then the attribute is a “must have”. Symmetrically, if the attribute is not checked for the ideal product (first row), then if the preference for the products that are not checked (cell [0,0]) is higher than when it is checked (cell [0,1]), then the attribute is a “must not have”. If the attribute is not checked for the ideal product (first row), and if the preference for the products that are checked (cell [0,1]) is about the same as when it is not checked (cell [0,0]), then the attribute is a “does not harm”. Finally, if (cell [0,1]) > (cell [0,0]), then the attribute is a “nice to have”.
For example, for the tasty attribute, 28% of the surveyed (not ideal) product records are checked for both the surveyed product and the ideal product. The average liking of these records is 7.8.
In the final summary table, we can see that 6 out of the 15 attributes are “must haves” and 7 attributes are a "must not haves". The remaining 2 attributes are "does not harms".
Ares G., Dauber C., Fernández E., Giménez A., & Varela P. (2014). Penalty analysis based on CATA questions to identify drivers of liking and directions for product reformulation. Food Quality and Preference, 32A, 65-76.
Meyners M., Castura J. C., Carr B. T. (2013). Existing and new approaches for the analysis of CATA data. Food Quality and Preference, 30(2), 309-319.