Need a hand? All the help you want just a few clicks away
What is the criterion in order to choose a threshold value in model selection option of discrimiant analysis? e.g. stepwise (forward) 0.05 (threshold value to enter and remove) vs 0.10 vs 0.15 etc.. I have noticed that playing around with different values affects the confusion matrix and cross-validation results..The higher value (e.g. 0.50) the higher percentage is well-classified..is this approach correct before choosing a threshold value? thank you in advance
It is normal that the results change since the model equation changes. There is not really a unique way to decide which entry or removal threshold to select. Different opinions exist on this subject.
Here is a quote below that I find interesting:
From Draper & Smith’s “Applied Regression Analysis, Second Edition:
Some workers like to set the ‘exit alpha’ to be larger than the ‘entry alpha’ to provide some ‘protection’ for predictors already admitted to the equation. Such variations are a matter of personal taste which, together with alpha-values actually selected, have a great effect on the way a particular selection procedure behaves, an how many predictors are retained in the final equation. . . . To readers without strong personal opinions on this matter, we suggest setting alpha=0.05 or alpha=0.10 for both entry and exit tests, if the regression package being used allows this option. These levels can then be changed as experience dictates. As we discuss in the next section, the alpha values are not accurate measures anyway, so that detailed agonizing over the precise choice of alpha is hardly worthwhile. Typically the alpha=0.05 choice is conservative, that is, the actual value of alpha is much larger than 0.05 (some limited studies have been done and indicate this) and thus theire will be a tendency to admit more predictors than the user might anticipate
This question has received the maximum number of answers.
XLSTAT first steps and statistical learning resources
The place where you will master all of XLSTAT features
Any feature or option you’d like us to develop?
For those of you who prefer video help