 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Math Forum / Mathematics / Statistics / October 2008which test can be used?
I have the following problem: there are two samples (30 data each one), which don't follow a normal distribution, I want a test to affirm with an \alpha=0.01 that the to samples follow the same distribution. All the test I known are to make affirmations like "the two samples are significantly different" or "the two samples are not significantly different", but in this case I need a test to affirm that the two samples are significantly "similar". > I have the following problem: > there are two samples (30 data each one), which don't follow a normal [quoted text clipped - 6 lines] > different", but in this case I need a test to affirm that the two > samples are significantly "similar". First of all, you're never going to be able to make your affirmation. All you may be able to say is that you don't have enough data to disprove it. Your wording suggests you don't understand this important aspect of hypothesis testing. Please find a good elementary textbook on the subject (or look online) and do some reading. In your shoes I would probably look at an empirical Q-Q plot. See http://www.itl.nist.gov/div898/handbook/eda/section3/qqplot.htm (The Wikipedia article on QQ plots describes a theoretical QQ plot.) If you really must use a test, you could start here: http://en.wikipedia.org/wiki/Kolmogorov-Smirnov_test >> I have the following problem: >> there are two samples (30 data each one), which don't follow a normal [quoted text clipped - 8 lines] > >First of all, you're never going to be able to make your affirmation. Absolutely true! At least, not by a "test". Tests show differences, not similarity. >All you may be able to say is that you don't have enough data to >disprove it. Your wording suggests you don't understand this important [quoted text clipped - 7 lines] >If you really must use a test, you could start here: > http://en.wikipedia.org/wiki/Kolmogorov-Smirnov_test "Insufficient evidence to prove a difference" is never sufficient evidence to prove no-difference. About the best you do to show no-difference is (say) to use confidence limits. A CI will show that, all other things being equal, the expected range of a difference is small. - Drug treatments to show "equivalence" use relatively large samples, like, in the hundreds for each group; and they focuse on a *single* paramater of difference, such as the mean morbidity rate. - If you don't even specify the nature of the difference, you have a tougher problem. The K-S test mentioned above has the mixed virtue of testing indirectly for variances as well as for means. It makes more sense to test for means, as most people are most interested in; or to test for outliers (suitably extreme scores) in one direction or the other (or, occasionally, both).  SignatureRich Ulrich The problem is that, in the hypothesis testing paradigm, you need to know the distribution of the test statistic in order to establish a p-value. It is easy (well, depending on the case...) to calculate what the distribution of many test statistics is for a null hypothesis that there is no difference between the populations. It is much harder to determine the sample statistic distribution for samples from populations that _do_ differ in some aspect under the null hypothesis. There is also a more fundamental issue: in which way do they differ, and by how much? This involves certain choices. I think that, in your situation, most would take the common pragmatic approach of using classical test procedures with a high alpha as a maximum criterion of "significance" or whatever you want to call it. That is, if they want to make the point that two samples are equal rather than different, they calculate p-values in usual ways and consider their claim supported if p>0.2 or 0.3 for example. Ideally, this is accompanied by a check of the value of the procedure in terms of power: what is the chance that you would actually detect a significant deviation from the null hypothesis that there is no difference _if_ populations actually do differ in a specific way by some meaningful amount? I suspect that the Bayesians among us will have another solution for you. Or you could go "visual", as Allen suggested. Whether you use a QQplot or something else will depend on the nature of your data (discrete/continuous?). |
Reply to this Message |
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2009 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.