{smcl}
{* *! version 1.3.5  03may2018}{...}
{cmd:help dunntest}
{hline}


{title:Title}

{p2colset 5 17 20 2}{...}
{p2col:{cmd:dunntest} {hline 2}}Dunn's test of multiple comparisons using rank sums{p_end}
{p2colreset}{...}


{marker syntax}{...}
{title:Syntax}

{p 8 16 2}
{cmd:dunntest} {varname} {ifin} {cmd:,} {opth "by(varlist:groupvar)"} 
           [{opt ma(method)} {opt nokwallis} {opt nolabel} {opt wrap} {opt li:st} {opt rmc} {opt l:evel(#)} {opt altp}]


{synoptset 28 tabbed}{...}
{synopthdr:dunntest options}
{synoptline}
{syntab:Main}
{synopt :{opth by:(varlist:groupvar)}}variable defining the {it:k} groups.  
Missing observations in {it:{help varlist:groupvar}} are ignored.{p_end}
{synopt :{opt m:a(method)}}which method to adjust for multiple comparisons{p_end}
{synopt :{opt nokwallis}}suppress Kruskal-Wallis test output{p_end}
{synopt :{opt nolabel}}display data values, rather than data value labels{p_end}
{synopt :{opt wrap}}do not break wide tables{p_end}
{synopt :{opt li:st}}include results of Dunn's test in a list format.{p_end}
{synopt :{opt rmc}}report row - col rather than col - row{p_end}
{synopt :{opt l:evel(#)}}set confidence level; default is {opt level(95)}{p_end}
{synopt :{opt altp}}use alternative expression of {it:p}-values{p_end}
{synoptline}
{p2colreset}{...}

{p 4 6 2}
Missing observations in {varname} are ignored.


{marker description}{...}
{title:Description}

{pstd}
{cmd:dunntest} reports the results of Dunn's test ({help dunntest##Dunn1964:1964}) for 
stochastic dominance among multiple pairwise comparisons following a 
Kruskal-Wallis test of stochastic dominance among {it:k} groups 
({help dunntest##Kruskal1952:Kruskal and Wallis, 1952}) using {helpb kwallis}.  
The interpretation of stochastic dominance requires an assumption that the CDF 
of one group does not cross the CDF of the other.  
{cmd: dunntest} performs {it:m} = {it:k}({it:k}-1)/2 multiple pairwise comparisons.  
The null hypothesis in each pairwise comparison is that the probability of 
observing a random value in the first group that is larger than a random value 
in the second group equals one half; this null hypothesis corresponds to that of 
the Wilcoxon-Mann-Whitney {helpb ranksum:rank-sum} test.  Like the rank-sum test, 
if the data can be assumed to be continuous, and the distributions are assumed 
identical except for a shift in centrality, Dunn's test may be understood as a 
test for median difference.  In the syntax diagram above, {varname} refers to 
the variable recording the outcome, and {it:{help varlist:groupvar}} refers to 
the variable denoting the population.  {cmd:dunntest} accounts for tied ranks.  
{opt by()} is required.{p_end}

{pstd}
{cmd:dunntest} outputs both {it:z} test statistics for each pairwise comparison
(corresponding to the column mean minus the row mean, unless the {cmd:rmc}
option is used) and the {it:p}-value = P({it:Z}>=|{it:z}|) for each, where 
{it:z} is the difference of mean ranks divided by the population standard 
error of that difference.  Reject Ho based on {it:p} <= alpha/2 (and in combination with 
{it:p}-value ordering for stepwise {opt ma} options).  If you prefer to work with {it:p}-values 
expressed as p = P(|{it:Z}| >=|{it:z}|) use the {opt altp} option, and reject Ho 
based on {it:p} <= alpha (and in combination with {it:p}-value ordering for stepwise 
{opt ma} options).  These are exactly equivalent rejection decisions).{p_end}


{marker option}{...}
{title:Options}

{phang}{opth "by(varlist:groupvar)"} is required.  It specifies a variable
that identifies the groups.{p_end}

{phang}{opt ma(method)} is required.  It specifies the method of adjustment used 
for multiple comparisons, and must take one of the following values: {opt none}, 
{opt bonferroni}, {opt sidak}, {opt hochberg}, {opt hs}, {opt bh}, or {opt by}.  
{opt none} is the default method assumed if the {opt ma} option is omitted.  
These methods perform as follows:{p_end}

{p 8 8}{opt none} specifies no adjustment for multiple comparisons be made.  Those 
comparisons rejected without adjustment at the alpha level (two-sided test) are 
underlined in the output table, and starred in the list when using the {opt li:st} 
option.{p_end}

{p 8 8}{opt bonferroni} specifies a "Bonferroni adjustment" where the 
{browse "https://en.wikipedia.org/wiki/Family-wise_error_rate":family-wise error rate} (FWER) is adjusted by multiplying the {it:p}-values in 
each pairwise test by {it:m} (the total number of pairwise tests) as per Dunn 
({help dunntest##Dunn1961:1961}).  {cmd:dunntest} will report a maximum 
Bonferroni-adjusted {it:p}-value of 1.  Those comparisons rejected with this 
method at the alpha level (two-sided test) are underlined in the output table, 
and starred in the list when using the {opt li:st} option.{p_end}

{p 8 8}{opt sidak} specifies a "Sid{c a'}k adjustment" where the FWER is adjusted 
by	 replacing the {it:p}-value of each pairwise test with 1 - (1 - {it:p})^{it:m} 
as per Sid{c a'}k ({help dunntest##Sidak:1967}).  {cmd:dunntest} will report a maximum 
Sid{c a'}k-adjusted {it:p}-value of 1.  Those comparisons rejected with this 
method at the alpha level (two-sided test) are underlined in the output table, 
and starred in the list when using the {opt li:st} option.{p_end}

{p 8 8}{opt holm} specifies a "Holm adjustment" where the FWER is adjusted 
sequentially by adjusting the {it:p}-values of each pairwise test as 
ordered from smallest to largest with {it:p}({it:m}+1-{it:i}), where {it:i} is the 
position in the ordering as per Holm ({help dunntest##Holm1979:1979}).  {cmd:dunntest} will 
report a maximum Holm-adjusted {it:p}-value of 1.  Because in sequential tests 
the decision to reject the null hypothesis depends both on the {it:p}-values and 
their ordering, those comparisons rejected with this method at the alpha level 
(two-sided test) are underlined in the output table, and starred in the list 
when using the {opt li:st} option.{p_end}

{p 8 8}{opt hs} specifies a "Holm-Sid{c a'}k adjustment" where the FWER is 
adjusted sequentially by adjusting the {it:p}-values of each pairwise test as 
ordered from smallest to largest with 1 - (1 - {it:p})^({it:m}+1-{it:i}), where 
{it:i} is the position in the ordering (see {help dunntest##Holm1979:Holm, 1979}).  
{cmd:dunntest} will report a maximum Holm-Sid{c a'}k-adjusted {it:p}-value of 1.  Because 
in sequential tests the decision to reject the null hypothesis depends both on 
the {it:p}-values and their ordering, those comparisons rejected with this 
method at the alpha level (two-sided test) are underlined in the output table, 
and starred in the list when using the {opt li:st} option.{p_end}

{p 8 8}{opt hochberg} specifies a "Hochberg adjustment" where the FWER is adjusted 
sequentially by adjusting the {it:p}-values of each pairwise test as 
ordered from largest to smallest with {it:p}*{it:i}, where {it:i} is the 
position in the ordering as per Hochberg ({help dunntest##Hochberg1988:1988}).  
{cmd:dunntest} will report a maximum Hochberg-adjusted {it:p}-value of 1.  Because in 
sequential tests the decision to reject the null hypothesis depends both on the 
{it:p}-values and their ordering, those comparisons rejected with this method at 
the alpha level (two-sided test) are underlined in the output table, and starred 
in the list when using the {opt li:st} option.{p_end}

{p 8 8}{opt bh} specifies a "Benjamini-Hochberg adjustment" where the {browse "https://en.wikipedia.org/wiki/False_discovery_rate":false discovery rate} 
(FDR) is adjusted sequentially by adjusting the {it:p}-values of 
each pairwise test as ordered from largest to smallest with {it:p}[{it:m}/({it:m}+1-{it:i})], 
where {it:i} is the position in the ordering (see {help dunntest##Benjamini1995:Benjamini & Hochberg, 1995}).  
{cmd:dunntest} will report a maximum Benjamini-Hochberg-adjusted {it:p}-value of 1.  Such 
FDR-adjusted {it:p}-values are sometimes refered to as {it:q}-values in the 
literature.  Because in sequential tests the decision to reject the null 
hypothesis depends both on the {it:p}-values and their ordering, those 
comparisons rejected with this method at the alpha level (two-sided test) are 
underlined in the output table, and starred in the list when using the {opt li:st} 
option.{p_end}

{p 8 8}{opt by} specifies a "Benjamini-Yekutieli adjustment" where the false 
discovery rate (FDR) is adjusted sequentially by adjusting the {it:p}-values of 
each pairwise test as ordered from largest to smallest with {it:p}[{it:m}/({it:m}+1-{it:i})]{it:C}, 
where {it:i} is the position in the ordering, and {it:C} = 1 + 1/2 + ... + 1/{it:m}
 (see {help dunntest##Benjamini2001:Benjamini & Yekutieli, 2001}).  {cmd:dunntest} will 
 report a maximum Benjamini-Yekutieli-adjusted {it:p}-value of 1.  Such 
FDR-adjusted {it:p}-values are sometimes refered to as {it:q}-values in the 
literature.  Because in sequential tests the decision to reject the null 
hypothesis depends both on the {it:p}-values and their ordering, those 
comparisons rejected with this method at the alpha level (two-sided test) are 
underlined in the output table, and starred in the list when using the {opt li:st} 
option.{p_end}

{phang}{opt nokwallis} suppresses the display of the Kruskal-Wallis test table.{p_end}

{phang}{opt nolabel} causes the actual data codes to be displayed rather than the
value labels in the Dunn's test tables.{p_end}

{phang}{opt wrap} requests that {cmd:dunntest} not break up wide tables to make
them readable.{p_end}

{phang}{opt list} requests that {cmd:dunntest} also provide a output in list form, 
one pairwise test per line.{p_end}

{phang}{opt rmc} requests that {cmd:dunntest} reports {it:z} statistic based on the mean rank 
of the row variable minus the mean rank of the column variable. The default is 
to report the mean rank of the column variable minus the mean rank of the row 
variable. The difference between these two is simply the sign of the {it:z} 
statistic.{p_end}

{phang}{opt level(#)} specifies the compliment of alpha*100.  The default, 
{opt level(95)} (or as set by {helpb set level}) corresponds to alpha = 0.05.

{phang}{opt altp} directs {cmd:dunntest} to express {it:p}-values in alternative 
format.  The default is to express p = P(Z >= |z|), and reject Ho if p >= alpha\2.  When 
the {opt altp} option is used, {it:p}-values are instead expressed as p = P(|Z| >= |z|), 
and reject Ho if p <= alpha.  {ul on}These two expressions give identical 
test results.{ul off} Use of {opt altp} is therefore merely a semantic choice.
{p_end}


{marker example}{...}
{title:Example}

{pstd}Setup{p_end}
{phang2}{cmd:. webuse census}{p_end}

{pstd}Test for equal median age by region type simultaneously{p_end}
{phang2}{cmd:. kwallis medage, by(region)}	{p_end}

{phang}Dunn's multiple-comparison test for stochastic dominance using a 
Bonferroni correction{p_end}
{phang2}{cmd:. dunntest medage, by(region) ma(bonferroni) nokwallis}


{marker saved_results}{...}
{title:Saved results}

{pstd}
{cmd:dunntest} saves the following in {cmd:r()}:

{synoptset 15 tabbed}{...}
{p2col 5 15 19 2: Scalars}{p_end}
{synopt:{cmd:r(df)}}degrees of freedom for the Kruskal-Wallis test{p_end}
{synopt:{cmd:r(chi2_adj)}}chi-squared adjusted for ties for the Kruskal-Wallis test{p_end}

{p2col 5 15 19 2: Matrices}{p_end}
{synopt:{cmd:r(Z)}}vector of Dunn's z test statistics{p_end}
{synopt:{cmd:r(P)}}vector of adjusted {it:p}-values for Dunn's z test statistics {cmd:--OR--}{p_end}
{synopt:{cmd:r(altP)}}vector of adjusted {it:p}-values for Dunn's z test statistics when using the {opt altp} option{p_end}
{p2colreset}{...}


{title:Author}

{pstd}Alexis Dinno{p_end}
{pstd}Portland State University{p_end}
{pstd}alexis.dinno@pdx.edu{p_end}

{pstd}
Please contact me with any questions, bug reports or suggestions for 
improvement.  Fixing bugs will be facilitated by sending along:{p_end}

{p 8 8 4}(1) a copy of the data (de-labeled or anonymized is fine),{p_end}
{p 8 8 4}(2) a copy of the command used, and{p_end}
{p 8 8 4}(3) a copy of the exact output of the command.{p_end}


{title:Suggested citation}

{p 4 8}
Dinno A. 2014. {bf:dunntest}: Dunn's test of multiple comparisons using rank 
sums. Stata software package. URL: {view "https://alexisdinno.com/stata/dunntest.html"}{p_end}


{title:References}

{marker Benjamini1995}{...}
{phang}Benjamini, Y. and Hochberg, Y. 1995. {browse "https://www.jstor.org/stable/2346101":Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing}.  
{it:Journal of the Royal Statistical Society. Series B (Methodological)}. 57: 289-300.{p_end}

{marker Benjamini2001}{...}
{phang}Benjamini, Y. and Yekutieli, D. 2001. {browse "https://www.jstor.org/stable/2674075":The control of the false discovery rate in multiple testing under dependency}.  
{it:Annals of Statistics}. 29: 1165-1188.{p_end}

{marker Dunn1961}{...}
{phang}Dunn, O. J. 1961. {browse "https://sci-hub.io":Multiple comparisons among means}.  
{it:Journal of the American Statistical Association}.  56: 52-64.{p_end}

{marker Dunn1964}{...}
{phang}Dunn, O. J. 1964. {browse "https://sci-hub.io":Multiple comparisons using rank sums}.  {it:Technometrics}. 
 6: 241-252.{p_end}

{marker Hochberg1988}{...}
{phang}Hochberg, Y. 1988. {browse "https://sci-hub.io":A sharper Bonferroni procedure for multiple tests of significance}.  {it:Biometrika}. 75: 800-802.{p_end}

{marker Holm1979}{...}
{phang}Holm, S. 1979. {browse "https://www.jstor.org/stable/4615733":A simple sequentially rejective multiple test procedure}.  
{it:Scandinavian Journal of Statistics}.  6: 65-70.{p_end}

{marker Kruskal1952}{...}
{phang}Kruskal, W. H. and Wallis, A. 1952. {browse "https://sci-hub.io":Use of ranks in one-criterion variance analysis}.  
{it:Journal of the American Statistical Association}.  47: 583-621.{p_end}

{marker Sidak1967}{...}
{phang}Sid{c a'}k, Z. 1967. {browse "https://sci-hub.io":Rectangular confidence regions for the means of multivariate normal distributions}.  
{it:Journal of the American Statistical Association}.  62: 626-633.{p_end}


{title:Also See}

{psee}
{space 2}Help: {help kwallis:kwallis}, {help ranksum:ranksum}, {net "describe https://alexisdinno.com/stata/conovertest.pkg":conovertest}{p_end}
