help tostregress
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Title

    tostregress -- Linear regression tests for equivalence


Syntax

        tostregress depvar [indepvars] [if] [in] [weight] [, eqvtype(typelist) eqvlevel(numlist) uppereqvlevel(numlist ) relevance level(#) regress_options]


    options                       Description
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Options
      eqvtype(typelist)           specify equivalence threshold with Delta or epsilon for each indepvar
      eqvlevel(numlist)           the level of tolerance defining the equivalence interval for each indepvar
      uppereqvlevel(numlist)      the upper value of an asymmetric equivalence interval for each indepvar
      level(#)                    set nominal type I level; default is level(95) coressponding to alpha=0.05
      relevance                   perform & report combined tests for difference and equivalence
      regress options             all the options allowed with regress
      svy(svy options)            all the options allowed with the svy prefix, including vcetype
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    indepvars may contain factor variables; see fvvarlist.
    depvar and indepvars may contain time-series operators; see tsvarlist.
    time-series operators may not be combined with factor variable operators.
    by, and statsby are allowed; see prefix.
    svy is allowed via the svy() option, but not as a prefix command.
    bootstrap and jackknife are allowed using vce(), or svy(), but are not allowed as prefix commands. See Remarks.
    Weights are allowed with neither vce(bootstrap), nor svy(bootstrap).
    aweights are allowed with neither vce(jackknife), nor svy(jackknife).
    hascons, tsscons, vce(), beta, noheader, notable, plus, depname(), mse1, and weights are not allowed with svy().
    aweights, fweights, iweights, and pweights are allowed; see weight.
    See [R] regress postestimation for features available after estimation.



Description

    tostregress tests for the equivalence of each regression coefficient and zero within separate symmetric equivalence intervals defined by eqvtype and eqvlevel for using a two one-sided t tests approach
    (Schuirmann, 1987).  Typically ('positivist') null hypotheses are framed from an assumption of a lack of difference between two quantities, and reject this assumption only with sufficient evidence.  When
    performing tests for equivalence, one frames a ('negativist') null hypothesis with the assumption that two quantities are different by at least as much as an equivalence interval defined by some chosen
    level of tolerance.

    An equivalence null hypothesis takes one of the following two forms depending on whether equivalence is defined in terms of Delta (equivalence expressed in the same units as the coefficient for an
    independent variable), or in terms of epsilon (equivalence expressed in the units of the T distribution with the given degrees of freedom):

        Ho: |_b_x| >= Delta,
        where the equivalence interval ranges from _b_x - Delta to _b_x + Delta, and where _b_x is the parameter being estimated for x.  This translates directly into two one-sided null hypotheses:

            Ho1: Delta - _b_x <= 0; and

            Ho2: _b_x + Delta <= 0

        -OR-

            Ho: |T| >= epsilon,

        where the equivalence interval ranges from -epsilon to epsilon.  This also translates directly into two one-sided null hypotheses:

            Ho1: epsilon - T <= 0; and

            Ho2: T + epsilon <= 0

        When an asymmetric equivalence interval is defined using the uppereqvlevel option the general negativist null hypothesis becomes:

            Ho: _b_x <= Delta_lower, or _b_x >= Delta_upper,

        where the equivalence interval ranges from _b_x + Delta_lower to _b_x + Delta_upper.  This also translates directly into two one-sided null hypotheses:

            Ho1: Delta_upper - _b_x <= 0; and

            Ho2: _b_x - Delta_lower <= 0

        -OR-

        Ho: T <= epsilon_lower, or T >= epsilon_upper,

            Ho1: epsilon_upper - T <= 0; and

            Ho2: T - epsilon_lower <= 0

        The two one-sided test statistics corresponding to Ho1 and Ho2, t1 and t2, are both constructed so that their p-values are upper (right) tail probabilities:

            p1 = P(T>=t1)

            p2 = P(T>=t2)
 
    NOTE: the appropriate level of alpha implied by level is precisely the same as in the corresponding two-sided test for mean difference, so that, for example, if one wishes to make a type I error %5 of the
    time, one simply conducts both of the one-sided tests of Ho1 and Ho2 by comparing the resulting p-value to 0.05 (Tryon and Lewis, 2008).


Options

        +------+
    ----+ Main +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    eqvtype(typelist) defines whether the equivalence interval will be defined in terms of Delta or epsilon (delta, or epsilon).  These options change the way that evqlevel is interpreted: when delta is
        specified, the evqlevel is measured in the units of the coefficient being tested, and when epsilon is specified, the evqlevel is measured in units of the T distribution; put another way epsilon =
        Delta/standard error.  The default is delta.

        eqvtype must be specified with no entries, one entry, or the same number of entries as parameters (coefficients) being estimated, including the intercept term, if any. The below examples are for a
        model with three parameter estimates -- one for weight, one for foreign, and one for _cons (the intercept term):

        The first example for eqvtype gives the default equivalence type (delta) either omitting the option (as shown here) or by leaving it empty):
            . tostregress mpg weight foreign, eqvlevel(5)

        This example for eqvtype gives a single equivalence type to be used for all three parameters:
            . tostregress mpg weight foreign, eqvtype(epsilon) eqvlevel(2.6448536)

        The last example for eqvtype gives a separate equivalence type for each parameter:
            . tostregress mpg weight foreign, eqvtype(delta delta epsilon) eqvlevel(5 5 2.6448536)

        Defining tolerance in terms of epsilon means that it is not possible to reject any test for mean equivalence Ho if epsilon <= the critical value of t for a given level and degrees of freedom.
        Because epsilon = Delta/standard error, we can see that for the same reason it is not possible to reject any Ho if Delta <= the product of the standard error and critical value of t for a given
        level and degrees of freedom.  tostregress reports when either of these conditions obtain.

    eqvlevel(numlist) defines the equivalence threshold for the tests depending on whether eqvtype is delta or epsilon (see above).  Researchers are responsible for choosing meaningful values of Delta or
        epsilon.  The default value for each coefficient is 1 when delta is the eqvtype for that coefficient, and 2 when epsilon is the eqvtype for that coefficient.

        eqvlevel must be specified with no entries, one entry, or the same number of entries as parameters (coefficients) being estimated, including the intercept term, if any.

    uppereqvlevel(#) defines the upper equivalence threshold for the test, and transforms the meaning of eqvlevel to mean the lower equivalence threshold for the test.  Also, eqvlevel is assumed to be a
        negative value.  Taken together, these correspond to Schuirmann's (1987) asymmetric equivalence intervals.  If uppereqvlevel==|eqvlevel|, then uppereqvlevel will be ignored.

        eqvtlevel must be specified with no entries, one entry, or the same number of entries as parameters (coefficients) being estimated, including the intercept term, if any. The below examples are for a
        model with three parameter estimates -- one for weight, one for foreign, and one for _cons (the intercept term):

    level(#) specifies the nominal type I error rate.  The default is level(95), which corresponds to alpha = 0.05.

    relevance reports results and inference for combined tests for difference and equivalence for specific level, eqvtype, and eqvlevel choices.  See the Remarks section more details on inference from
        combined tests.

    svy(svy options) estimates the model as if with the svy prefix, and any svy options you include. You may also specify vcetype options for svy here. See Remarks about jackknife and bootstrap options for
        svy.

Remarks

    As described by Tryon and Lewis (2008), when both tests for difference and equivalence are taken together, there are four possible interpretations:

    1.  One may reject the null hypothesis of no difference, but fail to reject the null hypothesis of difference, and conclude that there is a relevant difference between _b_x and zero as large as Delta or
        epsilon.

    2.  One may fail to reject the null hypothesis of no difference, but reject the null hypothesis of difference, and conclude that _b_x is equivalent to zero within the equivalence range (i.e. defined by
        Delta or epsilon).

    3.  One may reject both the null hypothesis of no difference and the null hypothesis of difference, and conclude that _b_x is trivially different, from zero within the equivalence range (i.e. defined by
        Delta or epsilon).

    4.  One may fail to reject both the null hypothesis of no difference and the null hypothesis of difference, and draw an indeterminate conclusion, because the data are underpowered to detect difference
        or equivalence for _b_x and zero.

    Caveat Emptor: jackknife and bootstrap options for svy estimation have been implemented only at a basic level.  If you run into problems with these options, especially if using suboptions for either
    estimator, please share your syntax and data with me so that I may improve tostregress.


Examples

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    These examples correspond to those written in the help file for regress:

    Example 1 setup
    . sysuse auto

    1a Report equivalence tests for a linear regression; equivalence interval is
    +/- 1 sd beyond the critical value of T for alpha = 0.05 and df = 71.
    epsilon = invt(71, .95)+1 = 2.6665997
    . tostregress mpg weight foreign, eqvtype(epsilon) eqvlevel(2.6665997)


    1b Report relevance test for a linear regression; equivalence interval is
    +/- 1 sd beyond the critical value of T for alpha = 0.05 and df = 71.
    epsilon = invt(71, .95)+1 = 2.6665997
    . tostregress mpg weight foreign, eqvtype(epsilon) eqvlevel(2.6665997) relevance


    Example 2 setup
    . sysuse auto

    2a Fit a better linear regression, from a physics standpoint, but add
    asymmetric intervals, and report relevance test results.  The lower end of
    the equivalence interval = invt(71, .95)+1.5 = 3.1665997 meaning equivalence
    must lie no more than 1.5 sd beyond the critical value of T for alpha = 0.05
    and df = 71.  The upper end of the equivalence interval = invt(71, .95)+1
    = 2.6665997 meaning equivalence must lie no more than 1 sd beyond the critical
    value of T for alpha = 0.05 and df = 71.
    . gen gp100m = 100/mpg
    . tostregress gp100m weight foreign, eqvtype(epsilon) eqvlevel(3.1665997) upper(2.6665997) rel


    2b Obtaining beta coefficients for the above example requires refitting the model
    with the beta option; equivalence interval is +/- 1 sd beyond the critical
    value of T for alpha = 0.05.
    . tostregress gp100m weight foreign, beta eqvtype(epsilon) eqvlevel(2.6665997) rel


    2c Report equivalence tests when suppressing the intercept term
    . tostregress weight length, noconstant eqvtype(delta) eqvlevel(5)


    2d Report equivalence tests when the model already has constant; express
    equivalence interval in units of the coefficient only for length (5 mpg/in.),
    and in units of the test statistic for each level of foreign. For the latter,
    the equivalence interval is +/- 1 sd beyond the critical value of T for
    alpha = 0.05.
    epsilon = invt(71, .95)+1 = 2.6665997
    . tostregress weight length bn.foreign, hascons eqvtype(delta epsilon epsilon) eqvlevel(5 2.6665997 2.6665997)
  

----------------------------------------------------------------------------
    These examples are for equivalence and relevance tests for regression with
    various types of robust standard errors:

    Example 3 setup
    . sysuse auto, clear
    . generate gpmw = ((1/mpg)/weight)*100*1000

    3a Example revelance test using OLS regression with equivalence threshold
    expressed in units of test statistic for foreign.  The equivalence interval
    is +/- 1 sd beyond the critical value of T for alpha = 0.05.
    epsilon = invt(72, .95)+1 = 2.6662937
    . tostregress gpmw foreign, eqvtype(epsilon) eqvlevel(2.6662937) rel


    3b Above example using regression with different robust standard error estimators:
    . tostregress gpmw foreign, vce(robust) eqvtype(epsilon) eqvlevel(2.6662937) rel
    . tostregress gpmw foreign, vce(hc2) eqvtype(epsilon) eqvlevel(2.6662937) rel
    . tostregress gpmw foreign, vce(hc3) eqvtype(epsilon) eqvlevel(2.6662937) rel


    Example 4 setup
    . webuse regsmpl, clear

    Example revelance test using regression and adjusting standard errors for
    clustering by id.  The equivalence interval is +/- 1 sd beyond the
    critical value of T for alpha = 0.05.
    epsilon = invt(4698, .95)+1 = 2.645178
    . tostregress ln_wage age c.age#c.age tenure, vce(cluster id) eqvt(epsilon) eqvl(2.645178)


    ----------------------------------------------------------------------------
    This example is for equivalence tests for weighted regression:

    Example 5 setup
    . sysuse census

    Regression with equivalence tests using analytic weights.  The equivalence
    interval is +/- 1 sd beyond the critical value of T for alpha = 0.05.
    epsilon = invt(45, .9)+1 = 2.3006493
    . tostregress death medage i.region [aw=pop], eqvtype(epsilon) eqvlevel(2.3006493) level(90)


    ----------------------------------------------------------------------------
    These examples are for equivalence tests for linear regression with survey
    data

    Example 6 setup
    . webuse highschool

    6a Perform linear regression using survey data
    . svy: regress weight height


    6b Perform corresponding linear regression tests for equivalence using survey
    data.  The equivalence interval is +/- 1 sd beyond the critical value of T for
    alpha = 0.05.
    epsilon = invt(50, .95)+1 = 2.675905
    . tostregress weight height, eqvt(epsilon) eqvl(2.675905) rel svy()


    Example 7 setup
    . generate male = sex == 1 if !missing(sex)

    Perform linear regression using survey data for a subpopulation
    . svy, subpop(male): regress weight height


    7a Perform corresponding linear regression tests for equivalence using survey
    data for a subpopulation.  The equivalence interval is +/- 1 sd beyond the
    critical value of T (with degrees of freedom = N - the 50 stage 1 sampling
    units, or 4071-50 = 4021) for alpha = 0.05.
    epsilon = invt(4021, .95)+1 = 2.6452327
    . tostregress weight height, eqvt(epsilon) eqvl(2.6452327) rel svy(subpop(male))


    7b Perform the above survey data estimate using the jackknife variance
    estimator using the same equivalence threshold as the above example.
    . tostregress weight height, eqvt(epsilon) eqvl(2.6452327) rel svy(jackknife subpop(male))
    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Saved results

    In addition to the information saved by regress, tostregress saves the following in e():

    Scalars   
      e(alpha)            The nominal Type I error rate based on the level option (multiple comparisons issues will arise in a multiple regression context).

    Macros    
      e(cmd)              tostregress
      e(cmdline)          canonical form of command as typed
      e(title)            title of estimation output reflecting the svy option
      e(eqvtype)          the eqvtype for each coefficient/test
      e(lowereqvlevel)    the lower eqvlevel for each coefficient/test
      e(uppereqvlevel)    the uppereqvlevel for each coefficient/test
      e(rel_conclusions)  relevance test conclusions if the relevance option is used

    Matrices  
      e(T1)               t1 test statistics
      e(T2)               t2 test statistics
      e(P1)               p-values corresponding to the test statistics in T1
      e(P2)               p-values corresponding to the test statistics in T2


Author

    Alexis Dinno
    Portland State University
    alexis.dinno@pdx.edu

    Development of tost is ongoing, please contact me with any questions, bug reports or suggestions for improvement. Fixing bugs will be facilitated by sending along:

        (1) a copy of the data (de-labeled or anonymized is fine),
        (2) a copy of the command used, and
        (3) a copy of the exact output of the command.


Suggested citation

    Dinno A. 2025. tostregress: Linear regression tests for equivalence.  In: tost Stata software package.  URL: https://www.alexisdinno.com/stata/tost.html


References

    Schuirmann, D. A.  1987.  A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability.  Journal of Pharmacokinetics and
        Biopharmaceutics.  15: 657-680

    Tryon, W. W., and Lewis, C.  2008.  An inferential confidence interval method of establishing statistical equivalence that corrects Tryon's (2001) reduction factor.  Psychological Methods.  13: 272-277


Also See

      Help: tost, pkequiv, regress
