Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis (usually a null hypothesis of no effect).

Statistical hypothesis testing assesses whether data are inconsistent with a given hypothesis (usually a null hypothesis of no effect). In practice, this is often associated with the Null Hypothesis Statistical Testing framework, based on ideas from Fisher as well as Neyman & Pearson. However, other approaches exist, such as Bayesian hypothesis testing.

Note that testing a statistical hypothesis is only one component of testing a research hypothesis. Careful experimental or study design is needed for the statistical hypothesis to lead to insight regarding the research hypothesis.

11168 questions
390
votes
15 answers

Is normality testing 'essentially useless'?

A former colleague once argued to me as follows: We usually apply normality tests to the results of processes that, under the null, generate random variables that are only asymptotically or nearly normal (with the 'asymptotically' part dependent on…
shabbychef
  • 15,344
309
votes
16 answers

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It seems that students easily learn how to perform the…
Sharpie
  • 4,544
  • 6
  • 26
  • 18
188
votes
15 answers

Are large data sets inappropriate for hypothesis testing?

In a recent article of Amstat News , the authors (Mark van der Laan and Sherri Rose) stated that "We know that for large enough sample sizes, every study—including ones in which the null hypothesis of no effect is true — will declare a statistically…
Carlos Accioly
  • 5,183
147
votes
10 answers

How to choose between t-test or non-parametric test e.g. Wilcoxon in small samples

Certain hypotheses can be tested using Student's t -test (maybe using Welch's correction for unequal variances in the two-sample case), or by a non-parametric test like the Wilcoxon paired signed rank test, the Wilcoxon-Mann-Whitney U test, or the…
Silverfish
  • 24,613
141
votes
9 answers

Is Facebook coming to an end?

Recently, this paper has received a lot of attention (e.g. from WSJ ). Basically, the authors conclude that Facebook will lose 80% of its members by 2017. They base their claims on an extrapolation of the SIR model , a compartmental model frequently…
LessFaceMoreBook
  • 1,023
121
votes
7 answers

T-test for non normal when N>50?

Long ago I learnt that normal distribution was necessary to use a two sample T-test. Today a colleague told me that she learnt that for N>50 normal distribution was not necessary. Is that true? If true is that because of the central limit theorem?
even
  • 2,417
  • 6
  • 20
  • 13
119
votes
10 answers

ASA discusses limitations of $p$-values - what are the alternatives?

We already have multiple threads tagged as p-values that reveal lots of misunderstandings about them. Ten months ago we had a thread about psychological journal that "banned" $p$-values , now American Statistical Association (2016) says that with…
Tim
  • 146,039
112
votes
3 answers

What are examples where a "naive bootstrap" fails?

Suppose I have a set of sample data from an unknown or complex distribution, and I want to perform some inference on a statistic $T$ of the data. My default inclination is to just generate a bunch of bootstrap samples with replacement, and calculate…
raegtin
  • 10,290
107
votes
9 answers

Is this really how p-values work? Can a million research papers per year be based on pure randomness?

I'm very new to statistics, and I'm just learning to understand the basics, including $p$-values. But there is a huge question mark in my mind right now, and I kind of hope my understanding is wrong. Here's my thought process: Aren't all researches…
n_mu_sigma
  • 1,131
104
votes
5 answers

When to use Fisher versus Neyman-Pearson framework?

I've been reading a lot lately about the differences between Fisher's method of hypothesis testing and the Neyman-Pearson school of thought. My question is, ignoring philosophical objections, when should we use Fisher's approach to data testing and…
Stijn
  • 2,022
102
votes
11 answers

How to obtain the p-value (check significance) of an effect in a lme4 mixed model?

I use lme4 in R to fit the mixed model lmer(value~status+(1|experiment))) where value is continuous, status and experiment are factors, and I get Linear mixed model fit by REML Formula: value ~ status + (1 | experiment) AIC BIC logLik deviance…
ECII
  • 2,221
101
votes
2 answers

How much do we know about p-hacking "in the wild"?

The phrase p -hacking (also: "data dredging" , "snooping" or "fishing") refers to various kinds of statistical malpractice in which results become artificially statistically significant. There are many ways to procure a "more significant" result,…
Silverfish
  • 24,613
93
votes
10 answers

Why is it possible to get significant F statistic (p<.001) but non-significant regressor t-tests?

In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests? In my model, there are 10 regressors. One has a p-value of 0.1 and the rest are above…
Ηλίας
  • 1,589
88
votes
10 answers

Regarding p-values, why 1% and 5%? Why not 6% or 10%?

Regarding p-value s, I am wondering why $1$ % and $5$ % seem to be the gold standard for "statistical significance" . Why not other values, like $6$ % or $10$ %? Is there a fundamental mathematical reason for this, or is this just a widely held…
Contango
  • 1,519
  • 1
  • 17
  • 16
80
votes
3 answers

How to interpret type I, type II, and type III ANOVA and MANOVA?

My primary question is how to interpret the output (coefficients, F, P) when conducting a Type I (sequential) ANOVA? My specific research problem is a bit more complex, so I will break my example into parts. First, if I am interested in the effect…
djhocking
  • 2,091
1
2 3
…
99 100