Most Popular
1500 questions
303
votes
8 answers
Bagging, boosting and stacking in machine learning
What's the similarities and differences between these 3 methods: Bagging, Boosting, Stacking? Which is the best one? And why? Can you give me an example for each?
Bucsa Lucian
- 3,139
- 3
- 17
- 3
298
votes
13 answers
Is there any reason to prefer the AIC or BIC over the other?
The AIC and BIC are both methods of assessing model fit penalized for the number of estimated parameters. As I understand it, BIC penalizes models more for free parameters than does AIC. Beyond a preference based on the stringency of the criteria,…
russellpierce
- 19,480
298
votes
11 answers
How would you explain covariance to someone who understands only the mean?
...assuming that I'm able to augment their knowledge about variance in an intuitive fashion ( Understanding "variance" intuitively ) or by saying: It's the average distance of the data values from the 'mean' - and since variance is in square units,…
PhD
- 15,115
295
votes
7 answers
What does AUC stand for and what is it?
Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
josh
- 3,309
294
votes
6 answers
Is $R^2$ useful or dangerous?
I was skimming through some lecture notes by Cosma Shalizi (in particular, section 2.1.1 of the second lecture ), and was reminded that you can get very low $R^2$ even when you have a completely linear model. To paraphrase Shalizi's example: suppose…
raegtin
- 10,290
292
votes
158 answers
Famous statistical quotations
What is your favorite statistical quote? This is community wiki, so please one quote per answer.
robin girard
- 6,885
285
votes
2 answers
Interpretation of R's lm() output
The help pages in R assume I know what those numbers mean, but I don't. I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. There might (will) be mistakes, as I'll just write…
Alexander Engelhardt
- 4,361
280
votes
13 answers
How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?
Maybe the concept, why it's used, and an example.
Neil McGuigan
- 10,232
- 14
- 59
- 65
273
votes
7 answers
ROC vs precision-and-recall curves
I understand the formal differences between them, what I want to know is when it is more relevant to use one vs. the other. Do they always provide complementary insight about the performance of a given classification/detection system? When is it…
Amelio Vazquez-Reina
- 19,986
273
votes
15 answers
What are the differences between Factor Analysis and Principal Component Analysis?
It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use one over the other. A real example would be…
Brandon Bertelsen
- 7,552
260
votes
47 answers
What are common statistical sins?
I'm a grad student in psychology, and as I pursue more and more independent studies in statistics, I am increasingly amazed by the inadequacy of my formal training. Both personal and second hand experience suggests that the paucity of statistical…
Mike Lawrence
- 14,193
260
votes
10 answers
Why is Newton's method not widely used in machine learning?
This is something that has been bugging me for a while, and I couldn't find any satisfactory answers online, so here goes: After reviewing a set of lectures on convex optimization, Newton's method seems to be a far superior algorithm than gradient…
Fei Yang
- 2,591
259
votes
19 answers
Intuitive explanation for dividing by $n-1$ when calculating standard deviation?
I was asked today in class why you divide the sum of square error by $n-1$ instead of with $n$, when calculating the standard deviation. I said I am not going to answer it in class (since I didn't wanna go into unbiased estimators), but later I…
Tal Galili
- 22,224
259
votes
8 answers
Algorithms for automatic model selection
I would like to implement an algorithm for automatic model selection. I am thinking of doing stepwise regression but anything will do (it has to be based on linear regressions though). My problem is that I am unable to find a methodology, or an open…
S4M
- 2,752
258
votes
12 answers
Why is accuracy not the best measure for assessing classification models?
This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. Accuracy , the proportion of correct classifications…
Tim
- 146,039