UPENN | ML | DA Practice

Question

Consider the statements:

$P1:$ It is generally more important to use consistent estimators when one has smaller numbers of training examples.

$P2:$ It is generally more important to used unbiased estimators when one has smaller numbers of training examples.

Which of the following statement(s) is/are correct?

(A) Only $P1$ is true

(B) Both $P1$ and $P2$ are true

(C) Only $P2$ is True

(D) Both $P1$ and $P2$ are False

Answer 1

P1: 

False.

• Consistency: A consistent estimator is one that converges to the true value of the parameter it's estimating as the sample size increases. In other words, it becomes more accurate as you have more data.
• Smaller Sample Sizes: With smaller sample sizes, the issue isn't so much about consistency (which is an asymptotic property) but rather about bias and variance.
• Bias: In smaller samples, it's often more important to prioritize unbiased estimators. Unbiased estimators have an expected value that equals the true value of the parameter, even in smaller samples. This ensures that the estimator is not systematically over- or underestimating the parameter.
• Variance: While consistency is desirable, it doesn't guarantee good performance with small samples. A consistent estimator might still have high variance in smaller samples, leading to less reliable estimates.

P2: False. The importance of using unbiased estimators is not necessarily higher with smaller numbers of training examples.

500. both are False