Resolved: Hypothesis testing: P value or T/Z score

Hi Lucia!

Thanks for reaching out!

Indeed, hypothesis testing can be conducted both by comparing the P-value to the significance level or by referring to t/z scores from statistical tables. However, the two methods can give us different insights regarding what we observe. 

- The P-value approach is more direct in the context of hypothesis testing - if the P-value is less than alpha, we reject the null hypothesis. A p-value below alpha indicates statistical significance, meaning there's enough evidence to reject the null hypothesis, regardless of how much lower the p-value is than alpha. This method is very intuitive and widely applicable, especially when you need a quick decision on the null hypothesis. 

- The second method involves comparing the calculated |T/Z score| with t/z value from the table that corresponds to a certain alpha level. If the test statistic exceeds the critical value, we reject the null hypothesis. However, this method tells us not only if the effect is significant, but how significant it is relative to the variability in the data. 

Imagine you're testing if a new teaching method improves student scores. The P-value of 0.04 suggests the method is effective (since P < 0.05, we reject the null hypothesis stating that it's not). However, looking at the T score being very close to the critical value suggests that while the method is statistically significant, the actual difference in scores, or the effect size, might be minimal.

This dual approach provides a more detailed understanding of the obtained results. So, whether to use the first, the second, or both methods, depends on the specifics of your study, the nature of your data, and the details you wish to explore. P-values provide a quick check for statistical significance, while t/z scores offer deeper insights into effect sizes and are more informative when data distributions are well-understood. Generally speaking, P-values are used for broad significance testing, and t/z scores for gathering more detailed insights into your data.

Hope this helps.

Best,

Ivan