Resolved: Rejecting Null Hypothesis Issue in Customer Engagement Project Task 4.
Hello, I hope you're all doing well.
In project problem task 4, there's an issue highlighted with red circles. We have two decision rules to either accept or reject the null hypothesis:
1) If the T-score is less than or equal to the negative critical t-value, we reject the null hypothesis. Otherwise, we accept it.
2) If the p-value is less than or equal to 0.05, we reject the null hypothesis. Otherwise, we accept it.
For students on the paid plan, this verification is working correctly. However, for students on the free plan, there's a discrepancy. When applying the first decision rule, it suggests accepting the null hypothesis. However, considering the second decision rule, the p-value is nearly 0.000, indicating that we should reject the null hypothesis.
What should we do in such case. I'm attaching picture for reference.
Dear Muhammad,
Thank you for reaching out with your concerns.
I understand the confusion surrounding the T-score interpretation for the free-plan students you mentioned (as shown in the left-side picture). The T-score given is 29.78. As you rightly pointed out in the decision rule, a left-tailed test should be conducted. This means the T-score should be compared to the negative critical value. However, it appears that Excel is performing a right-tailed test, comparing the T-score with the positive critical value. This discrepancy explains why the p-value appears so low.
When interpreting these results, it's crucial to exercise discretion. It's worth noting that with experience, such nuances become more apparent, and you'll be better equipped to spot them.
I hope this clarification assists you in your understanding.
Warm regards,
The 365 Team
These pictures are of the solutions as provided when we submit the project.
I didn't get it. As I mentioned before.
Even if we use online calculator as mentioned in the lectures to find p value, we get .00001 (p-value) it is still less than 0.05 so we reject the null by rule 2. But rule 1 is saying failed to reject null. So what should we do in such cases.
We have two decision rules to either accept or reject the null hypothesis:
1) If the T-score is less than or equal to the negative critical t-value, we reject the null hypothesis. Otherwise, we accept it.
2) If the p-value is less than or equal to 0.05, we reject the null hypothesis.
Hello Muhammad!
Thanks for reaching out!
You correctly outline the two rules:
1) If the T-score is less than or equal to the negative critical t-value, we reject the null hypothesis. Otherwise, we accept it.
2) If the p-value is less than or equal to 0.05 (or any other significance level), we reject the null hypothesis.
In general, the two rules should coincide. Put simply, there should not be any difference for the outcome.
Let me alaborate further on the case that you mentioned.
Your null hypotheses should include the following:
· The engagement (minutes watched) in Q4 2021 is higher than or equal to the one in Q4 2022 (μ1 ≥ μ2). We test free-plan and paying students separately.
Put simply the alternative hypothesis should be that the average minutes watched in Q4 2021 should be lower than the average minutes watched in Q4 2022.
What this means is that you should perform left-tailed one sided test. Therefore, you should compare the t-statistic to negative critical value. If it's less than that, we reject the null hypothesis. For reference use this calculator to perform the test.
As you can see the p value is above 0.05, so you cannot reject the hypothesis.
In the coming days we will also launch our version of the calculator that you can use but the alternative calculator should do the purpose for now.
Hope this helps!
Best,
The 365 Team
Hello Muhammad!
Here is some more information on the difference between two-sided and one sided tests that you can find useful:
Тhere are two types of hypotheses: two-sided and one-sided hypotheses.
A two-sided hypothesis, also known as a two-tailed, determines if there is a significant difference between the means of two groups without specifying the direction. The null hypothesis assumes that the means are equal, while the alternative hypothesis states that they’re not.
On the other hand, we use a one-sided (or one-tailed) hypothesis to calculate whether one mean is greater than or less than the other. In fact, the null hypothesis states that one mean is less than or equal to the other mean, while the alternative hypothesis states that one mean is greater than the other mean.
Please also note that we have developed multiple calculators that you can you as part of your studies on statistics. You can find them here.
Hope this helps.
Best,
The 365 Team
Hi Ivan,
Thank you for your feedback. Your message helped me understand the issue better.
It's clear that there are some important details missing in the Excel solution, which led to the problem being raised in the first place. Additionally, the discrepancy in the p-value calculations between the suggested calculator in the lectures and the Excel solution is something worth noting. I assume the suggested calculator was performing a right-tailed test as well, which explains the difference in the p-values.
Suggested Calculator Link: https://www.socscistatistics.com/pvalues/
Please suggest another calculator that performs proper calculations. Like you used to perform the calculations.
Lastly, I suggest to include a p-value calculator for the t-score value as well in 365data science calculator menu, in addition to the existing one for the z-score. This enhancement would make the tool even more useful.
Thank you again for your help.
Hello Muhammad!
Thanks for the suggestions!
Please check the p-value calculator that we have developed. It includes all the four distributions that you will ever need.
You have to select from the dropdown the distrution and then type in the score. I suggets that you try it. I would be grateful for your feedback on the calculator as well!
Best,
The 365 Team
Hi Ivan,
I just completed the project and had the similar issue for Task 4 as broached upon in this topic. Even after reading through the questions and replies, I still have some confusion regarding the solution for Free students:
1st Question: The video lessons in the "Statistics" Course do not mention anything about left-tailed or right-tailed tests. Do you have any other lesson in 365 which would elaborate on these topics?
2nd Question: All video lessons on Hypothesis Testing indicate the rule to compare Absolute value of Z/T with the positive critical z/t and reject H0 if Abs (Z/T) > positive critical (z/t). However, the project solution shows a different aspect (comparing a positive T with a negative critical t). Can you provide some authentic links which would help me understand this aspect as well?
3rd Question: Your link to the Calculators developed by 365 as mentioned in one of your replies doesn't work (shows a "Page Not Found" error). Can you please provide this link?
Regards,
Dhaivat Parekh
Hello Dhaivat,
Thanks for reaching out!
Regarding you questions:
Q1- The project goes one step deaper and explores the difference between the tests. The project so to say builds on the knowlege already covered in the course.
Q2-Great Question! To understand the aspect of comparing a positive T with a negative critical t in hypothesis testing, it's important to consider the nature of one-tailed tests. Typically in hypothesis testing, whether you compare the test statistic (like a T-value) to a positive or negative critical value depends on the direction of the alternative hypothesis you are testing.
In a one-tailed test, the critical value can be either positive or negative based on the hypothesis:
Right-tailed test: If your alternative hypothesis suggests that the mean is greater than a certain value, you would use a positive critical value. You reject the null hypothesis if your T-value is greater than this positive critical value.
Left-tailed test: Conversely, if the alternative hypothesis suggests that the mean is less than a certain value, you would use a negative critical value. Here, you would reject the null hypothesis if your T-value is less than this negative critical value.
Two-tailed tests involve both positive and negative critical values, where you reject the null hypothesis if the absolute value of your T-statistic is greater than either the positive or negative critical value.
Understanding when to use which critical value essentially hinges on your hypothesis's direction. The choice of negative or positive critical values in one-tailed tests allows the test to focus on deviations in one direction, which can be particularly useful when a specific change direction is hypothesized before data collection.
Q3- Appologies for that. Please use the following link of all the calculators that we have developed (including the one on hypothesisi testing): https://365datascience.com/calculators/confidence-interval-calculator/
Hope this helps!
Best,
The 365 Team
Hi Ivan,
Thanks for your prompt and detailed reply. Things are clearer now. I would like to offer 1-2 suggestions: the video lessons be altered to include the left-tailed and right-tailed tests. And also a modification in the video lessons: the concept that Absolute value of Z/T should be compared with positive critical z/t is so well explained that it is impossible to unlearn - this calls for a correction after reading this entire thread. I feel that these modifications will cover appropriately and completely the important topic of Hypothesis Testing
Hello Dhaivat,
Thanks for reaching out!
I would also suggest that you read my theoretical section below each calculator and particularly about hypothesis testing. I have tried to explain everything in a clear and concise manner but also cover specifics not touched upon in the lectures.
This extra material will help you understand the whole statistics material considerably better I hope.
Best,
The 365 Team
Wow!! A lot of information! Many thanks, Ivan. Very Very Helpful!!
Hi Ivan, incidentally, would you have a PDF of the theory that you have included with the calculators? That would be very helpful for reference and further learning.
Regards,
Dhaivat Parekh
Hello Dhaivat,
Unfortunately, I have not created a word document covering it all.
But that's a great suggestion that I will explore in the future.
Best,
The 365 Team