Skip to main content
Loading Explore guides…
Inferential Statistics Tips · Acedra
Home
Explore
Mathematics
Inferential Statistics Tips
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
52 segments
Copy
Hide video
0:00
Hey there welcome to Grad Coach TV where we demystify and simplify the oftentimes confusing world of academic research. My name is David and today I am chatting to one of our trusted coaches Kerryn about six tips to keep in mind to ace your inferential statistics.
0:22
So hey Kerryn, welcome back to the Coach Cast it is great to have you. Today we are talking about inferential stats. Hi, David, great to be back thanks for having me. So today we are talking about those inferential statistics and we have got a few tips in mind for everyone. And the first one is to ensure your hypotheses or models are clear and consistent with your topic. Kerryn what are we looking out for here? So I think there are a few things to consider and the first
0:46
and most important thing is what is a hypothesis? A hypothesis for the vast majority of people is a statement and it is a statement that often can be a statement of difference, a statement of change, a statement of a relationship and there are many other things that a hypothesis can be but those are for the vast majority of people what a hypothesis would look like. So something like group A differs from group B in some way in a variable or variable X is related to variable Y
1:20
like weight is related to height or something like that. And you could also have something like there is some change so group A changes in some variable between time and maybe it is their attention span or their knowledge of a subject or something that you are testing. And this is often a statement that differs from its counterpart the null hypothesis which is its opposite if you will and we actually call it the null hypothesis being the dull hypothesis because
1:50
it is a statement where there is no difference, no change, no relationship to see here. And essentially what you are doing with a hypothesis is you are testing it you are saying hey is there something to see here in the sense that is it different, is it changed, is there a relationship? If not then you reject the hypothesis and you accept the alternative hypothesis or if there is no relationship or there is no change then you will reject the one hypothesis
2:20
and you will accept the null hypothesis and then vice versa. Essentially these have to be testable in order for a hypothesis to even exist. So if you want to have a testable hypothesis you need to make sure that it is very specific. So you need to make sure that one variable is related to another variable ideally at its simplest for more simplistic hypotheses. I think that is really a good point and probably an example of a less than ideal hypothesis is the more a person lifts
2:55
the bigger they will be. Kerryn, what is sort of wrong there with that hypothesis. Yes, so you are not being very specific right so lifting incorporates a lot of different things a lot of different kinds of exercise and bigness can be measured and evaluated in a lot of different ways. So a better hypothesis is one that we will be much more specific so something like the more reps somebody performs a week in terms of bicep curls the larger the bicep diameter will be
3:26
relative to the time that they started maybe the exercise program or something. In this context you can measure the you know the circumference or whatever it is for the biceps and you can count the number of reps that somebody does so it ends up being far more specific and a lot easier to test yes or no. Whereas bigness a whole bunch of things is related to maybe somebody's weight, somebody's overall size there are a bunch of different kinds of lifts somebody can do that are
3:54
related to that as well if at all. Various things that are in your particular example that could be measured or that could be you know impacting each other that are not really easy to test for whereas something much more specific is a lot easier to test. And that is not to say that it cannot be done it is just to say that you probably need to at least unpack it into more specific variables. For me, personally like it when there is some sort of direction
4:24
to the hypothesis as well so something like instead of X is related to Y, X is positively related to Y says that as X goes up so does Y therefore you have got a direction. Or in this context the bigger the diameter of the bicep will be is a direction because it is larger, not smaller. So something that is a little bit more specific it tends to be a little bit easier to test a little bit more theoretically easy to connect to the literature or to you know
4:56
the theory behind it. And so I usually encourage something a little bit more specific even then. I think that is really helpful when we think about sort of what a hypothesis is. But something that we should really keep in mind as well is that a hypothesis should be based on something observable whether it is in the literature that previous researcher found or if it is as a result of practical observations. We are not just making a guess at something right, that is not a hypothesis
5:24
then that is that is a guess. And so a hypothesis in general and in most academic research will be based on the literature or something that has been observed or quantified prior.
5:38
So our second tip and it is an important one is to not neglect your descriptive or exploratory data analysis steps. And this is something we have kind of spoken about in a previous video but Kerryn, maybe you can give us a little bit more detail about why we need to keep this in mind specifically when we are working with inferential stats. Yes, it is absolutely important to really consider your descriptive statistics and exploring
6:03
your data before conducting your inferential statistics mainly because the outputs regardless of what software you are using are quite complicated for inferential stats. And if you
6:15
are trying to figure out what is going on and it is not you know the output that you are trying to consider but maybe all of these other issues that you just never really considered initially then that can lead to some problems. Some good things to look out for in these exploratory steps that you know almost come naturally to that phase are things like are the variables that are meant to be normally distributed normally distributed and have a bell-shaped curve. Are your
6:42
sample sizes appropriate for the various categories that you are wanting to compare? Are you understanding the kind of data that you are working with? Is the categorical data looking like categorical data is the scale data looking like it is meant to like having some kind of you know variation? And if you kind of do not know what I am talking about it is okay, but it is really important to at least have that initial intuition about your data before
7:08
conducting the tests otherwise you might come across some problems. I think that is really good advice and one thing we want to do is taking it further is think about cleaning and preparing your data. So when we were speaking about it earlier having an intuition about the type of data you have and how it should be presented if you have got a categorical variable that has three groups red, green and blue if you do descriptive analysis and see you have six groups because you
7:34
have got a capital green and a lowercase green that is a problem right, that is going to throw out all your analyses. So we just want to do these exploratory analyses to identify issues before they are a problem and also to help guide us as we move through subsequent statistical analysis.
7:54
So our third tip really follows on from the previous one and that is to ensure the assumptions behind your tests are met. So you have got a test you have got in mind each test has assumptions. Kerryn, maybe tell us a little bit more about this. Yes, so a lot of people get a little bit scared about what do you mean I have to test some assumptions before even doing the test. But they are not very scary they are just really common ones. There are some really common ones
8:17
that are used for many tests. One assumption for a lot of common tests like the T-test or ANOVAs or Linear regressions is that there is some kind of normal distribution to your numeric data set. So and that basically means your data looks like a bell curve which we sort of touched on in the previous session. This is not you know a strict thing in the sense that like your data you know can be maybe a little bit more peaked or slightly skew
8:48
as long as it is near normal it should be fine. But if you have got a very skewed data set or one that has a weird bimodal distribution it is got two weird peaks that are far apart then that might actually impact your articles, it will impact your outcomes. And so it is worth just looking to see does your data look like really wonky or maybe a little bit skewed that is not bad but it means you might need to think of a different test. Same with sample size
9:12
some tests really require larger sample sizes than others. Often you are required to make sure that you get the adequate sample size before even analysing your data but some good ones to look out for is the Chi-square test often you need a really big sample in order to be able to detect any differences. A third thing to look out for in my opinion is the data type. So making sure that if your test needs a categorical variable and a numerical scaled variable that you have those
9:42
variables and the variables that you want to test look like that. And if you do not you might need to consider a different kind of test. And there are some resources that you can use online to see what kinds of variables you need to use but we are not going to go into that for too much now.
9:58
All of these kinds of tests that you need to sort of you know maybe think about your assumptions does not mean that you cannot do anything. So for instance, let us say your data is not normally distributed or that your sample size is not large enough then you might want to just consider a different kind of test. So for an ANOVA if your data is not normally distributed you might consider something like a Kruskal–Wallis and if your sample size is
10:23
not big enough then you instead of a Chi-square test you might want to consider something like a Fisher's exact test which is not as sensitive to sample size. So those are some things that you can maybe look at to maybe mitigate the problems that you might be facing with your data set. I think it is important to note that every single test has its own assumptions and it is very easy to feel overwhelmed oh, my word do I need to know every single assumption
10:48
for every test? And the answer is very clearly no there are a ton of resources out there online in stats tests but in stats textbooks that are going to give you all the information you need. But the important thing is that you go and check out what those assumptions are and you figure out how you are going to test them and whether it is right. And an easy example of why this is important is you might want to be doing that ANOVA to compare categorical variables.
11:15
But if your data is not saved or organized as a categorical variable that can lead to problems in the interpretation. And so we just really want to be careful with these before we do the test rather than trying to do it after the fact and getting really confused why nothing is working.
11:34
So for our fourth tip, it is really something that is really near and dear to my heart but no significance does not mean you have done something wrong. Kerryn what am I getting at with that statement? Hopefully, this relieves you but if you are doing quantitative research at all it is because you are wanting to test something usually. And sometimes there is just no difference between
11:55
groups or no relationships and the null hypothesis the dull hypothesis is what your data is pointing to you know and accepting. And that is just the way it is that is that is why you did this research in the beginning because there was an opportunity for it to go either way. Obviously, my first piece of advice is make sure it is not something else, make sure it is not something like you know your sample size is too small or like you know maybe there was the wrong test or the
12:23
wrong assumption and maybe you need to be a little bit less conservative or something with you know some of your choices. And those are some things to consider but to be honest it is fine if there is no significance. You can still have a good thesis if it is well executed. It is not going to be the most exciting thing maybe but it might be it depends on where you know it depends on what you are doing but you can still pass and you can still do very well. It is
12:50
about execution it is about consistency there are so many other things that come into play. If you describe your data well enough and you have you know a significance value that is not significant that is fine that is that is why you did this research. I cannot agree more, especially because when we are marking dissertations and thesis we are not looking at oh, is it ground-breaking research? You are being assessed on can you do research,
13:17
can you effectively convey your research, can you argue it can you present it. So do not feel worried if you are not getting a significant result. That is totally normal and I will say it is way more common than you think it is. It is just unfortunate that most people do not publish non-significant results. But that is a whole different ballgame. What I will say though is do not feel always that those non-significant results mean your research is a failure. Sometimes
13:44
non-significant results are the most interesting results particularly if they are contrary to what the literature is stating should be happening. That is then a great place for you to try and figure out what is happening and that is where really exciting papers actually come from.
14:01
So our fifth tip again follows on from the previous one and that is when you are interpreting your significance. Try and understand what that significance means. So Kerryn, what is the importance of the P-value or significance value? Yes, and I mean I am guilty of this as well we sort of just mindly accept a what we call a cut-off threshold or a significance level as 0.05. Which means that if you get a P-value that is 0.04 it is significant but if it is 0.06
14:31
it is not significantly different. And you can see why that is maybe a little you know maybe not quite one hundred percent very descriptive of what you might be seeing. So just because something is you know significant or not does not mean anything if we cannot explain that a little bit further. So it is really important to know what we mean by significance. If you choose a cut-off of 0.05 you are essentially saying that you are okay
15:00
with a five percent chance that the value that you are going to be getting if it is significant is a false positive. That you are saying the outcome is significant when in actual fact it is not it is just due to maybe sampling or you know chance essentially. And this threshold of five percent or 0.05 is usually because of historical reasons that we do not really need to get into. But because of this, we need to really look at the P-value often with a little bit more depth.
15:31
So for instance, if you ran one test and you had one hypothesis that you wanted to test and you had one T-test that you ran and the significance value that you got was P equals 0.06
15:44
you know you probably do want to talk a little bit more about that. You probably want to say in your discussion that hey there is something going on here that it is it is not significant at the cut-off level that I chose but it is what we call approaching significance or near significant. And some people like that some people do not like that but it is valuable to say that maybe future research with maybe a larger sample size maybe considering a few more alternative variables
16:10
might actually be able to unpack this a little further and maybe even see significance where your test might not have done. Similarly, if you are running twenty tests the chance of you getting a false positive is very high. So you actually want to be more conservative the more tests you run. So what we tend to do is we tend to lower that P threshold sometimes or at least report on a lower P threshold like P is equal to 0.01 in order to try to make more conservative
16:42
examiners or statisticians happy if you will. So if you are running a bunch of tests I will rather consider if you can maybe put into a single model or a single statistical test like instead of a bunch of T-tests. Maybe consider a nano or something like that and if and yeah, so just give it a little bit more thinking before running a bunch of tests or if you have to run a bunch of tests consider lowering that P threshold just to be a little bit more conservative in your
17:13
final discussion and outcomes. I think it is also worth noting you do not have to feel like you have to just choose a critical threshold. So there are some great techniques out there on how to adjust for those multiple comparisons. The classic example is a Bonferroni correction but there are other examples out there as well. You could do a Benjamini-Hochberg correction or a whole Bonferroni method. The clear answer is there are ways to deal with this and if you are worried
17:43
about it, if you are concerned about multiple testing take a look at the literature and see what multiple test comparison corrections you can use. It is only going to help you going further. The other bit of advice that I have and maybe it is not super commonly engaged within the literature is you should be explicitly reporting your P-value. Do not just say it is less than 0.05 there is meaning to that data as well. So be specific it is significant
18:11
at that threshold and my P-value was 0.0024 and that gives you some insight there as well.
18:21
So for our final tip and this again follows on from the last one we want to be looking past our P-value. Kerryn, what should we be looking for other than that significance value? Yes, so one mistake, once again we all do it is we see the test, the test outcome we look at the P-value and it is significant or it is not significant and now what? But you have to you have to be able to explain what is going on and there are two things to consider in this. The first is to look at the
18:55
other statistics do not just look at that P-value. There are other things to look at. For instance, if there is a difference between your groups it is good to know which is bigger which is smaller right, you need to be able to understand that. And in that case, the exploratory statistics might come into play or something else. Maybe there is a relationship between your variables but what is the relationship not just is it significant but is it positive or is
19:21
it negative? Is it a strong relationship and the variables really go across a single line or is it quite a weak and loose relationship? You do not get that from the P value you get that from this the correlation coefficient or the R-value or if you are doing a regression maybe an R squared value. In this context, it is really important to then know that it is not just important that it is significant it is important to also think about why it is significant what that difference,
19:48
what that relationship is and how big it is, how big is that relationship, is it very strong, is it a big difference and that is what we normally referred to as effect size more or less. A good example of this and one that I really like is that of breastfeeding research. There is a lot of research that has thousands and thousands of people in their samples and they all seem to show that there is a significant difference when we look at samples of thousands of babies
20:17
in terms of their breastfeeding and their IQ. Now that seems pretty alarming right that means that oh no breastfeeding is really good but if we look at the actual IQ points differences we are talking about one or two points. And so at an individual level, we are not talking about a big impact on IQ and it is something that can be mitigated or controlled with a bunch of other kinds of things that you can do for your child during their childhood or whatever maybe read to
20:44
them before bed or something like that it seems to have a much bigger impact. So in that context effect size is probably far more meaningful in some cases than just the significance. I think it is really a good point to think about this. This is often misused in sort of media reports on scientific articles you will see that sort of heading saying oh no double the chance of cancer if you do X, Y or C. But often enough they have not accounted for the effect size. So that
21:15
doubles the risk is changing your risk from one in a hundred thousand to two in a hundred thousand. So that is worrying but not actually as alarming as you think. So it is important that as scientists we report on effect size and really engage with it critically. We are not just using it as a I am doing it because I was told to. So Kerryn, thanks so much for joining us today it was really great to chat to you about inferential stats. Always fun to talk about stats, David.
21:47
All right so that pretty much wraps up this episode of Grad Coach TV. Remember if you are looking for more information about inferential statistics be sure to check out our blog at gradcoach.com/blog.
21:59
There you can also get access to our free dissertation and thesis writing mini course which will give you all the information you need to get started with your research journey. Then do not forget if you are looking for help with that analysis or dissertation be sure to check out our one-on-one private coaching service where you can work with one of our friendly coaches just like Kerryn. For more information and to book a free consultation head over to gradcoach.com.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat