Skip to main content
Loading Explore guides…
Data Science · Acedra
Home
Explore
Computer Science
Data Science
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
146 segments
Copy
Hide video
0:02
>> Welcome, everyone, to session two of the Data Science Summit, the next session is going to be end to end data science with our studio. And our speaker today is Ryan Estrellado from South County SELPA at SDCOE
0:18
So I'm going to turn it over to him. But just so you all know, we're going to be taking questions in the chat today, and I will be reporting those questions back toward the end of the session. So he's going to make a little bit of time in the last five minutes of the session to take any and all questions that you put into the chat. So feel free to do that. And we are also recording the session. And now I'll turn it over to Ryan.
0:41
Thanks, Carrie, I appreciate it. And good morning, everybody, I'm so excited to do this and hang out with you all, like I feel like this is something you don't get to do very often. We just talk about the techniques in code and all those things that I think we've got a common interest around. So, so really excited to do this. My name is Ryan Estrellado. I work in South County SELPA and I work for a project called the Equity Disproportionality and Design Project. And our goal at this project
1:08
is to work with school districts in California to promote equity and special education. A big part of that work is learning with our community about how we can use data practically to promote our equity goals. We also build a data tool that estimates equity related compliance scores, and the original tool was built on R and all the diagnostics and things that we that we do. I still do in R and so it's a very it's it's come into quite practical use for me.
1:40
And and I'm looking forward to sharing about just the very beginnings of sort of what that what that looks like. We're going to talk today about what it's like to use the statistical programing language R is not the only language you can do. Stats in Python is another popular one, and there are others. But R is the one that we're talking about today. And we'll be spending the bulk of our time doing a live data analysis so we can all sort of walk through the very basics of data analysis.
2:06
I wrote this talk with the beginning users in mind or somebody who is just thinking about learning R and maybe hasn't dipped their toe in yet.
2:17
But aside from the, the live data analysis, I want to make the case that using data science tools and education is more than just using the tools. It's putting the tools to work for the benefit of students. And so I want to say a little bit more about that in the form of the story. I have a friend who was a programmer in a few years back. He was talking to me about talking to me about his personal work style. And he said that he wished he could just shut the door of his office
2:44
and then write code for a few hours. And then every so often somebody would come by and slide a slice of pizza under the door to keep him nourished, which I thought was funny. And I get the appeal because I also like to spend a lot of my time writing code and solving interesting data problems. But something that I've learned over time is that if I can't share my data science work with a wider audience, including non data scientists, Then number one, that gets lonely really fast
3:13
And then number two, I'm giving up an opportunity to use these really great tools to improve the lives of students. So on the surface, this talk is about using a technical tool for data analysis. That's mostly what it's about. But I also want to challenge you to do the unspoken work of data science and education, which is figuring out for yourself how you can use these tools to solve meaningful problems. So Donald Knuth, a computer programmer, wrote in his 1974
3:40
essay computer programing is an art. He says, I must confess that I really enjoy writing computer programs and I especially enjoy writing programs which do the greatest good in some sense. So we've got about 40 minutes together today. I think it's important to pick a goal that we can win it together. So I suggest that you use your imagination to envision how you use the tools that I share today to get the results that you want in your job. We'll be looking at a lot of code,
4:10
but if you're brand new to that, I assure you, don't worry, you'll be leaving with everything today. The slides, the code, the PDF printout and the data, all for your reference. So I know that it takes some time to process. If you've never seen this before, I'm going to give you everything. So just know that that's coming. If anything speaks to you today, share post on Twitter or story on Instagram and tag me, I'm ry_estrellado That's ry underscore estrellado.
4:38
And I'll share it with our community after after the talk that I share a lot with both data scientists, but also educators on both of these platforms. So do a post,tag me. I'll see it and then I'll re share for you.
4:53
OK, so what are we going to learn today? I'm so excited about this, we can do three things. The first is we're going to describe the tools. So I'm going to show you like R and RStudio or kind of the main things that we're going to talk about. But I'll mention a couple of other things that I use, like almost every day. The second is we'll do a live analysis in RStudio. I've got an imaginary scenario. It's not a real scenario, but hopefully it feels plausible to you.
5:17
And then the third is we're going to talk just very briefly about what it's like to bring data science tools to your job if they're not already there.
5:26
The first to kick us off here, we're going to talk about the tools,
5:30
the first one that I'm going to introduce is the main one, which is R, the statistical programing language. R, R is a programing language. All the code that I'll be showing you today is in our 1991 Ross Ihaka and Robert Gentleman at the University of Auckland in New Zealand developed it based on a statistical language called S
5:55
and in 1995, they went and they made their version of that which they called R open source. So that's why R and almost anything related to it is free. There are some elements that aren't that like I assure you, I've been writing in R and doing a lot of work in R since 2012 and I haven't paid a cent to use it or it's related products.
6:17
Now the second one is RStudio, which is the software application that you'll use to write your code. And it's not the only one that you could use, but it's the one that most people use and certainly is the one that I use. I'm going to talk about these two things in a little bit more depth in the next slide. I want to also introduce this concept of packages so R packages are ways that, that the R community extends the use of R to improve it. And so there are all kinds of packages
6:45
that you can sort of bolt on to your base installation of R that'll help do things, help you do things like fit statistical models or prepare your data sets or run reports or do graphics. There's all kinds of sort of additional things that you can add based on what your job is like. And all of those are made, most of them are made by the community. So very community driven thing.
7:08
Finally last tool, R Markdown, which is a kind of document, so I'll just say what the authors of this book called R Markdown The Definitive Guide, the way they describe it, they say it interweaves narratives with code in a document. You can knit the document to dynamically generate results from the code and you will get a report that's from our markdown, the definitive guide by Yihui Xie, JJ Allaire & Garrett Grolemund I'm going to put the link to that in the last slide
7:40
that is available for purchase, but also available for free to read. So I'll put the free version of that or I did put the free version of that in the last slide. That'll that'll be there, too, with the authors here saying about R markdown is a tool, is that it's a tool for making reports that include not just the description that you have about your analysis, but also the code that you use to produce the analysis and then any related graphics. It has a lot of similar functions
8:07
that you might recognize from, say, like a word document or Google Docs. It does spell checks. It allows you to to to edit the actual text so that your narrative is readable. Almost many of the things that you can do in a word document, you can also do with R markdown. But of course, the special thing about it is you can also put your code in there and then any graphics that you produce
8:33
are also sort of visible all in one place. And so when we do the live demonstration and a couple of minutes here, all of that is going to happen in an R markdown document.
8:43
OK, so those are the four tools R, RStudio, R Packages and then R Markdown, all of these are free. You can literally start using them after this talk. You just download all of the tools and begin using them. I put the link. I did put the link for each of these in the slides. In the last couple of slides of this talk,
9:05
I want to talk about something that can get really confusing. At least it was for me when I started learning that this and this is the relationship between R and RStudio. Sometimes when I was first learning this, I got these two things mixed up. I thought they were the same thing. And they're definitely not in the way I'd like to explain that is to use an analogy of of creative writing versus creating a data analysis.
9:27
And when you think about writing and sharing writing, we kind of think of these three components. Right? The first is you've got to have some language that you speak and write in. And so, for example, Spanish or something like that. Right. That's what you use to convey the ideas, but you've got to get that onto paper in a way that's shareable. And so in the modern era, we use software like Google Docs or word document or something like that. We write our Spanish in the word document
9:57
and then and then we have to share it. And so with the word document does is it allows us to output that information in a way that shareable with other people. And so we print it or we render it as a PDF for we put it on a website or something like that. The relationship between R and RStudio, very similar thing. You need a language to be able to do your statistical analysis and that's R Ah, and then you need some software to be able to manage your writing of that language.
10:25
That's RStudio. And then RStudio facilitates the sharing of that analysis.
10:35
And so from from from RStudio and in combination with R Markdown, you can render a PDF.
10:42
You can put stuff on the website, put stuff on GitHub, all kinds of ways to share. But the point is, is you kind of need these sort of three things, right? So so Spanish is to Google Docs roughly what R is to RStudio.
11:00
OK, so now let's go into the data analysis again, I want to make sure, because I know that this is a lot to sort of squeeze into 40 minutes, there's no need to remember exactly how I'm doing this. I'm I just think that's too much to remember in 40 minutes. But what I do hope is that you walk away with an improved understanding of what's possible when you do data analysis in code. And again, I'll be sharing just a bunch of free stuff at the end that you can use to continue your journey.
11:25
But I think what we can get out of this together, if this is a brand new to you, is just some of the vocabulary and some of the kind of thought process behind what it's like to use a statistical programing language versus something like Excel or Google Sheets or something like that with any data analysis, no matter what to use or if you use a tool,
11:46
what matters is that we come up with a useful and interesting question. Right? It's super important as I'm going to set up this live data analysis with a little bit of context.
11:58
Let's say you work at a county office and your supervisor is building a training service and they want the training service to be available to all the school districts in the county. But resources, of course, are limited. And since in this imaginary scenario, you're in the early phases of planning, you want to learn about how many school districts in our area, I'm sorry, how many schools each district in our area has. And so you're asking planning type questions like are
12:27
is that numbers similar across all the districts? If they are not similar, how different are they? If we just picked one number and bought materials in bulk, would that do the trick? Can we pick a uniform set of numbers, all these sort of planning related things? Right. So that's a question we're going to explore together. We're going to use an analytic workflow that looks something like this, we are going to identify the question which we just did.
12:52
We're going to import a publicly available data set to help us with this analysis. I'm going to prepare that data for analysis. We're going to pick a way to answer those questions. So like a method, we'll do a very brief visualization
13:08
and then we'll analyze the data and then we'll talk about how we would share that. This is admittedly a simplified version of of what we tend to use R for, but I think it illustrates the concepts. Well, a couple of side notes here. The first is when we do the visualization, I am not going to go it's going to be very brief. But I do encourage you to go to David Dobrowski talk, which I think is at 11 o'clock, where he's going to do a really deep dove
13:35
into data visualization and in particular a deep dove into a tool that I use every day called ggplot I encourage you to go to that for more on data visualization. Second side note is I've adapted the analytic workflow that I use is adapted by one that Hadley Wickham and Garrett Grolemund could use in their book "R for data science" also available for free and also in the link in the final slide of this talk. But that's what their version looks like. OK, so I'm going to stop sharing here
14:08
and I'm going to switch to.
14:15
R console.
14:21
OK, so what you're looking at is are two of the things that three of the things that we talked about, the first is R is an example of our code here. Any time you sort of see this these sort of gray text areas, that's where code goes into your R Markdown document. This whole area right here is an R Markdown document. You know that because in the title of the file, it ends with the file extension .RMV, which stands for R Markdown. You'll see right away
14:56
that there's just some basic narrative in here.
14:59
So, you know, that's what I was saying earlier, that R Markdown combines the narrative that you write, but also the code all in one place. Right. Your reader doesn't have to sort of switch back and forth between things.
15:14
I think sort of that that the important thing to notice here is as I work through this analysis, you'll see these little gray areas and that's where our code is going to go. And I'll sort of type that out.
15:25
There's an interesting thing that happens or that happened when I start to learn this, as I realize, as I began to work more and more in R Markdown, that a lot of this these analytic workflows, especially when you do an R Markdown, feels a lot like a conversation with myself and sort of allowing my curiosity to to sort of spread a little bit. And then as I respond to these thoughts that come up in my head about what to analyze, I'll write some code, I'll run it. I'll see what the results are.
15:57
Maybe I'll go back and sort of interact with something I did earlier because that triggered another question. Is a lot of this sort of exploratory type of feel, which can sometimes be different, a different way of thinking about data analysis than we have before. But I want you to just kind of notice the way I sort of talk through it, because it's the way I actually do the work when I'm doing data analysis. And that just helps me sort of get to the right place
16:20
and answer the questions as meaningfully as possible.
16:25
OK, so let's let's let's do this first. And radios are going to load our packages. Remember that packages extend the use of R gives us more tools to work with. We're going to use to today. And I can explain too much about what these do, but I will summarize it. The tidyverse is a group of packages. When you run the when you load up the tidyverse you over to get all kinds of useful tools, things that will help you prepare data ggplot is in there to help you do your visualizations.
16:54
I'm going to load that one. And then the other one is I'm going to load this package called here, which just helps me navigate through my file structure simply and just kind of keep the where I'm keeping each of my files straight in my head and also makes this script more transferable. So if I sent this to David or to Shannon or to Carrie and they also use a package here, it'll help our find what it needs and a little bit more of a simple ways.
17:21
I'm going to go ahead and run that in this code chunk here.
17:27
And it just spits out a bunch of syntax that we don't really need to get into here, but this is kind of our cue that, OK, all the packages are load up and loaded up and ready to proceed to the next step. So just to keep things tidy, I'm going to click on this arrow here so we don't have to look at that. I've got my question written out in the narrative.
17:49
How many schools do districts in San Diego have on average is sort of the the question we're going to ask in order to analyze that data, we actually need the data itself. Right. So our first step is going to be to import the data. We're going to load the data into memory so that we can begin our analysis. And the data set that I'm going to use is publicly available data set from the California Department of Education. They hold enrollment data for many, many years.
18:16
And I'm going to pull the most the most recent year.
18:21
I'm going to use a function called Read underscore TSV stands for Tab Separated Values and that function is going to allow us to to to read the data set in. Let me show you what that looks like. read_tsv here
18:40
And then the name of.
18:44
The file.
18:47
And then when I run this, what I expect to see if it works, there you go, is the data set. So we've got now we've got this data set here. I'm going to name this data sets. It's we're going to be referring back to it frequently through our analysis. I'm just going to name this. I'm going to name this enroll like that and I'll run it again.
19:11
Now, we'll see down here in our environment pain that we have this data set loaded up into R got 131,616 rows and 23 columns. So that's loaded into R.
19:26
So one thing you'll notice right away is very different from working with a spreadsheet is you can't actually see the data super disorienting at first. Sometimes you just want to see the data. So there's a lot of ways to do that. But I'm going to do it this way and use this function called view. And we're going to view and roll,
19:44
and when we run that, what it does is it shows it to us and more like a spreadsheet kind of form, right?
19:51
Just opened up another tab and showed it to me, and you could see that what we have here is our 23 columns, over 100,000 rows, every county in California and indeed every school in California. The data structured and kind of an interesting way here where you've got each row is a unique county, district, ethnicity and gender. I've worked with this dataset a lot, so I'm fairly familiar with the way that it's structured.
20:20
But what I want for you to to notice here is that if we have to count and remember, our original question is we want to get the average number of schools in each district. And so in order to do that, we need to we need each row to be an individual school. And you can see here that that's not true in its current form. There is 1,2,3,4,5,6,7,8 9 Castle Rock instances which are all from Del Norte.
20:46
You look at a if you sort the signal, look at Alpine, because I think I use Alpine as an example to check this here.
20:53
I think Alpine's got 5. 5 schools in their district.
20:58
If that's what you see this way and filter.
21:03
They'll look at Alpine. There you go. So an Alpine elementary got 1,2,3 4,5,6,7,8. I misspoke. 8. OK, that's another Alpine, I guess it 8 in in that area. This is L.A. County. And of course, we want this to be just one one row per school. So that's that's what we're going to need to do as a first step to prepare it. A lot of these data sets that you use, the publicly available ones, are used for all kinds of research. So they're not going to be ready, made for you to analyze.
21:34
You have to clarify your question first and then you've got to prepare. So let's do that.
21:40
OK so.
21:43
How do we make this data set so instead of having many instances of the schools in there, we just have each row is just going to be one school. We're going to use this this function called distinct. So we're going to start with enroll and.
22:01
Actually, let's do this first, we know from a question that we just want to know San Diego County, so let's just filter the data set down to that. First, we're going to use this function called filter, which will allow you to filter the data set.
22:14
And we want only the values where the variable county has San Diego in it. So I'll just show you what that looks. I can run that. And then now you see notice a couple of things. First off, the data set is now ten thousand eight hundred seventy six rows, a much, much smaller, of course, because it only has San Diego in it. And if you just sort of visually inspect it, you can see here that the only county that you have now is San Diego in this data.
22:42
You also see if you kind of scroll over here to the right, that you've got a bunch of wolf canyons. That's we only want one Wolf Canyon. Otherwise will will be County Wolf Canyon more than once. Right. So how do we do that? We're going to use a function called distinct and distinct does. What distinct does.
23:03
Is is it drops every row, except for only the I think it's the first instance of whatever you pass to distinct in this case, we're going to say let's do it by.
23:17
District. And in school, I do it this way because there I learned the hard way. There's a couple of schools at the exact same name, but they're in different districts. So now we're going to tell our drop every row except for only the distinct district and school combinations. And then also, of course, only the ones in San Diego. I'm going to run that. Let's have a look. There you go.
23:41
Notice a couple of things here dataset just got a lot smaller because now you just have the unique schools. 795 total. And if we look at Chula Vista, there is indeed only one Wolf Canyon now.
23:53
And in fact, there's only one in each school. All right, so now our data sets ready to go do some counting, right? So we want to find the average number of schools in each district. We got to count them. Oh, actually, let's do this. And we're going to go back to this a bunch of times. I'm going to name this data set to real name is SD, just as it stands for San Diego, you name it, whatever you want.
24:15
And now every time you want to refer back to that data set, we'll just refer back to SD.
24:21
OK, how do we count the number of schools in each district? We use a function called count. All right. So we start with our data set aside
24:32
and then we're going to count district.
24:39
I'm going to sort it. So it's the highest count at the top,
24:44
so super interesting here it is a little data science trick. We want to know the number of schools per district. So, so, so why do we pass the function count? Why do we pass a district instead of school? Well, the reason why is if you look at.
25:02
Let's look at SD
25:05
If you look at it like this, you see that there is a school, a school only appears once in the data set. We don't really care about the name of that school, but we do care about the number of times the district appears in that data set, because if Chula Vista has 5 schools that have they've got like 40 something, almost 50, then the words Chula Vista will appear 5 times in that data set. So how do we know how many schools Chula Vista has in a data set that has only one real per school?
25:35
Will you count the number of times that Chula Vista appears? And that's why we did that here.
25:43
So if we run this, let's see what we get.
25:47
There you go. You've got a column of districts now and then this new column, which is by default named and which I think is like for sample size.
25:58
But we see that San Diego Unified appears in this data set 223 times. Why do they appear 223 times? Well, because they've got 223 schools. We also know our analytic brain starts to turn on and immediately we start to see some of the answers to the questions that we're trying to ask. And the one that pops out at me is I don't think the districts have the same or even similar amount of schools. You got this one district in San Diego County that's got many, many more times
26:26
than the next largest or the next district. That's got a lot of schools, which is Chula Vista, also has a lot of schools, but not as much as unified. So put a pin in that sort of save that for later, but just something to notice.
26:41
All right, OK, so next, we're going to pick a method to answer the question since we got this data set that has that has the district in one column and then in the next column, the count of each district. And the boss wants to know the average number of schools in each district. Well, I think the method that we'll use is will take an average of this column. And there's 50 rows here. If you take the average of it, we'll be able to report what that averages. But we also want to know
27:16
how different these are from each other. Like if the average happens to be 100, does that mean that we can safely guess that each district has 100 schools? Well, we can kind of see visually here, just look at the data. That's probably not a great way to do it so we can try to quantify that difference with a standard deviation also.
27:34
All right, that's our data. Let's do a quick visualization just so that we can see the brain in the eye can interpret and compare lengths really, really well, almost instantaneously. And so we're going to use a bar chart for this just to help us sort of grasp the magnitude of some of the difference. But we might also save this code for later in case we want to do a presentation. And we can and we can use this to demonstrate. Also, one problem we got to deal with up front is about 50 rows,
28:02
50 rows in a visualization is possible, but it can tend to get a little messy. So let's knock that back down to about maybe 10 rows. We can take a top 10 highest school districts. I'm going to visualize that. So first thing we got to do is we got to get to that top 10.
28:21
The way I'm going to do that is I'm going to filter again.
28:25
Uh. OK, so we start with our days at school count.
28:31
And then we're going to use a filter or did we name a school count? We didn't. Let's name the school count
28:43
There you go. All right, now, we don't have to keep on writing that code anymore, we'll just say school count R will know what we mean when you use this function called min_rank. And if you use SQL at all, you'll know that min_rank is, I don't know SQL. So forgive me, SQL people, if I'm saying this wrong, but I understand these are called window functions, which are ways to sort of like rank your values in the data set. And we're going to pick.
29:11
Just the ones that are in the top 10.
29:19
Uh. OK, that doesn't look right, you see here.
29:26
Try this again. Well, maybe I did it. OK, I'm going to clear this out and just run it again.
29:34
So let's run that.
29:40
I was expecting to see San Diego up at the top.
29:44
And I did not. So I'm going to try this again. oh it's still happening. OK, so what's happening here, Filter?
29:57
I had it backwards. OK.
30:01
That's not greater than 10, it's less than 10, or if we did this right. I'm going to run it and we'll know that we got it right because San Diego Unified should be at the top of that big number. So let's try it. There you go. OK, so then you got your top 10, actually top nine, and I think it's because there's like a tie in there somewhere. But but but you get the idea. We've taken it from 50 to, you know, to 9. So now we can visualize this a little bit easier.
30:29
So we're going to use one of my favorite tools ggplot. Again, friendly reminder, David Dobrowski doing Deep Dive into the plot at 11 o'clock. You go to that, but let's do this really quickly. The thing I love about you plot as you build it up in layers. And so the first layer is just going to be the grid, like the empty grid and the call JGI plot like this. We're going to tell it that we want to use this dataset called top 10. And name this top 10.
31:00
Top 10, and then we're going to say for the X axis, let's use the school district name for the Y axis we are going to use N which is our count of schools run that we should get a blank grade, do we? Yes, there's a blank grid now. There's a big old mess down here, but we'll fix that later. This is all the school districts. They just get jumbled up because there's because of the space there. But we'll make that more readable in a second. Let's build up the next layer,
31:30
which is the bars going to use this function called geom_bar.
31:36
And by default, it it does it like a histogram. So I'm going to change that by bypassing it. This argument stat identity. Now we've got our bars. So I don't like this because it's hard to compare. I'd rather sort it. So it's like you've got it from like lowest to biggest or biggest to lowest. So I'm going to change this X axis here using this function called reorder.
31:59
Like that.
32:02
Again, I know that this is a lot all at once, so no need to memorize all this, mainly what I want you to get out of this. See how the conversation with myself works and get the basic vocabulary. You're going to get all of this at the very end, the code, the data. You'll be able to run it all yourself also. But I just want you to see just sort of what it looks like. OK, so let's try that. That should reorder it. Does it? Yes.
32:26
OK, and then the other thing I'm going to do is I think we can clean this up, this access up by by turning it. So I'm going to turn it using this thing called cord flip.
32:43
Yes, much better. OK, I want to make sure I get through the rest of this stuff, but just know that, like, what I would also do was change, like the Axis titles and stuff like that make that a little more readable. But now you get the sense of what's possible here. And and you can say, like you can see the San Diego Unified many more times the schools than the next biggest, which is Chula Vista. OK, we've imported the data set. We've prepared it, done a little bit of visualization.
33:09
We pick out our method. Now, let's let's answer the question. We are going to use a function called summarise.
33:19
This.
33:23
And we're going to find three things here, the average number of schools, I should do two things in the interest of time so we can call this new column average avg_schools and we use this function called mean to get the average and we want the average of that n column. And then it's also get the standard deviation. So I'm going to make a new column called Standard_dev
33:46
We're going to use the function sd also on n.
33:51
And what this will do is it'll collapse this data set down to one row that gives you the average and the standard deviation of each of the whole dataset.
34:02
There you go, average number of schools in San Diego, area per district, about 16, standard deviation about 32. All right.
34:16
OK, so finally, how do we talk about this and how do we report it? You know, you could say something like districts in San Diego have about 16 schools in them on average.
34:28
Important to say here that we should interpret this with caution when we prepare the materials for this training. The reason we're putting out, because there's a lot of variation in the school count. So when you present it, you might show something like this and say, like you can see here, San Diego Unified has many, many more than Chula Vista, which we already know is big. If we bought the same amount of materials for both, somebody would go desperately wanting for materials.
34:52
If for the more statistically savvy audience, if you're presenting to folks who understand stats, you might give them a sense of the magnitude of that difference by saying something like the difference between San Diego Unified and the next biggest in the average is about seven times the standard deviation. So a lot of standard deviation is different. And so that has implications for how we plan our work. Right. We can't just buy a uniform number and then get it out to people.
35:22
That's that's not that's not going to work. OK, those are all the steps I'm going to talk a little bit more about why we do this code I'm going to switch back here and then we'll wrap up and and take some questions.
35:39
OK.
35:45
All right, so right about now, the question you should be asking is, Ryan, you're just like, why can't you just get a calculator out and just compute the the average and the standard deviation? Like you should be asking that question because that makes a ton of sense. You should use the tools that make sense for the thing you're trying to do, something like what Shannon Coulter did in his in his talk about pivot tables, like all that would be much easier to sort of get to what we got to.
36:08
I use an oversimplified sort of example here just to kind of help you imagine the way this stuff is used. But here's where the value of code comes in. Many sort of great things come out of that. Right. But like the one I'm going to focus on is scale. And the reason why code gets you scale is because if you imagine data analysis is like a delicious sort of container of cookies that you want to give to a friend, Excel is sort of like giving the cookies, right?
36:37
You give them the spreadsheets like a final product. When you have written in code, you are doing two things. You can give them the final product, but you're also going to give them the recipe. Right? So now they can take that recipe and they can say, I love the cookies that Ryan made. Not a big fan of chocolate. I am a big fan of peanut butter. I'm going to go into this recipe now. It can make my own. I just got to swap the chocolate out for the peanut butter. So you sort of now,
36:59
like with that recipe, you can do so much more. Some examples are you could use the same code that we wrote, but you can apply it to like 10 years worth of data each separately. Right. And you could look, has that average and standard deviation change over time? Second example, you could use the same code, but instead of doing on San Diego County like we did, you could do the entire state or you can do every county in the state and compare.
37:26
Last one is you can use the same code, but you can take your R Markdown file and you can generate custom reports for each of the counties. Right. So at this point, if you can imagine trying to do these three tasks with Excel, much, much more difficult to do.
37:42
So we're the front end takes a little bit longer to go write the code, especially if you're just learning. But the payoff is the scale. OK, want to end with a really quick story in 2012. It was just dipping my toe into data science courses and I've never done this before, so the way that would look. So I would come home from work and as soon as the kids went to bed, the laptop would open up and I would start struggling to make to learn and make my code work.
38:06
And a few weeks pass and slowly but surely, I started to see more and more code beginning to work, which is very exciting. But it's when I realized I could use this new skill at work. And so I set out to do my first data analysis using R which is quantifying the relationship between the amount of time that students in special education spent in the classroom and their performance on standardized testing. I did all the things right, all the stuff that we talked about today.
38:30
I prepared the data, I created the report and it had everything in it. the code, the text, the model and my conclusions. So I called my friend who works down the hall to show her this new thing, which I thought was my grand opus. And I put my work up on the big monitor in the conference room and I showed it to her. And I expected her to be inspired by all the possibilities that data science could bring. And then when she was done looking at it instead, what she said is she looked at me
38:58
and she goes, Dude, what is this? And so I realized in that moment that I never bothered to check if the question that I was answering was a question that mattered to anyone else. And so here's what I learned from that. If I only solve the problems that are interesting to me, then all I'm doing is I'm just bringing my fun hobby to work. And on the other hand, if we can solve problems that matter to others and we're truly being of service to students,
39:24
and that's what I hope we all sort of can do together as the analytic community.
39:31
OK, that's all the content that I've got today. Really quick to wrap up. I promise you, I get you all the materials. If you go to my link tree, which is linktr.ee/ry_estrellado or you can use the QR code. Everything is there. Your presentation slides, the references that I used and that inspired the talk. Some additional resources, all of which are free. You can get the markdown page if you're an advanced user and you're on GitHub, the entire code data,
40:01
everything is there so that you can reproduce the demonstration exactly as I did today. So go to linktr.ee/ry_estrellado all of that stuff is there. You can follow our project and be on social media at e_d_and_d be on Twitter and Instagram. That's our Equity, Disproportionality and design project. And of course, I am ry_estrellado on Twitter and Instagram. Also, I share a lot about writing data science and education. If you tweet and tag me, I'll share so that we can get our community going.
40:37
OK, that's all that I have. Thank you so much for having me here. This is so much fun. I'm so excited to be able to hear more about about what you'll do with this.
40:48
Carrie, I'm going to turn it over to you for questions. All right, you had quite you had some questions and quite a few comments to those questions. So I'll ask you the questions. And then if it works, I will read the comments. And if you want to chime in on your own comments for them to, does that work for us, however, is great. OK, so the first question from Mark is, is it typical to write the code directly in a markdown file? So there typically isn't a separate source file
41:17
and there were some comments to this. I personally write the code directly in the R Markdown file. It has a handy run previous code and run this code chunk in the top right corner of each code that I use all the time. Love the chunks feature to and we had another. I use the typical R script instead of our mark down when my goal is to output is csv file I use are marked down for everything else, sharing tables, graphs, writing text, etc.. The next comment was I do something
41:45
similar to Morgan in general. I start in an R script and move it over to R and D once I have explored it more and know how I am sharing it out with others. And then the last comment about that was I just explore and R and D the steps of moving it over drives me crazy. It's just a personal preference. So now I will turn that question over to you or you can I just say like I love that,
42:08
that all this community is happening in the chat and I always regret, like not, you know, not being able to to participate, but I love this. I mean, this is exactly, I think, part of the beauty of this kind of work, because it just kind of depends on like what what you're trying to do with these tools.
42:24
The advantage of of an R script, so not the markdown, but the R script is you can you can call it from within an R Markdown. Right. And so why that's useful is sometimes I've had some analysis where I have to do a very complicated hundreds of lines of code to prepare a data set. And I don't want that in my R Markdown because it clutters it. But also I want to be able to repair that code. If there's a problem, like in a few months and it's easier, just keep it separate.
42:53
So a lot of times I sort of use an R script just just for that. And you can sort of call an R script from within an hour mark down. On the other hand, you can't call another R Markdown file from within an hour markdown. And so those are I agree with all these sort of perspectives here. But in addition to that, that might be an instance where it makes more sense to have an R script as a separate thing. Perfect. Thank you. And then the next question was silly question.
43:19
How do I get the assigned arrow to look like an arrow instead of a carrot dash? And I do see a comment. I believe it's to that one, but it might be to something else. Without the tidyverse library, you can run the same function are by running dot, dot, dot, head, parentheses and comma. Ten. Oops, not for sorting lol All right. Yeah, that's awesome. Yeah. So I think the head and ten was when I was having trouble getting the sort correct on that one. Coach, thank you for the literature.
43:52
Yeah, it's exactly what David said there. I use this monist based font in our studio called Fira Code. I think, I think it's a David correct me if I'm wrong, but I think it's an open source. Yeah, it's open source. Yeah. Yeah. So that's what I use. And I just like the way it looks. It's just preference. All right, and I think that's actually it for the questions and just a whole lot of super well dones and great presentation. And if you don't have anything else to add,
44:20
definitely want to thank you for a wonderful presentation. And I want to thank all the attendees. So I'll just, like, turn over to you for a second if you had anything else to add, and then we'll conclude. I don't have anything else other than thank you again for the opportunity, because it's so much fun. OK, so just for everyone else to know, the next session in this data fundamentals track is going to be integrating data analytics with practice, instructional leadership and services.
44:50
And so you're invited to stay for this track or you can visit (blank) and jump onto another session. Thank you again to Ryan for this wonderfully informative session. You're welcome. Thanks, everybody.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat