Skip to main content
Loading Explore guides…
Epidemiology vs Medicine · Acedra
Home
Explore
Medicine
Epidemiology vs Medicine
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
176 segments
Copy
Hide video
0:08
- Welcome everybody. I'm Janet Rich-Edwards. I am one of the two science faculty advisers for the Radcliffe program. John is the other. I cover the life sciences. And I'm an epidemiologist. So for me to be able to have this day is really a thrill. It also marks the beginning of sort of a year of epidemiology and epidemics work that we're going to be focusing on. So epidemiology we usually define as the study of the incidence, prevalence, distribution, and control of disease in a population.
0:46
Now you contrast that with medicine which is about detection, prevention, and cure of disease in individuals. And so we consider epidemiology to be the science, the investigative branch of population health. And when we think about various levels of biomedical research, you think of-- generally, we talk about three, maybe four levels. You talk about your basic or pre-clinical science, which you think of as bench research, usually experiments, usually with animals or cells or cell lines.
1:24
Then you think about the level that we call clinical research, which very often involves recruiting patients to a study. That study might be observational in nature, watching the course of disease in a group of patients, or it might be experimental, think of randomized clinical trials for drugs as clinical research. Then we get to the level of population research, or epidemiology. And here we're usually talking about very large collections of data about individuals within a population.
1:57
Sometimes we go out and collect these data by forming what we call cohorts, an observational cohort. So you may have heard of, say, the Nurses' Health Study, which has been following 121,000 nurses with questionnaires every two years since 1976. Or you might be talking about secondary data analysis. You might be analyzing data that's collected for other purposes, say vital statistics registries. You might, for example, go to Norway where they've got fantastic ability to link all the records
2:28
through sort of a citizen ID. And you might be able to, say, link a woman's records of her births to the records of her hospitalization or even mortality and make connections statistically by using these secondary data sources. And there are many different methods of epidemiology and you'll hear about many of them today. One of the tricks about epidemiology and the fact that it's largely observational is that we very rarely can perform experiments.
3:00
And when you think about what is the gold standard of biomedical proof, you're really thinking about experiments or randomized clinical trials, usually placebo controlled, usually blinded to the investigator and the participant and the clinicians involved. Epidemiology can rarely have that kind of experiment. And so we're challenged to take questions-- to distinguish cause from correlation. And to give you a really basic paradigm-- you know, if you think about--
3:32
you could do a study looking at causes of lung cancer. And you could go out in a population and survey them about their cigarette smoking habits and ask them about the matches, whether they carry matches or not. And if you aren't really careful about your inference, you might conclude that carrying matches in your pocket causes lung cancer, right? Now, we all know that's a correlation. Carrying matches is correlated with smoking cigarettes, which is the actual cause of lung cancer.
4:05
That seems so patently obvious, painfully obvious, if you will. But in epidemiology we're often dealing with very closely woven together potential causes of disease that will also be correlated with non-causes of disease. And our challenge as scientists is to tease this apart. Because where we would all like to go is to translation where you take the science of epidemiology and you translate that into effective public policy, effective new possibly clinical protocols.
4:40
But you want it to be based on a pretty reasonable inference of cause. And one of the reasons we chose the four scientists you'll hear about today is because a lot of them focus on how can we get better at distinguishing cause from correlation. So I think you'll be hearing quite a bit about that today.
5:03
So the four young scientists we have here I'm really thrilled about. And each of them will talk for about a half an hour and we're going to group them into two. The first two are going to be more, what you might be familiar with, focused on infectious disease epidemiology. Our roots as a field, when you think the word epidemics, you think of infectious disease, you think of cholera outbreaks in London in the 19th century. We have since evolved. And epidemics has now writ much larger.
5:38
So we have, consider, we have epidemics of heart disease. We certainly have epidemics of obesity. We have epidemics of mental health. And so the second half of the day is focused on some of these other noninfectious disease epidemics, and particularly epidemics in mental health issues and how they're distributed throughout the population in terms of health disparities. So to kick you off-- so what we'll do is we'll have two speakers speak back-to-back
6:08
and then we'll do 20 minutes of questions after that. And then give you guys break and then we'll go to the second half of the program. So I will introduce to you our first two speakers. So Neal Goldstein is an infectious disease epidemiologist at Christiana Health Care in Newark, Delaware, and also hold a faculty appointment at Drexel University. His specialty is using secondary data sources, particularly medical records. And his research spans several disciplines,
6:38
including vaccine preventable diseases, sexual minority health, pediatric infectious disease, and women's health around pregnancy. Most recently, he's focused on this issue of translational epidemiology. So how do we move from knowledge generation to application and advocacy. And his talk today is going to be about infection in hospitals, whether we can be moving from a model that's based on the likelihood of disease being a function of the patient, the characteristics
7:09
of the patient, to really change that and look at functions of the system.
7:16
Our second talk will be Alana Brennan who is an instructor in the Department of Global Health and in epidemiology at Boston University. Her main focus is to apply bio stats and epidemiological observation to cohorts in sub-Saharan Africa, particularly around HIV and antiretroviral therapies. In particular, her work has been related to changing South African National treatment guidelines, early versus deferred pediatric anti-- it's so hard to say-- antiretroviral treatment,
7:51
and as well as the economic outcomes of patients receiving that kind of treatment. She also focuses on when a clinical trial is not feasible, right? Often for logistical or often for ethical reasons. How can we do quasi-experiments which are research designs in which investigators study quasi-random variation exposures that occur naturally. And she'll talk a bit about that. So with that introduction, Neal and Alana, actually, why don't you both come on up. And Neal, you can take it away.
8:23
[APPLAUSE]
8:28
- Thank you for that very nice introduction. It's really an honor to be here today. So I want to-- before I get into the talk itself, I just want to give a little bit of background as to why I chose this particular area. As was stated, I did my training in infectious disease epidemiology. And when I got to doing my post-doc at this hospital, it was a community hospital. It's a large community hospital, but it still very much focused on patient care. So the research that was ongoing there
8:59
was all about what are the characteristics of the patients and what leads them to become infected with some disease. So I wanted to bring more of my traditional training as an infectious disease epidemiologist into the hospital environment. And specifically, I worked in the neonatal intensive care unit. So I'm going to tell you everything I know about neonatology. It's only in two slides, though. And to start we need to focus on why do the patients get sick in the environment.
9:32
Well, one way is we can just look at the characteristics during pregnancy. So did the mother have some kind of markers of inflammation like chorioamnionitis? Was she group b strep carrier? And then these can lead to vertical transmission, so from mom to the baby. There's also, though, the characteristics of the babies themselves, in that maybe they were born pre-term. So this would mean they would have an underdeveloped immune system, underdeveloped skin system.
10:01
So these are what I would call these intrinsic factors. And these are definitely correlates of infection, in that if you are pre-term, if you had a mom that had chorio you are likely to become colonized and infected. So the questions I wanted to ask were all about what are other factors that can lead to infections. So you can-- certainly, it can be caused by your care team, your health care workers. If they are colonized or infected with some organism, they can very easily pass it to you.
10:32
It also may be a function, though, of the environment itself, or even what we call fomites, which are just kind of inanimate objects that harbor organisms so this could be a medical device or it could even be as simple as your cell phone. So when you deal with when you go through an epidemiology program, especially when you're in a communicable disease mode, you will talk about this notion of the epidemiologic triad. The idea is that if you can block one of these pathways,
11:01
you can potentially stop transmission of some organism. So it's funny that cholera was mentioned already, because I'm going to use cholera as the example here. The host would be people, it would be us. The agent would be the cholera bacteria. And then the environment could be where the cholera resides. So the idea is that if we can block these pathways, we can stop cholera transmission. For example, between the cholera bacteria itself and the host
11:29
you can make the host immune by, say, giving them vaccinations. You've blocked that pathway you can also block the pathway between the agent and the environment, say, by coordinating the water supply. And lastly, you can block the environment host pathway by, say, making sure that people have access to clean, potable water. So we're blocking these pathways and we're potentially stopping cholera from infecting the host. There's also this notion of a vector that kind of operates in
11:58
between these three areas. The easiest example of that would be Zika virus or malaria. The vector in this case is a mosquito. And that it functions, it operates kind of between all these pathways here. So as we then switch from this classical, you know, you're in school, you're getting trained in epidemiology to now how can we apply this at the hospital, we can start substituting things on here. So the host where I worked it became the neonate. The agent that I'll talk about are really
12:28
health-- mainly associated with health care acquired infections. So I'm going to talk about methicillin-resistant staph aureus, M-R-S-A or MRSA. Going to talk about something called sepsis, which is a bacteria in the blood. We'll talk about a variety of organisms that can lead to that. But I'm also going to talk about colonization of a few other organisms, pseudomonas and klebsiella. So those are our pathogens. The environment now becomes the neonatal intensive care unit.
12:59
The intermediary or the vector is the health care worker or visitor in that they are within this environment and they will interact with the babies as well as potentially the pathogenic organisms. Three research questions that we were looking at for this particular group of analyzes. The first is looking outside of what are the intrinsic factors, so what are the characteristics of the baby, we wanted to start incorporating these extrinsic factors. Like, how does occupancy lead to a greater
13:30
risk for becoming septic? And I'll get into what that means in a few slides. But also, how does the patient care network or who the providers are interacting with, how does that influence possible spread of MRSA. And lastly, if we can identify maybe some of these other factors, what is it about the environment itself, what is it about the NICU, the bed space, the location of the patient, that may have some correlation to becoming colonised or infected.
14:02
And I just want to-- I have one slide here that's kind of my methods-y slide. Because I want to just give you a sense for how we're able to do this type of study. It's not as if we are prospectively recruiting parents to recruit their children into this study. These data already exist, which is really-- that's kind of one of my wheelhouses is I like doing secondary analysis of data that's already in the electronic medical record. I feel like we have so much information
14:27
that already exists that was delivered during routine patient care, we can mine that and then ask some of these interesting questions. So what we have in the electronic medical record, it can be thought of as a cohort of neonates in that they are admitted to the neonatal intensive care unit, and while they're in the NICU we know what happens to them. We can follow them through time and see did they develop one of these outcomes of interest or did they have some other--
14:55
did they not have the outcome. And that would be were they discharged from the unit, were they transfer somewhere else, or did they expire while they were in the unit. So these three research questions, now, we'll go through them one at a time. I'm putting the epidemiologic triad back up here on the slide so you can get a sense for which are the pathways-- and I'm not sure if it shows up that well, but they're highlighted in red. So the intrinsic factors that we would traditionally focus on
15:22
are really what's happening between the pathogen and the neonate. But this talk is all about these extrinsic factors. So that's these other pathways that we're beginning to look at. So the first question that we're tackling is this notion of occupancy. And the reason that this question came about is because it's almost in some way an easy way to explain if there's an outbreak of, say, MRSA in the unit we could just be like, oh, it was really crowded at that time.
15:51
You know, maybe that's why there's some breakdown in hygiene going on. Maybe that's what's going on here. So we wanted to say, all right, is it really related to occupancy and what does one actually mean by occupancy, because it can be defined a whole host of ways. For example, it could just be the number of babies in the unit at any one time, and this is what we would call this census. Or could be something to do with the demands on the staff in the unit.
16:19
And a proxy for this would be what's the overall acuity, say, what is the proportion of pre-term babies in the NICU or what's the proportion of very low birth weight babies in NICU. Or it could be a function of, say, just staffing levels. What's the nurse to patient ratio. And one thing I want to emphasize with this is that it's not occupancy itself that leads to these outcomes of being infected or colonized, it's this notion of being a proxy for something else.
16:48
Because maybe it's a breakdown in sterile technique to access lines that's happening as a result of increased demands on the unit. Well, getting right into the results for this one. We looked at two measures, here. We looked at the notion of just the number of occupied beds or the census, we looked at the acuity. And what we saw was that there was a modest risk increase in late onset sepsis that was statistically associated with acuity, not so much for census.
17:19
And although this is, again, a very modest effect, the way that I initially did this analysis was it's kind of like for each percentage point increase in acuity in the unit-- so what I would stress for this one is that if we're looking at, say, the proportion of very low birth weight babies and we were comparing a unit that had like 10% very low birth weight babies to a unit that had 40% very low birth weight babies, now all of a sudden we're talking about a 60% association of late onset sepsis.
17:49
So it becomes a little bit more important of a predictor. Conclusions from this question-- we saw that in times of high unit acuity, we may need some additional infection control precautions. As well as if you're doing this type of work where you're looking at health care associated infections, it's probably a good idea to have some measure of occupancy in your analysis. All right, so switching to what I think are the more interesting ones, here. The second research question is how
18:19
does the patient care network of a NICU influence transmission of a MRSA. Again, the triad's up there. You can see highlighted in red which pathways we're operating on. And the outcome I'm interested for this study is MRSA. And MRSA is a not uncommon health care associated infection and the consequences are particularly dire in neonates because about a third of the ones that become colonized with MRSA will go on to develop invasive infection. So it's important pathogen for us
18:52
to target in the hospital environment. And what we know is that about 20% to 40% of a patient's flora are the result of cross-contamination via the health worker. They're this vector, they're this intermediary that's going between patients. And one of the main correlates of this transmission is proper hand hygiene. In fact, hand hygiene is in itself composed of a couple of different things. There could be the notion of compliance. So that just how many of your staff
19:23
are doing hand hygiene, whether it's the gel hand sanitizer, whether it's using sinks. But then you also have this idea that you could be compliant, but are you doing it correctly. And that's this notion of efficacy. Because it's comprised of these two things I don't think you could ever get 100% effectiveness. Because even if everyone in your unit is doing it, we don't know that they are doing it correctly.
19:53
So to do this kind of work we wanted to know how's the notion of which providers that interact with a baby connect infants together and then how could this, perhaps, influence the spread of MRSA. So we needed to ascertain what I was calling the patient care network. The idea is that I can go into the electronic health record and I can see which providers interacted with which babies. And then I can begin to link providers and babies together by saying they took care of this baby
20:25
and then they went on to take care of this baby and this baby. So you can develop these grafts and we'll see one in a few minutes. But you have this network that forms. And it's akin to something you may have heard something called a social network analysis and it's all about your social networks are and how that can spread ideas. It's a similar sort of idea, here.
20:49
To give you a sense, though, for what the type of interactions we're looking at in the health record, as you can imagine, when you're in an ICU environment, that's the most intensive care that you would have in the hospital. So there's all sorts of interactions that are happening. Some of those interactions are kind of meaningless for our work here in that they're just patient documentation sort of things, those really don't contain any risk of spreading a disease.
21:13
Other ones, though, they are more meaningful in that they have the propensity to spread MRSA here. So we ranked the ones in the medical record and I thought that it would really be the more invasive procedures are the ones that we would see here. It would be the chest tube placement, central line, that kind of stuff. But in fact, the majority of interactions that we see in our NICU are just really routine things like taking vital signs, since this is a population of babies,
21:42
doing like a stool assessment. Nevertheless, this still has a propensity to spread an organism from one baby to another. And so this picture up here that looks like a bird's nest, this is actually our NICU on a given day. So what you have is each red dot is a baby. And then the black lines are connections between babies by health care providers in common. When I first saw this, you know, it suggests that, man, there really is a lot going on a given day in the unit.
22:16
But each one of these lines is a potential for hand hygiene to break down, whether it's not being compliant or it's not efficacious because it wasn't done correctly. So we conducted this simulation study here to model what this NICU network looks like over a typical nine day admission. That that's the median length of stay in our NICU. So I'm going to switch over, see if I can play a video. All right, so what we're looking at here is each dot represents a baby in the NICU.
22:49
The blue dots are babies that are susceptible to MRSA colonization, the red dots are babies that are already colonized with MRSA. The lines that you're seeing drawing and as they update, these are interactions that providers are having that link babies together. It's updated every hour. So the idea is that you have these one hour care bundles. I look at everything that's happened with a given provider for one hour and all the babies they've interacted with.
23:16
And then I begin to connect these babies together. So now we can do this simulation model where we follow over time these interactions. And what I do here is I allow for there to be the possibility of MRSA transmission that's considered on, say, hand hygiene effectiveness or some characteristic of the baby. I don't know if you noticed, but we started with two red dots up there. That meant there were two MRSA-colonized babies. And here we are 80 hours or so into this NICU
23:51
and we're already at four colonized babies here. So if I let this go through the whole way, you end up with about five MRSA-colonized infants. So it clearly demonstrates that there is a lot of opportunity for MRSA transmission to occur. So this suggests to us that there's a lot of opportunities for MRSA transmission to occur. And now that we've had the environment constructed where we can mathematically model it, we can begin asking questions such as when we vary
24:18
the amount of MRSA that's in the unit at any given time, how does that affect the spread of it, we can vary the hand hygiene compliance, we can very care attributes of the baby. So we know that infants that are born pre-term or infants whose trachea is intubated require additional care. And on the basis of this, we came up with these graphs here the y-axis we're looking at the post-simulation, so that means after nine days in our simulated NICU,
24:46
we're looking at the prevalence of MRSA colonization. And then the group along the bottom, these four groups that I have here, one two three four, this is just kind of seeding our NICU with starting prevalence of MRSA, so this would be one baby out of the total amount, or about 2% of the babies have MRSA, and so on. The gray bars then represent the varying amount of hand hygiene effectiveness. The two extremes, the white bar and then the black bar, white bar would be no hand hygiene at all.
25:17
So this is just assuming that no one follows any hand hygiene at all, it gives us a usable comparison. The black bar represents 100% where everyone's doing it and everyone's doing it correctly. So we acknowledge that reality exists somewhere in between here. When I first started with this analysis I had a bet going with our ID physician. I thought there would be a plateauing effect. I really thought there would be this point of no return, where we just didn't see any improvement.
25:46
And he said that, no, I think you are going to see a linear trend. You're going to keep getting bang for the buck. I think in some way, the jury's still out on that one. I kind of see that-- I agree with him that there is this linear, you keep getting return for your money here. But there does seem to be a little bit of plateauing that occurs, in my opinion, towards the bottom here. But if we focus just on this one over here, on the one that has 2% starting prevalence,
26:12
I don't have the numbers in front of me. I just wanted to talk about the overall picture here. But the number-- after you follow this NICU, at 2% prevalence, even with 88% hand hygiene effectiveness, which I think is kind of a theoretic maximum, that still suggests that post simulation you have about two, and in fact, a little bit more than two babies colonized with MRSA. In my opinion, then, that really suggests that hand hygiene alone is not enough. There's never a good enough level
26:42
of hand hygiene in the hospital environment. So that's one of our conclusions from this work is that there's no good enough level. And the conclusion number two in some sense that may be a bit obvious or intuitive. The greatest risk of MRSA colonization was for those who have the most contacts. The neat thing about this work is we're actually able to quantify what that is. And then to point number three on here, in times of a high number of initially colonized babies
27:10
with MRSA, it really warrants other interventions in then NICU. And these would be things like we have isolation rooms in our NICU, so placing the babies in the isolation rooms. This is also something known as cohorting. Or when you have multiple babies with MRSA we like to group them together in a certain area of the NICU. It could also just be things like, should we consider decolonizing them, should we do universal gloving in addition to hand hygiene.
27:40
It's all food for thought for our infection control department. So the final research question here is, what are the characteristics of the NICU itself? We know that occupancy seems to matter, we know that hand hygiene is important. But what is it about the environment of the NICU that may lead babies to becoming colonized and infected with various organisms? All right, so a brief recap of where we've been. We saw-- and I just mentioned this-- that acuity of the unit
28:11
seemed to be an important predictor. The amount of care required seemed to be an important predictor. And that's a picture of our NICU up there. So we want to know, what is it about the NICU itself? In fact, this is nothing new. And I'm returning to cholera because it's such an instructional example. How many people have heard of John Snow? And not the Game of Thrones Jon Snow, but the-- All right, so it's about half, maybe a little bit more than half. In some sense, he's one of the originators
28:46
of this field of epidemiology. And you have to put yourself-- now for this picture to make sense, you have to put yourself back into mid 1800s London, there's a cholera outbreak. And what John Snow did that was so innovative at the time, and, again, may seem kind of obvious to us today, was he just put the cases on a map of London. And those are these dark rectangles up here, the black rectangles. So he's putting these places on a map and notices that they really appear to be centered
29:16
around one of these wells in London on Broad Street. And lo and behold, when they turned off access, when they locked the well so people can no longer draw their water from it, this particular outbreak subsided. This is what we would call today in the field spatial epidemiology. And also is something we often say in the field, in public health in particular, that place matters. So I wanted to know, does place matter in our NICU? We're looking at a bunch of outcomes now,
29:49
we're looking at MRSA but we're also looking at sepsis again, as well as being colonized among those babies that are being ventilated, are they colonized with certain organisms that have the potential to cause invasive infection. And we're going to consider a few different characteristics of our unit. Our NICU is what's known as an open pod design. So you have these pods and within a given pod you can have up to three babies, occasionally four babies that share this space
30:16
depending on the demands of the baby. But some of the areas in the NICU you can only fit-- some of the pods you can only get a single incubator in there, single bed. Other ones you can get up to four. So does that seem to matter? If you're in an area with there only being a single bed, are you less likely, say, to become colonized? As well as-- we're looking at historic data here from our electronic medical record. So at one point in our unit we had a leaking ceiling tile
30:44
and it turned out that the water supply was contaminated with pseudomonas and it was leaking near one of the pods. So we said, well, can we use this as a marker, then, to see did more kids that were in the pod during this period of time get pseudomonas. Also at one point, and this was really bizarre to me, but this predated when I was at Christiana. In one of the rooms-- the NICU is divided up in these three rooms. One of the rooms had carpet. It's like, why would you guys have carpet in an ICU?
31:16
Because you have environmental services that comes through, right, and they're vacuuming it and it's kicking up all sorts of dust and particles and organisms, as well, into the environment. But we're able to look at that back in time and say, all right, if you were in the room when it had carpet, did that cause an increased risk of colonization or infection? And, as well, we threw in a couple of markers of fomites. There is equipment-- there's spaces
31:39
in the NICU where you can fit more equipment for babies that require more care. There's also refrigerators that store breast milk and medicine sprinkled throughout the unit. And one of the neonatologists was asking me, you know, that she thought it was related to the refrigerators. She thought, I think those refrigerators have a contaminated-- some part of them is contaminated. Can we somehow model to see if being near a refrigerator leads to an increased risk for infection?
32:10
And one final thing I wanted to consider here is that, since we're doing this spatial analysis, say I'm in a pod and a baby in a neighboring pod becomes septic. They have some kind of bacteremia. Does that then influence my risk? Does proximity to other babies influence a given baby's risk?
32:37
So to do this work first thing we needed was a spatial representation of our NICU. Now it's not like if I was doing work in Boston, it's pretty easy to find a map of Boston to them plot cases of whatever. But there wasn't this map of the NICU so we had to create a map of the NICU. This is what it looks like. It's a u-shaped NICU. And one of the easiest things I can do is create what's known as a choropleth map. And you see these all the time, right?
33:03
If you look at like diabetes, obesity, heart disease in the US, you see these colored maps. And so it's very nice to get a quick qualitative sense as to what's going on. So I picked one of our outcomes here. The way that you would read this map is the darker the shade of red, the more numbers of these outcomes that occurred within one of these pods. So I want to do-- this is the point in the talk where it becomes a little interactive. I want to do a quick survey. There's two options here.
33:31
One is that you think, just by looking at this, it looks like it's just randomness. The other option is that you think there are patterns to what's going on here. In other words, there's some spatial correlation going on. All right, so show of hands, then. Who thinks this is just randomness? This is random noise, what we're looking at up here. One take? All right, I've got a couple of takers. Looks like two or three. And then raise your hand if you think that there's clear spatial--
34:01
like, there's patterns, here, there's hot spots. So the majority think there are patterns. I fell into the camp where I, when I first saw this, was like, there's clearly patterns going on. If you look at the bottom area here, there's definitely something going on here. There's definitely something going on here. So we're going to return to that question in a minute.
34:24
For this final research question, we had three sub questions that we wanted to investigate. First, was it random or are there patterns, are there hot spots? Second, what about the environment itself? Does this explain our infections and colonizations? As well as does, you know, if a neighboring baby becomes sick, does that influence my particular risk? So to the first question, was the distribution random? We do some fancy statistics and what we end up with is that for two of our outcomes, MRSA
34:59
and sepsis, there were patterns to it. It wasn't just randomness. For the other two of our outcomes, though, it statistically appeared to be random. So I got bad news in that the outcome we were looking at, which was klebsiella, in fact was statistically random. So that just goes to show you can't always-- a qualitative interpretation sometimes doesn't lead you down the road to the correct conclusion. All right, now to the environment itself. On an earlier slide I brought up the list of factors
35:35
that we looked at. So this was things like does being in a refrigerator matter? Does being in what we call the carpeted quiet room, did that matter? So we looked at all these factors, right? And what we found was that none of them really mattered all that much, except for pseudomonas. So now we're dealing with being colonized with this bacteria in your endotracheal tube, this is among things that were intubated. And we saw that for them, being in an area that
36:05
required more medical equipment seemed to suggest like a nearly two-fold or two and a 1/2 forward increase odds of being colonized with this organism. So that was of interest. And these models that we're doing right here, they're controlled for all of these characteristics of the babies themselves. So we're accounting for the fact that some of these babies are going to have been born very pre-term, have very low birth weight, may have had a central line present.
36:36
As well as how long that they're intubated for, right? Because it stands to reason that the longer you're intubated, the more likely your endotracheal tube will pick up some kind of bug, acknowledging that all the tubes start out sterile. And finally this notion of, does a neighbor influence your risk? Here, again, doing some fancy statistics. But what we ultimately find is that MRSA colonization appear to be the only one where if there was a neighbor that
37:07
was MRSA-colonized, that I was at increased risk for perhaps being MRSA-colonized. When I first saw this result I got very excited by it. But then I thought that-- well, this is actually probably just returning something that's already existing in our unit. In that MRSA-colonized babies are already being placed in the isolation rooms and they're already being cohorted to certain areas in the NICU, so it's probably actually just returning that. It's just telling me that, yeah, there's
37:34
this spatial auto-correlation but you're the one causing it. It's not that MRSA is more communicable than other organisms. So something I thought was going to be more interesting, maybe not. We still have to talk about that with the infection prevention group. The conclusions, though, from this final research question here. What we saw was that the intrinsic baby-specific factors-- so, again, these are things like your birth weight, your gestation, did you have a central line--
38:01
those appear to explain most of the outcomes. But we still see statistical evidence that there's extrinsic factors at play here. We identified maybe a few of them. I still think there's more to be done here, that there's more factors we just haven't uncovered. Because with this type of study, where you're working with data that's already existing in the medical record, you kind of can only test for what you have data on. So if we were designing a prospective study from this,
38:32
we could really investigate a lot more. Maybe there's other factors out there we just haven't uncovered. But definitely with MRSA colonization, you know, it warrants isolation and cohorting given the potential for transmission. When I give this talk at our hospital I like to conclude with, so what should we do as the NICU? Like, what should our infection prevention group do? But I'm guessing that none of you are with the Christiana Care NICU and if you are,
39:04
see me in advance because that's a tremendous coincidence that you're here right now. So I thought that would be nice to end it with what can we do as potential patients in the health care system? Really one of the easiest things to do is the notion of hand hygiene, right? It's making sure that your providers have done hand hygiene when you're seeing with them. You're not going to insult them by they come into the room and you don't see them scrub down, you're not going to insult them by saying,
39:31
can you please use a gel sanitizer while you're here. As well as you should do that yourself when you're in a hospital environment. They're gross places to be because there's organisms all over. No matter how vigilant environmental services can be in the hospital, there's still more organisms there than you would just encounter walking around elsewhere. But that's another thing that you can do in addition to hand hygiene, ensuring that your staff have hand hygiene,
39:57
they follow those practices, making sure the environment is clean. Like, you should be able to look around and it should be clean where you are in there. If it's not clean, talk to someone to get environmental services in there to go ahead and clean the area. Well, there's other things that we can do as well. Like ensuring that we prescribe antibiotics correctly. Because kind of what are the implications behind this work is that if you are over-prescribing an antibiotic,
40:25
you can develop what's known as resistance to it. And particularly with MRSA, right, that's already resistant to the methicillin class of antibiotics. So we see that by being more vigilant with antibiotic prescribing, you can potentially reduce the burden that's caused by these. So that would just be, you know, if you feel like you have the cold, there's no need for an antibiotic, right? I think most people know that. But if you're a senior provider and they give you
40:53
prescription for an antibiotic, know the reason why they're giving that antibiotic, make sure that it's the appropriate class for what you have. These are all good questions that they would be happy to talk to you with.
41:08
So in closing, clearly, work of this magnitude, I couldn't do without a whole host of people that are much smarter than myself, many of them are listed on this slide over here. And I also want to acknowledge the Society for Epidemiologic Research because they were kind enough to select a portion of this work for one of their awards. Thank you for listening. Here's my contact information. I'm happy to chat about any of this with you now or after the fact. [APPLAUSE]
41:44
- All right so I'm very excited to be here today. My name, again, is Alana Brennan and I'll be talking about the use of quasi-experimental designs for evaluating HIV care and treatment in sub-Saharan Africa and observational work. Specifically about two analyses I've done over the last four years. I'll give you a brief context of HIV in sub-Saharan Africa and then also South Africa. Talk about observational versus experimental studies. And then talk about the two specific quasi-experimental
42:12
studies that we did using propensity score matching and regression discontinuity to evaluate HIV care and treatment. And this is all secondary data analyses. So the HIV epidemic-- many people are probably very familiar of what's going on worldwide-- but in case you're not, there's roughly 37 million people who are HIV positive worldwide. And the vast majority of them are in sub-Saharan Africa where there's 26 million who are HIV positive.
42:39
And just to give you an idea of how treatment has rolled out in this setting, treatment wasn't available in resource-limited settings until about 2004. And this is based on a directive by the World Health Organization and guidelines that we're bringing antiretroviral therapy into government sector clinics. And so in 2004 treatment became available. And it was based on certain eligibility and guidelines because of resource and financial constraints.
43:05
Over time, in 2010, as better drugs became available and the eligibility criteria changed, more people became eligible for treatment. 2013 there was another increase in eligibility or expansion of eligibility criteria. And then currently we're at what we call universal test and treat or treat all. And that means that anybody who is HIV positive and comes into a clinic, they are eligible for treatment that day. And this, even though the policy was started in 2015,
43:32
most countries didn't roll it out until 2016. September of this last year. And universal test and treat is what it is, as far as I understand, what is protocol in high income countries since roughly 2004. And then what has happened also over this time period is we've gone from this huge pill burden of patients having to take two to three handfuls of drugs twice a day, two, three times a day, to what one is a fixed dose combination. So it's one drug, twice a day.
43:59
So it's actually improved patient care substantially. And patients are-- this is easy to do. Easier to do, to adhere to. And this graph is just showing where we're at in different regions throughout the world in regards to antiretroviral therapy coverage for those that are eligible for HIV. And I'm just highlighting here in the red box southern Africa and eastern Africa regions because those are the regions that the data is from that I'll talk about today.
44:24
And if we look at-- this is 2010 to 2015. And in 2010 in the southern African and east African region, we see that about 25% of people who are eligible for treatment actually got treatment for HIV. We see a substantial increase to 2015 where about 55% of the population that is eligible is actually accessing antiretroviral therapy. And although there's still a large unmet need, this jump is massive. A massive undertaking. So although there is about 50%, roughly, at least
44:55
in eastern and southern Africa, that are in need of treatment, this is not a small feat to get to this point. So briefly about observational studies and experimental studies. And Janet had talked about this. Most of the work that I do, until recently, is in the world of observational analytical research. And so our typical prospective cohort design is where you have a population of people, you choose a subsample, you classify them as exposed or unexposed.
45:23
And then you follow them forward over time to assess who develops disease or your desired outcome. And then you compare the risk or rates in the exposed to the unexposed to look at the magnitude of an association. And as Janet had mentioned, basically, we're only looking at what we believe to be correlations or associations causality is difficult to establish when you're in observational research. Where we can do that is in the experimental study design,
45:49
specifically the randomized clinical trial, which is our gold standard. So you have your population, your underlying population, you choose a subsample, and then you randomize that sample to either received treatment or control. And then again, follow them forward over time to assess your desired outcomes and to compare the risk or rates amongst the treated to the control to look at the actual causal effect. So if you're randomisation-- the key here is that randomisation.
46:14
If you're randomisation held, you can actually establish causality in a randomized clinical trial, which is what we're always trying to go for. So in my mentorship for the last five years with Professor Matthew Fox, he's always challenged us to think about when we're working observational research, how can we get as close to a clinical trial design as possible with secondary data analyses. Because we know in experimental studies we can balance confounders in measured and unmeasured
46:43
because of the randomization, if it holds, but that's not a luxury in observational research. So quasi-experimental designs, the two that I'll talk about today, can help us, if the assumptions are met and they're applied appropriately and the question is appropriate, help us get closer to that establishing causality. So the first analysis that I'll talk about is looking at outcomes of stable HIV positive patients that have been down-referred from a doctor-managed antiretroviral
47:11
clinic to nurse-managed primary health care clinic for monitoring and treatment of HIV in South Africa. And this is a prospective cohort study where we use propensity score matching to balance confounding between exposed and the unexposed. So we're trying to simulate clinical trial. So here we see that the context of this is in South Africa. And South Africa, just to give you a bit of background, is the largest HIV epidemic in the world. About seven million people are HIV positive, about 50%
47:40
of those are actually receiving care for HIV. So again, there's a huge unmet need, but that 50% is not a small feat, by any means. In the clinic that the data is from and where I've worked at for about four years a while back is Themba Lethu Clinic, which is associated with one of the largest secondary hospitals in Johannesburg. It's called Helen Joseph Hospital. And it's central Jo-berg. So the population is a lot of migrants back and forth, Malawi's, Zimbabwean,
48:13
but also a lot of South African nationals. And so what was going on in this time, roughly around 2007, is all of the model of care for HIV treatment in South Africa was doctor-initiated and then doctor-managed care. So they always felt that HIV care had to be managed by doctors. They were the only ones who could do it, nurses weren't going to be capable of doing that. But what they started to see was this overrun clinics, right? There were long lines, patients were leaving,
48:41
getting lost to follow up. It was just very difficult to access care in a timely manner. So what they started to do is de-centralize or down-refer patients who were stable to local primary health care clinics that were either within the vicinity of the actual main clinic, which is Themba Lethu, or a primary health care clinic that was closer to the patient's home. And so this began in 2007 where Themba Lethu began to down-refer their stable patients to primary health care clinics.
49:09
In the primary health care clinics the patients are managed by nurses, not by doctors, OK? So at the same time that this was going on my colleagues were competing what was called the CIPRA trial. It was a non-inferiority trial comparing nurse to doctor-managed outcomes in patients. And what they showed was nurses can manage patients in a clinical trial setting and have comparable outcomes to doctors. But what this doesn't answer is can nurses do this in an observational setting
49:38
where there's not close supervision or strict inclusion criteria of these patients. So this is where we went to the Themba Lethu data set. It's a massive database. It's a longitudinal prospective cohort and the data is collected in a database capturing system called Therapy Edge which was designed specifically for HIV care and treatment. And then at each clinic we collect demographic information, clinical information, drugs, and conditions. And currently there's about 35,000 patients
50:08
that are on treatment in the database. The study population for this analysis was patients who were adults and initiated onto a standard first line regimen at Themba Lethu Clinic after 2004. And they had to meet these eligibility criteria for being down-referred to a primary health care clinic. And so that was stable on treatment, they had to have no opportunistic infections, they had to have a higher CD4 count, be virally-suppressed. And then the clinicians, also, there
50:36
were some subjective measures there of whether the clinician felt they were good candidates for down-referral. And so in this population, we were limited once we-- the eligible patients, rather, there were about 5,000 eligible patients. And they were all eligible for down-referral. So they all met these criteria. But 1,579 were actually transferred to the primary health care clinic. So we'll consider them the exposed group. And the other remaining 3,421 were eligible but not transferred.
51:08
So we'll refer to them as the unexposed group. And we used logistic regression to model the propensity of receiving exposure. And I'll go through this in a little more detail. And then we matched patients exposed and unexposed based on their propensity score, OK? So we're trying to balance observed and unobserved confounders between the exposed and the unexposed. And then we conducted a survival analysis to look at our outcomes.
51:35
So when we're talking about modeling a patient's propensity score, we're looking at their probability of receiving exposure. And exposure in this case is being down-referred to the primary health care clinics. Because remember, they're all eligible for it. It's just those that are down-referred are the ones that are actually exposed. So we modeled that based on a set of observed confounders, which is gender, age, CD4 count, hemoglobin, body mass index, and other factors.
52:04
And so what we have is we have the entire population. The red circle is our population of patients that are down-referred to the primary health care clinic. The blue line is those that are not down-referred but are eligible. So we modeled the probability that the entire population would have the exposure of being transferred to the primary health care clinic. And then we matched those with that overlap of who had similar propensity scores. And then that matched sample, that overlapped sample,
52:33
is what we use for our analysis. And I'll break that down in a little more detail. But just a basic calculation of a propensity score. So this is a simple two-by-two table. On the left you have patients that have the actual confounder of interest. On the right you have those that don't have it. And then we have simple exposed and unexposed groups within both of those strata. You can ignore disease, d equals one, d equals zero. It doesn't even matter in this case
52:59
because we're just looking at exposure and confounders. So to model someone's propensity score, the probability of receiving-- probability of being exposed given that you have a confounder is simply 84 over the total in the strata. Which gives us a probability of 0.62 or 62%. The same thing goes for the confounder, or those without the confounder. So if we want to model the probability of exposure given that you don't have the confounder, is simply 177, the total exposed, divided
53:29
by the total in the strata. And that gives us a probability or propensity of 0.44, 44%. So you can think of in our analysis we have person A who's been down-referred, person B who is eligible for down-referral but not down-referred. So we have our exposed and our unexposed. And they both have a predicted probability or propensity of 85% of being down-referred. So they're identical on those observe confounders, in regards to how we created our propensity score, and their exposure just differs.
54:01
So you can think of them as essentially randomly assigned, OK? At least on observed confounders. And I'll keep saying that because we can't say anything about unobserved. So this is the actual data from the study. So we had 5,000 patients. On the left we had 1,579 which were down-referred to the clinic. 3,421 which were not down-referred but eligible. So on the left is exposed, on the right is unexposed. After getting rid of some due to exclusion criteria,
54:31
we had a final population of exposed of 693 and a final population of unexposed of 2,968. And so we had the propensity scores for that entire population. And then we matched the 693 to the 2,968 one to three, OK? So we had a final population of 693 exposed and 2,079 unexposed. So that's our final analytical population matched under propensity to be exposed. And this table, this first table is basically what you would see in table one of a clinical trial, right? What we're trying to show here is
55:09
that we're just balanced unobserved characteristics between exposed and unexposed groups. And you can't probably see it, the numbers are quite small, but you can trust me. It is published, so you can actually check. So our exposed group is down-referred and our unexposed is not down-referred and all this shows is that these characteristics, the proportions across both, are comparable. But there's no significant difference. That the propensity score matching did work
55:33
and it helped us balance observed confounders between the two groups.
55:39
And so what we were able to conclude with this analysis that we couldn't in the randomized clinical trial is that patients managed by nurses compared to patients managed by doctors for HIV care and treatment can have, if not comparable, if not better outcomes in this case. Because what we showed, and I won't go into all the results, but what we showed was that those patients that were treated by nurses had an 80% decrease in risk of death, 70% decrease in risk of loss, and a 40% decrease in risk
56:09
of viral load rebound or failure, in this case. So they had better outcomes. And it's important to note that the outcomes in these-- the events were quite few in number because these are stable patients. So our conclusion is that the results are comparable, if not maybe better, with patients that are managed by nurses.
56:29
The next analysis here, and this is from my dissertation work, using a regression discontinuity design to evaluate a policy change for HIV in 2010. This is an application of a method that I'll continue to use going forward in my career. If it's appropriate, obviously. So just a little bit more background here. As I mentioned before, in 2004 the WHO rolled out treatment in resource limited settings. And at that time, first line therapy was two drug classes, and three drugs total.
57:04
And you don't have to worry about the fancy names. The drugs were two nucleoside reverse transcriptase inhibitors, which was lamivudine, and then stavudine or zidovudine. And then one non-nucleoside reverse transcriptase inhibitor. And again, the drug names are not important, but you will hear me say them a lot. So stavudine and zidovudine are the two older drugs that were of interest in this analysis. And then in 2010, the WHO updated their guidelines
57:29
when new drugs became available in resource limited settings. So the drug tenofovir, which has been used in the high income countries since early 2000, finally made its way to resource limited settings. It was delayed because of the cost, and because one of the main side effects of tenofovir is renal insufficiency and potentially renal failure, which is really expensive, but rare. Also during that time, efficacy trials were coming out, and safety trials, saying that the side effects of tenofovir
57:57
compared to these older drugs is a better toxicity profile and less side effects. So it's a better drug, better quality of life for patients. And then also, the Clinton Foundation did a really good job of getting the prices down. Even though tenofovir is still four times the cost of the older drugs, it's still much cheaper than it was back in the mid 2000s. So what the change in the WHO guidelines set us up to do here, and this is more of the quintessential quasi-experimental design,
58:28
propensity score. There wasn't a cut off, and there wasn't, obviously, any randomization outside of trying to simulate that with propensity score. But this is more of the quintessential ones. What we're doing in this case is we're not assigning, randomly, patients to receive a treatment or control, but they're randomly assigned based on a policy change. And I'll explain this in more detail, but as long as nothing else is changing, the population or outcomes within that population,
58:55
that policy change is essentially random. And patients on either side of that policy change can be considered similar. So we can get what is called local randomization.
59:10
At that time, there was a handful of observational work. Some of which I had put out previously, myself and my colleagues. That was all prospective cohort work. And so what was coming out was this kind of mixed bag of, when comparing the old drug to the new drug, results. Some were showing a higher risk of mortality, some were showing a higher risk of loss, some were showing that they were comparable. But we know that in observational research, a lot of this is based on the assumption
59:36
that when we balance based, or adjust for observed confounders, that we're balanced on unobserved. And that's a strong assumption. Also what was happening at that time is two of the studies had included patients that were initiated onto the new drug prior to the change in the policy. So these patients had contraindications, or were sicker. So they couldn't take the drugs that were available, they could only take the new drugs. This leads to what we refer to as confounding by indication,
1:00:08
which is going to cause them to be at increased risk of poor outcomes.
1:00:13
Regression discontinuity is best shown visually, at least when I learned it, visuals helped. So this is a method that's been used in econometrics in the educational world since the 1960s. And it's kind of foreign to epidemiology because I think people get really nervous when I don't adjust for anything. So there's been some concern and difficulty in getting it published, but we'll sort that out. So what this is is, in this graph we have time on the horizontal axis.
1:00:45
And then you have what we will refer to as the threshold, or the cutoff. Which in this case, this dotted line represents the date of policy change. And then based on where in this analysis patients will be assigned to either the control group or the experimental group depending on the date of initiation of treatment for HIV. So if a patient initiates treatment prior to the guideline change, they're in the control group. If they initiate treatment after the guideline change,
1:01:14
they're in the experimental group. And if there is no effect of the exposure, no effect of that change in the guideline or change in treatment, then we would expect to see something like this. No discontinuity or break at that threshold. But if it's the appropriate method, what we will see is this break at the threshold, and I'll show you this in the results. So this break at the threshold is the actual effect of the intervention. So as long as nothing else is changing along this timeline,
1:01:46
outcomes are continuous, patient demographics and clinical characteristics are the same, this is essentially randomized. Right around the threshold. So we're getting close to simulating a randomized trial.
1:02:01
The data source that we use for this analysis is a huge database for HIV care and treatment. It's the International Epidemiological Database for Evaluation of HIV and AIDS. It's a NIH funded project where they pool observational HIV databases into regions all over the world. The region that we used for this was southern Africa. Specifically, we used Zambia in South Africa. This analysis is a prospective cohort study where we're applying a regression discontinuity design.
1:02:37
It's a fairly big population, over 52,000, the majority of which came from Zambia, where there's over 36,000. Their HIV positive treatment naive, non-pregnant, patients were greater than 16 years of age. Then they had to initiate treatment 12 months prior, or 12 months after, the change in the guideline. And that date of change varies between South Africa and Zambia. South Africa was right along lines with the World Health Organization recommendation, in April 1, 2010.
1:03:09
That's when they said change, and South Africa did that. Zambia changed a lot earlier because they got a huge donation from the pharmaceutical companies to roll out tenofovir, the new drug, at an earlier time. So that's why it's three years earlier. But these two dates are the thresholds, or the cutoff, that we used in the analysis. Then, all patients had the potential for 24 months of follow up to get a look at our desired outcomes, which were death, attrition, which is a combination of death
1:03:42
and loss, immunological response, which is based on your CD4 change, which measures your immune status, virologic failure, and this is HIV failure. So two consecutive viral loads greater than 1,000. And then single-drug substitution, and this is an indicator of toxicity from a drug. And we're only looking at that actual substitution within that one class of stavudine, zidovudine, and tenofovir. We're not looking at anything else, just seeing if those drugs are being switched out.
1:04:14
Coming back to this graph, regression discontinuity can be implemented whenever exposure is determined by the threshold rule. So now that data [? verite ?] initiation is that threshold. And I'll refer to what's called the assignment variable, which is going to assign patients to either side of that threshold. And that's the date they initiate ART. So if a patient initiates ART before the date of guideline change, they are considered in the old drug era for the control group.
1:04:45
If a patient initiates after the guideline change, they're in the new drug era of tenofovir. And then, again, as long as there is no change in outcomes, the patient population is consistent over time, we can assume that those patients that are just above and below the threshold are essentially randomized. And we call it local randomization because you can only say that right around that line. The further you go away from that line, the assumption doesn't hold. And so they're essentially random,
1:05:18
or balanced unobserved and unobserved factors, and the only difference should be the treatments that they receive, is that exposure.
1:05:27
I'm going to go through this quickly, if you're interested in the linear regression equation. It's a simple linear regression equation. We have a constant. We have time and then interaction with the actual treatment. We have an exposure indicator. And then we have this beta2. Within a randomized clinical trial, you're estimating what we refer to as the intent to treat effect. So you're analyzing it based on what people were assigned to receive for treatment or placebo.
1:05:54
So this beta2 is what we're interested in. We're not controlling for anything else or any other factors because we're assuming that they're balanced. And there's ways to check that. There is one main threat to regression discontinuity in regards to the validity of it, and that is manipulation of the assignment variable, so that data [? verite ?] initiation by the individual. And you can think of potentially two different scenarios, which I think are highly unlikely, but you could see happening.
1:06:23
Where a patient knows that there is a new drug coming out after a certain date, and so they hold off on going into the clinic to initiate care until that drug is available. That's highly unlikely. Patients aren't really in tune to what's going on at a national level and what the policies are. So that's most likely unlikely. You could also see a scenario where a clinician could delay treatment because they know this new drug is going to be available. But that's unethical.
1:06:49
These patients are coming into the treatment extremely sick, their CD4 count is very, very low. So to send them away without drugs is unethical. So that could be occurring, but we don't believe it is, and there's ways to test for that. We can test for that by looking at if our observed confounders are balanced on either side of the thresholds. So is our patient population consistent? And then we can also see if there's that manipulation, if people are delaying treatment until after the guideline
1:07:16
change, by just basically graphing the numbers of initiates bend in weekly intervals in our time period to see if there's any, what we refer to as bunching, on the right side of the threshold. So the results. This is basically a table 1 in a randomized clinical trial. And all it's showing you is we have South Africa here, we have Zambia here. We have our observed confounders here, which the only things we could actually measure were age, gender, CD4 count, weight and hemoglobin.
1:07:45
These are all clinical and demographic characteristics of patients. What we saw, there was no significant difference between our exposed and our unexposed group, or a treatment and control on either side of the threshold. On either side of that line, they were similar in South Africa and in Zambia. And that's all that's showing. To see if there was any of that manipulation of the assignment variable, people were delaying treatment until after the guidelines actually happened,
1:08:14
this is just the graph where we're looking at the number of patients initiating. So we have the red line-- and this is for the other graphs I'll show you-- is always the threshold or the cutoff in Zambia and in South Africa. And then patients, again, are initiated on either side of this threshold and either on the control group, or the treatment group. Then we have just number of initiates on the y-axis. Then time on the x-axis, so 12 months on this side, and 12 months on this side.
1:08:42
These black arrows represent seasonal trends in South Africa, so we expect to see dips in patients, the numbers initiating treatment. What we're looking at here is, if those patients who could have initiated over here were delaying for some reason, we would expect to see this huge increase in patient numbers on the right side of that red line, because they were manipulating the date that they actually initiated treatment. We don't see that. We see a slight dip in South Africa and in Zambia,
1:09:09
but then the numbers basically increase to the numbers that were previously there, or previously being initiated. So we don't feel like there's evidence of manipulation of the assignment variable. This figure is just showing the actual probability of tenofovir. So again, to orient you, red line is the date of the guideline change, and then control or treatment on either side. And this is just the proportion of patients that are initiating tenofovir.
1:09:36
This is the reason that we felt like regression discontinuity was a good design option, because you can see right at this threshold, the exposure substantially increases in South Africa. We go 81%, a risk difference of 81%. And a risk difference of 41% in Zambia. And remember, they delayed, and it wasn't nationally done, so Zambia didn't have as stark of a contrast. So that's great. That's where we're like, OK, regression discontinuity. There is a discontinuity in our exposure
1:10:05
at that policy change, which we were very excited about that. This was the only outcome that there was a significant difference. The orientation's the same, we're looking at the proportion of single-drug substitution on the y-axis, and time on the x. South Africa and Zambia. And single-drug substitution is just that indicator of toxicity. So we see a significant decrease in toxicity, a risk difference of negative 15. And then a risk difference of a decrease of negative two in Zambia.
1:10:35
So not as large, but still a significant decrease. And this was expected since tenofovir has a better side effect and toxicity profile from the other drugs. Then these other graphs, this is just South Africa alone. These are the other outcomes. And remember how I said if we had expected that there'd be no effective exposure on the outcome, we would just see a flat line. There was really no effect on any of these other outcomes. We have death, attrition, mean CD4 count, and viral load failure.
1:11:05
So there really was not a difference. The exposure did not affect the outcome. Which should be. These drugs are comparable, they're effective at treating HIV if they're taken. Then this is just Zambia. And we see again, we didn't look at viral load failure, but you see death, loss, and then CD4 count. They're flat. So the policy change, or that exposure to tenofovir, didn't have a significant increase or decrease in the outcomes in this case. In summary, the study allowed us to say
1:11:39
that starting patients on tenofovir standard of care reduced single-drug substitutions, so reduced toxicity. But there were no changes in other observed outcomes, in this case. Strengths were huge sample size, you'd be hard-pressed to get this in a clinical trial. It'd be very expensive and you would spend your whole entire life, and your children's life, trying to get that.
1:12:01
If the assumptions are met in a regression discontinuity design, it's less vulnerable to major threats to validity, like a lot of observational research that we do is, because of the assumptions. We can check those threats to validity by looking at if our observed confounders are balanced, or if there's any manipulation of the assignment variable. And we saw that there was none. Hopefully what I've shown you by these two examples is that, as we move from these descriptive
1:12:28
studies in observational world into analytical, down towards the randomized clinical trial, our gold standard, quasi experimental designs can help us, if applied properly and the assumptions are met, to get a little bit closer to that establishing causality. So instead of correlation, we can say that an exposure actually causes the outcome that were of interest. Thank you. [APPLAUSE] - That was absolutely fascinating. Thank you. And now we have 15 or 20 minutes for your questions.
1:13:07
And while you are thinking of your questions, maybe I'll ask one or two of mine.
1:13:14
Neal, I was struck by the slide up there that showed the proportion of patient contacts by what the procedure was. Checking vital signs was right there at the top, like 36%, 38%, something like that. As you think about the propensity for those contacts in that network to cause infection, do you think about changing the frequency of those procedures? Is there any talk about, maybe you don't need to take vital signs quite as often? I'm sort of wondering how you think about the procedures
1:13:56
in that mix. - So the procedures themselves, we actually ranked the procedures based on propensity to spread in organisms. So something like vital signs, it was like the lowest threshold that's possible that we thought could actually spread MRSA. But some of the other procedures, like the stool description and testing procedure that I think was the second or third most common, yeah. I think that does warrant reviewing that policy to say, why are we doing this so much?
1:14:26
Unless there's a good reason for doing it, this is a clear opportunity for MRSA transmission to occur. Even though it's done under non-sterile gloving, there still is the potential for something to happen. These are the kinds of conversations that, this works, spawned with our infection control group at the hospital. And actually, I hadn't particularly thought of that, so I want to go back right now and-- - Excellent, great. - --say, how often are we actually doing these things?
1:14:56
And do we need to do them as often? - Yeah. A really, really interesting analysis. Thinking also about acuity, and the increased risk of acuity, which is that mix of severity of the cases in your NICU. Is that an individual characteristic, or is that also as a proportion of your population. It's sort of interesting to think about the tension. Like when the acuity is high, is it the highly acute cases who get sick, or is it everybody's risk that increases? - Yes it's both. It's both.
1:15:30
It's the highly acute patients because they are at the greater risk. They're having more done to them and they're sick, or they may have a less of a developed immune system and skin system. But then it's that breakdown that happens overall, especially if there's a code in the unit. If there's a code in the unit, then all the demands get focused right away on that very critical infant. And then you have to think about, as the groups disperse from the code
1:15:56
to go back to caring with the other, at that moment, they are most vulnerable to spreading something if that particular patient happened to have been colonized or infected. - Really Interesting. So Alana, I think the regression discontinuity is really interesting. And I'm thinking of other opportunities for doing that, and also thinking about sometimes we get natural experiments or disasters that afford the same kind of opportunity or threshold as a policy change.
1:16:31
We once did a study where we were looking at the risk of preterm delivery in a pregnancy cohort that we had going on here in Boston. And we were interested in the role of acute stressors. And September 11 happened right in the middle of that cohort, so we were able to look on either side of it. But one of the things I remember we wrestled with is how much time is eligible before the event, and then after the event. And does that need to be comparable length,
1:17:06
and how do you define that against the risk of-- the risk to these things, of course, is there's perhaps a secular change going on in the background. I wondered if you'd talk about that. - So in this analysis, you can have a window around the threshold if you want. Something like, I mean there's obviously a date with September 11, but there could be some window around that, where it could help you kind of tease out the discontinuity more clearly. But you're right.
1:17:34
With something like a natural disaster or a terrorist attack, there are other underlying factors that may not be consistent. Where that local randomization or continuous outcomes assumptions that you have to make would be actually consistent, those assumptions would be met, is what I would be a bit concerned about. Just because of everything that also could be changing around that time date. We're using this going forward. In South Africa, we're building a national level cohort
1:18:07
of HIV, and then also with chronic diseases, diabetes and cardiovascular disease, by doing a lot more big data work. We're using a laboratory system that they have in South Africa to link labs to each other to create individuals and create prospective cohorts and then assess policy changes over time where it's like a clear cut policy, and there isn't this kind of-- I think a natural disaster or anything around that may be a little bit more difficult to make sure the assumptions are met.
1:18:38
But it'd be interesting to try. - Let's open up to questions.
1:18:49
- I'm a particle physicist, so I come from a complete different domain of how we do analysis. One of the things I was curious about is, we have this set of tools called Monte Carlos. And basically, we take a model of the system we're studying, including random stuff, diffusion, that sort of thing. It just struck me that when you try to do a validation, the nice thing about Monte Carlo is that you can run 100 times what you just did for the HIV study, or 100 times what you just did, and then you
1:19:24
can judge the statistical significance rather than just using the more refined statistical tools. I was wondering if there's any attempt to move towards that in the field that you're using these secondary sources?
1:19:42
- Yes. In fact, I have many colleagues that would be quite encouraged by your comment there. Some of the other work that I do, I use a lot of Bayesian methods, where we bring in prior information. We don't start out assuming everyone's at equal risk. We assume some groups are at greater risk than others, and then we run these simulations, many hundreds of thousands of times. So there's definitely a movement of foot in the field to go more towards that direction.
1:20:08
But from a teaching aspect, it's still being taught in a traditional framework. Just like, here's how to get started. So in some sense, it has its own momentum and it's being taught this way. But there are more and more people using the techniques that you're describing. - And I think as myself and my colleagues move into more of the big data world, which is kind of a broad term, a lot of those techniques we're talking about applying. So yes, there's movement.
1:20:40
But you're right, it's taught more in a traditional sense. - I don't know if I'm speaking for you as well. Our field, I feel, uses a lot of methods that come from other places. For example, social sciences. The methods that I'm using for the work that I presented on multilevel analysis, a lot of that comes from the social sciences. So I feel that your group is pioneering some of these methods, and speaks well for our group and it'll just take a little bit of time to catch up to you.
1:21:13
- Other questions?
1:21:21
- Hi. It's a question for Neal. How much variation are there in NICUs in other places? Is there a, kind of, gold standard where they don't have these problems at all? - Absolutely, critical care management of babies differs institution to institution. So in some sense, I wonder with you, how generalizable are our findings to other NICUs. A lot of the day to day what goes on in NICU is fairly well prescribed, and they follow protocols and guidelines. So in some sense, there will be a variation.
1:22:00
I would love to actually repeat this analysis with a whole group of NICUs and see what's out there. But if another NICU is doing some infection control procedure because they have a flag in their EMR to say it's a high census right now or it's a high unit acuity, then our findings may not necessarily apply to them. That is kind of a next step, I think, with this work is let's see if we can replicate this in other NICUs. Is this something just specific to Christiana care,
1:22:31
or is this truly a generalizable issue that's happening across the board.
1:22:40
- As a follow up question, the size of the NICU that you described is considerably larger than the community hospitals that I've worked at. It would be interesting to compare the size of the NICU in terms of the outcomes, because this would certainly have a significant effect on advice as to whether regionalize this kind of care or whether it can be as diffusive as it is now. - Yes. I'm dealing with a level 3 NICU that can go up to 70 patients.
1:23:16
And if you consider a smaller regional hospital that maybe has like a 10 or even just a very, very small 5 bed NICU, I think that's a great question to explore. - All right. I know we have tea and coffee and cookies waiting outside, so let's take a break. I want to thank you guys, these were really fascinating-- [APPLAUSE]
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat