Skip to main content
Loading Explore guides…
Epidemiology Risk Assessment Collaboration · Acedra
Home
Explore
Other
Epidemiology Risk Assessment Collaboration
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
67 segments
Copy
Hide video
0:03
All right. Good morning. A good afternoon and good evening to everybody. And I'd like to start by thanking the conference organizers for inviting me to speak with you today. It's really an honor. In this talk, I'm going to be hewing closely to the conference theme of the interactions between the disciplines of epidemiology and risk assessment. And I'm going to be offering some thoughts on how our disciplines might work together more harmoniously
0:28
to advance the shared goal of public health protection. Oops. Sorry I'm just getting the hang of this this program here. You should see an outline. So the the outline for this talk is pretty straightforward. I'm just going to provide a disclaimer, give you a little bit of background information, describe some barriers and then close with a brief description of a couple of tools that I think might help move us past these barriers. So the
1:04
disclaimer I'm going to be talking about both epidemiology and risk assessment, but I wanted to start by making it clear that I am not an epidemiologist. I am an exposure scientist and a risk assessor, and more specifically, I'm an American exposure scientist and risk assessor. And the reason that I focus on that is that I know this is an international conference, but a lot of my experiences and the work that I've been doing in these areas comes from my work here in the United States.
1:31
And so that's kind of the prism that that I will be looking through when talking about these topics. But I'm really hoping that what I'm sharing with you today will be generalizable to really any country. Now in looking at the conference agenda, it's really clear that we're all going to be diving deep into the weeds right away and staying there. And so I thought for the opening talk, it might be good for me to at least try to stay more at the 10,000 foot level. So let me just start with
2:03
some background. For the risk assessors in the room I know that you're all aware that risk assessment relies heavily on data from a wide array of disciplines and really only a fraction are shown at this slide. But one of those disciplines is in fact epidemiology. Now, in my early training in risk assessment, which started in the 1980s, my colleagues and I at the time, we were very focused on the need for data for what I'll call quantitative activities.
2:34
So these are the kind of activities, let me give you an example. We might be faced with a highly contaminated military base and the folks who are running the base, they want to know how many tons of dirt I need to get off of this base in order to bring the risks down to a level that's acceptable to people on the base and perhaps to neighboring communities? They needed a number. And so we needed to find data that would help supply them with that number.
3:02
And for that, and certainly at the time, back in the 1980s, we were getting most of our information from toxicological data and really not so much from EPI data. And the protocols that we followed to do these risk assessments came from EPA or Environmental Protection Agency documents that were really just being developed at the time. And these were their multiple volumes of risk assessment guidance for Superfund. And we use these protocols
3:29
to help us do pretty much everything from the time we got on the site all the way through the risk assessment and then into the risk characterization. Now, as time went by, I think more and more of us got interested in thinking about what the totality of the literature was telling us about chemical interactions and chemical interactions with the human body and associations with health outcomes. And we were doing that in the form of weight of evidence assessments,
3:59
and then more recently in the form of systematic review. And here clearly we rely heavily on epidemiological data. And you know, epidata is incredibly important because the studies have many important attributes, not least of which the target species is directly relevant. Epidata reduces the need for high to low dose extrapolations. If we're interested in health endpoints for which we have no laboratory animal models or at best poor laboratory animal models,
4:31
Epi becomes critically important once again. And then in more recent years, societally, we've become very interested in minimizing the use of animals in chemical testing. And so once again, epi fulfills that role. But at the same time, there are barriers to the use of epidemiology in risk assessment. And this idea of having these barriers, these epi-risk assessment barriers, or sometimes we talk about an epi-risk assessment gap, it's not a new issue.
5:04
For decades there have been calls for improving the suitability of epidemiology studies for risk assessment. And given the theme of this conference, this seems like a really good time to shine a bright light on some of these barriers and just talk about them. And, you know, putting this talk together. I was really well aware that by starting off the talk, by focusing on barriers, it might feel like I'm trying to set up a risk assessor epidemiology brawl, but I'm really not.
5:32
I think that if we don't talk about the barriers first and acknowledge them, it's going to make it much harder for us to move forward and think about ways that we can work together better. So let's talk about some barriers. Let me start by putting up here a quote from the conference flier. This is from the flyer for this meeting and the quote that you see there is directly from the flyer, except for the underlining, which is mine. And what this phrase says is the presumed inability
6:03
of observational studies to demonstrate a causal relationship may even lead to their exclusion from the evidence assessment. So I think this is kind of a good place to start, because in this statement we're talking about the presumed inability. But in fact, many observational studies can't demonstrate a causal relationship. And I think we shouldn't be afraid to acknowledge that and to talk about it. Many of these studies weren't designed to demonstrate a causal relationship.
6:30
And the idea that this could even lead to their exclusion from evidence assessment I think is also important to be thinking about, because there are reasons why studies may be excluded from evidence assessment and they may be good reasons. And again, we shouldn't be afraid to talk about this. So let me give you a few examples from two main, kind of topics of barriers. In the first one I'll talk about it has to do with quality, and I'm going to focus on the U.S. EPA just for the moment.
6:59
So what you see on this slide is some information from a paper that was published by EPA scientists back in 2008 by Persad and Cooper. And what they did was they looked at health assessments within EPA's Integrated Risk Information System or IRIS. And I assume most of you are familiar with IRIS. And at the time, IRIS contained 545 health assessments. And what Persad and Cooper found was that only about 8% derived their non-cancer or cancer risk estimates based on human data.
7:33
Now, it isn't just that for 92% of the assessments, there were no human data. In fact, there were epi studies, but there were issues around various aspects of methods in the studies that limited their use in these assessments. And some of those issues are shown on this slide. Selection and characterization of comparison groups, sufficient length of follow up, adequate sample size. And this last one you'll see a few times this morning accuracy of the exposure characterization.
8:04
In a similar kind of effort, additional EPA scientists took a look at IRIS in the context of their use of neurodevelopmental testing in children as the basis for their health assessments. And they were interested in this, because we know that there are many, many chemicals that are thought to be neurodevelopmental toxicants. But in fact, in the IRIS database there are very few health assessments based on neurodevelopmental function.
8:33
And so they were looking to see why that was and they were noticing that these studies were limited by a wide array of factors. And I'm not going to read this whole list you, but I'll just pull out a few, adequacy of sample size I already mentioned in the last slide. Inconsistent testing methodologies, questions about selection or implementation of testing procedures. Once again, uncertainties regarding exposure. So, you know, the different EPA offices are in fact
9:02
looking at quality of epidemiology research. And the different offices within EPA do this in different ways. I'm showing you in this slide an example from the office that handles National Ambient Air quality standards or NAAQS.. I really picked this one because some of the other offices have much more complicated approaches to assessing epi quality, and it was just too hard to put on a slide. But you can see as an example, one of many questions that EPA asks when looking at an epi study is:
9:32
Are the air quality exposure or dose metrics of adequate quality? And are they sufficiently representative of or pertinent to ambient conditions? So when nobody likes to talk about the quality of research. None of us likes to have our research judged by quality. In fact, quality assessments have been going on for a long time and they are most likely here to stay, or at least they should be here to stay. So so that's the quality part. But now I think it's worth exploring just a little bit
10:01
kind of how we feel about each other. You know, how do epidemiologist feel about risk assessment and how to risk assessors feel about epidemiology? So let me start with the risk assessor attitudes. So the information on this slide came from a really interesting poll that was conducted by Burstyn et al. and what these folks did, they polled 75 risk assessors, and these risk assessors came from different disciplines epidemiology, toxicology, exposure science.
10:30
And they asked them their views on the value of epidemiology research for risk assessment. So I just pulled out a few thoughts here from their work. 16% of those polled found epidemiology to be not useful but has potential. 19% found that epi was not useful and has no potential. 39% said useful, but could be more so and 20% said useful as is. So I think you can kind of feel a problem brewing here. The other common theme that they found in their polling results
11:06
was a call for more useful exposure measurements, and they included this quote from one of the people they polled, who said: this needs to be done right for the results to be useful, and it usually is not. Now, as an exposure assessor, all I can think of when I read that line is, you know, ouch, this is not good. And then finally, there were multiple calls for epidemiologists to sit down with risk assessors and learn what they need.
11:33
I want to say this last one again, and I'm going to ask you to keep it in your head for a couple of slides, because I think this is really important. There were multiple calls for epidemiologists to sit down with risk assessors and learn what risk assessors need. Okay. So a few years ago before the pandemic, a colleague of mine, Dr. Carole Burns, who's an epidemiologist, she and I took a road trip and we went all around North America and we met with lots
12:00
and lots of epidemiologists from academia and government and business. And really we were doing a couple of things. We were, first of all, talking about a tool that I'll describe to you later, but we wanted to get people's views on what they felt about risk assessment. What did epidemiologists think about risk assessment? We didn't take a formal poll. I'm not going to show you anything quantitative, but I just want to share with you
12:23
some general big picture thoughts that we got from these conversations. So the first important point in this follows up from two slides ago. What we heard from epidemiologists was, wait a minute, why should we sit down with them? We're doing the research the way we want to. We're answering the questions that interest us. And if risk assessors can't use our data, if they can't figure out how to use our data, then maybe it's the risk assessors who need to change and maybe they need
12:53
to change the risk assessment processes. And the other thing we heard was, you know, there's no reward for us for changing how we do things. There's no benefit. We don't get promoted more readily if we make our epi studies more suitable for risk assessment. And in fact, if our studies are used by risk assessors, we usually don't even know about it. You know, it's not like a risk assessors coming back to us and saying, wow, your study was critically important for our risk assessment
13:19
thank you so much. You know, they don't even know for the most part when their work is used, there's no benefit. And the other thing they pointed out was there's no funding. You know, they get their grants funded and they're doing the work that they've designed it to do. They're working on the research they intend to do. They don't have the time and the resources to do lots of extra things. And sponsoring agencies are not saying, hey, this is a great proposal,
13:46
we're going to fund this, but we want to give you another 15% in terms of support so that you can make your study more suitable for risk assessment. So there were all these barriers that we heard on this on this tour. But having said that, the window was not shut all the way. I think it was it was cracked open because from some of the epidemiologists, we heard more things. We heard things like, you know, I actually would like my epidemiology research to be used by risk assessors.
14:19
I think that's pretty interesting. But I only had one risk assessment class or I didn't have any. And so how can I learn more? You know, where can I get training and what is it that risk assessors need? So, you know, I think that if we had a room and we put 50 or 100 risk assessors in a room, maybe like you all are right now, and we gave you a pen and paper and we asked you to make a list of everything you'd like to see from an epi study that would help you use it for your risk assessment.
14:50
I think we'd end up with a list of like 50 or 100 things, and that's basically a conversation stopper. Nobody wants you to get a to do list of 50 or 100 items. But I think we can look to the words of the philosopher Mick Jagger and we can say, you know, you can't always get what you want, but if you try, sometimes you might find you get what you need. And that leads us to the last part of this talk, which is about a couple of tools that I want to briefly explain to you.
15:22
Let me start by saying that these tools are not rocket science, and that's kind of the point. They're meant to be easy to understand. They're meant to be easy to use. They're meant to be a way a starting point for our two disciplines to talk to each other. The other thing I'll say is that you'll notice that the two tools have some overlap, and that's also the point, because I think once again, using that example of if we put 50 risk assessors
15:46
in one room and another 50 risk assessors in another room, and you each develop your wish list, I think we'd find that there would be a lot of overlap and that certain things would rise to the top in both groups or things that we're all looking for. Now, these tools cover a lot of aspects of epidemiology, but I'm going to focus on exposure for sake of time. Okay, so the first tool that I'll talk about is called the BEES-C, and this stands for the Bio Monitoring
16:16
Environmental Epidemiology and Short Lived Chemicals Instrument. And you can see from this slide who was involved in this workshop and in this effort. This group got together about a decade ago, and at the time what we were struggling with was the following. I think around this time period we knew we had gotten really good at doing bio monitoring and epi studies for persistent chemicals, the dioxins of the world, the PCBs. We knew how to collect the information and we knew how to interpret it.
16:47
But around this time we were faced now with a wave of epidemiology studies that were focused on short lived chemicals, ones that only stay in the body for a day or a week or maybe a few weeks. And a lot of issues were arising that were really impacting our ability to use the data and to interpret it. And that's what this instrument was designed to help with. I will say that in hindsight, I think that even though it was focused at the time on bio monitoring,
17:14
the tool is really applicable to any environmental medium. And even though we were focused on short lived chemicals at the time, I think similarly it works for pretty much any chemical. So let me just show you what it looks like. If you look at the figure on the left and you are not meant to read the words, it's really up there just to show you the color organization, the color scheme. The idea is that there are a number of elements
17:39
that were considered by the group to be most important, and for exposure you can see the list in the right hand column on the slide. The idea is that for each of these elements, the group came up with guidance. And the guidance was meant to say, look, if you want to develop the most robust information that you can, that can be easily interpreted and used by people like risk assessors, these are the things you should be thinking about. And if you do these things,
18:05
then you would have what's called a tier one study that would be shown in green. If you can do some of those things, but not all of them, that would put you in a tier two study, which is shown by that sort of yellow orange color. And if you just can't do any of them for that element, then that would put you in tier three or the red color. Even though it looks like a grading system, it's not. It is really meant to help us organize our thinking.
18:29
So for example, as a risk assessor, there may be one element in the study, one methodological element that's critically important to you. You can only use studies that do that element really well. You don't care about the other elements. And so you're looking for tier one studies just for that element. The other ones you can set aside. So let me show you two examples that I think maybe will make this a little more clear. Let me first talk about avoidance of sample contamination.
19:00
So for anybody who's done field work or done lab work, I think that you're well aware of how difficult it can be to keep your sample free of contamination from the analyte of interest. This is true for bio monitoring studies. It's true for environment sample collection. Contamination can happen during sample collection, transport, storage in the lab. And in fact, there are some beautiful and eloquent papers that were written
19:26
by scientists at the Centers for Disease Control and Prevention here in the U.S., talking about how difficult it is to avoid sample contamination even in their tightly controlled environments. And so if you're a risk assessor, you want to know that the exposure data that you're getting is true exposure data. It's not an artifact of contamination. Otherwise it's going to limit your ability to use it for things like, say, dose response assessment.
19:51
And so you're going to need some documentation that shows how the researchers avoided sample contamination. So how often is this an issue in the epi literature right now? So a few of us explored the epi literature on 2,4-D and we were looking to see how many studies we could find that would fall into Tier 1. In other words, there was documentation about how the researchers kept their samples free of contamination. So I don't have a pointer, but let me just kind of describe this.
20:23
That left hand column under Matrix. Those are the different matrices for which we found research on 2,4-D measurements. And the number next to those simply shows the number of papers we found. And in that right hand column you can see the breakdown in the pie charts of the numbers of Tier 1, Tier 2 and Tier 3 studies. So just to focus in on one example, if you look at the last row, you can see that we found 52 papers that measured 2,4-D in urine. And of those only three provided
20:54
documentation of how they kept their samples free of contamination. 36 had no documentation. Now, this isn't the end of the story in terms of our disciplines working together, because just because the 36 papers didn't provide the documentation, it doesn't mean they didn't take the steps. It may just mean they couldn't fit it in the paper. And it could be this simple call from a risk assessor to an epidemiologist to say, You know, I'd really like to use your paper in my risk assessment,
21:21
but I'm missing this documentation. Can you send me your quality assurance plan? Because if you do that, your paper goes from a Tier 3 in this element to a Tier 1, and I can use it. One more example from the BC has to do with establishing that exposure to the chemical of interest precedes the health outcome of interest. This is obviously a critical and basic tenet of epidemiologic research. And the question is, can one biomonitoring sample do this for you?
21:51
And I'm just going to focus on urine to use as an example for for talking about this. The reason this is so important is that if you look in the literature, you'll find that the dominant portion of the literature focuses on and bases its assessment on one urine sample per participant. So the sample is either collected before the health outcome occurs. It can be collected at the same time as you're collecting information on health outcome, or it might be collected afterward.
22:19
But either way, you're making the assumption that one sample represents that person's long term exposure. So how good an assumption is it, that that one sample is going to do that for you? The way that you'll see this described in the literature is with something called an ICC or an intraclass correlation coefficient. And I don't have time to go into this in detail, but if you just look at the right hand side of this slide, this is what you'll see in publications. ICC values range from 0 to 1
22:53
and they have traditionally been described the way you see it on the right hand side of this slide and ICC of less than 0.4 means poor reproducibility. In other words, one sample probably isn't going to tell you something about long term exposure. 0.4 to 0.75, you're in the fair to good range, 0.75 and above you're in the excellent range. Now, as scientists, you know, we're not, we're not really most of us aren't really comfortable thinking about something scientifically in terms
23:21
of such what I call squishy words, you know, poor, fair, good, excellent. What does that even mean? And if you're a risk assessor, trying to hang your hat on a study that's looking at a chemical that might have an ICC of 0.6, what does that mean in terms of your confidence of the results in that research? Well, we were interested in looking into this a little bit. So first, let me get you oriented to this slide. On the y axis, you see the ICC values ranging from 0 to 1,
23:50
and then that red line is that 0.4 line, the 0.4 value that separates poor from fair to good. And then the green line is the 0.75 value. That separates fair to good from excellent. So can we quantify this a little bit? So Dan Naiman, who is a statistician at Johns Hopkins, and for full disclosure, he's also married to me. He did some modeling around this and I don't have time to get into the model. But what he found was that based on one urine sample
24:21
for an ICC of 0.4 about 60% of people in your sample population may be incorrectly classified according to exposure. And even at an ICC value of 0.75, about 40% of people may be incorrectly classified according to exposure. And that's in the excellent range. So as an epidemiologist, as a risk assessor, you need to look and say, Well, am I comfortable with that level of error? Now you might be sitting there going, Well, okay, but maybe most ICC out
24:52
there are like 0.9 or 0.95, and maybe that's the kind of error that I'm comfortable with. Well, we were interested in trying to figure that out. Where are most of the ICCs? And so a few of us scoured the literature looking for as many ICC values as we could find for chemicals that tend to be part of biomonitoring studies. And I want to show you the results of that effort and I'm just going to lay the results right on top of this slide. Okay. You are not meant to see the words across the x axis.
25:22
Just take my word for it that that's just a list of most of your favorite bio monitoring chemicals. It's phthalates and parabens and so on. But really, what should you, which should be jumping out at you is that the preponderance of the ICC values are falling below that 0.75 value. They're mostly in that range where you could have, you know, upwards of 40, 60, 70% exposure misclassification. Okay. Now, at the start of this talk, I promised you
25:51
I wasn't going to get into the weeds, but I went ahead and did it anyway. But we're coming out of the weeds now, and I just want to say that the take home in the context of the BC is that if you want to have a Tier 1 study, one that the risk assessors can really rely on, it's important to before you conduct your study, when you're in the study design phase, be really thinking carefully about how many samples you need to collect in order to properly characterize exposure.
26:16
And that might address that risk assessors complaint from several slides ago, saying exposure assessments are often not done well. Okay. So now we have just a little bit of time to talk about the second tool, which we call the Matrix, and what this is really all about are asks for epidemiologists. And this came out of a workshop that happened just before the pandemic. And you can see from this slide who was involved in this workshop. And let me give a special shout out to Dr.
26:46
Greiner, who was who was part of this effort. And the idea here was to kind of do the exercise that I mentioned earlier in the talk, get a bunch of people in the room, not all risk assessors. This was a multidisciplinary group. But think about what is it we really need from studies in order to help us bridge that epi risk assessment gap. And what we came up with we called Matrix for obvious reasons it's a matrix. And what you can see here is that there are nine asks
27:16
and they're organized around the three fundamental steps of risk assessment, which you can see on that left hand column. And I would just remind you again, these are asks these are not demands. I don't have time to talk about all nine this morning, but I would like to focus in on one. And that's when you can see in the lower left hand corner: source to intake pathways. And I want to talk about this one because these asks are meant to be easy,
27:43
but source to intake pathway is actually the most difficult one. So if you think this one isn't too difficult, the rest would be really easy for you. And the other reason that I want to talk about this one is that my epidemiology colleagues tell me that source to intake pathway is not part of the typical lexicon for epidemiologists. And so I think it's useful to bring it up today. So what do we mean by this ask? On the left hand part of the slide, so envision
28:10
you're setting up an epidemiology study and you're going to use urine sampling to assess exposure to your sample population. And so you collect hopefully the right number of urine samples per participant. And that's going to help you figure out and characterize your sample population's exposure. And that's great. But what it doesn't tell you is what the source of the exposure is, because as you know, biomonitoring integrates all sources of exposure.
28:37
From a risk assessment perspective, though, source is really important and so if as part of your study planning, you can arrange to have some multimedia sampling that might tell you where the exposure is coming from, is it coming from exposure to dirt? Is it coming from their diet? Is it coming from the air they breathe? If you know that, then that can turn a really good study into a really high impact study, because it allows for action. It allows the risk assessor to focus in on the medium
29:06
that they need to be considering in terms of limiting exposure. So, big picture for the Matrix is that it was designed as a communication tool to advance an understanding of risk assessment. So that goes back to that question that epidemiologists asked us, What is it risk assessors are looking for? It's designed to help increase the translation of epidemiology data. These are asks for epidemiologists, not demands. Nobody's telling anybody to do anything.
29:38
These were designed to be elements that have a lot of impact. If these could be done, it could really enhance the value of an epi study for risk assessment. It's not intended to supplant current best practices. Epidemiologists know how to do epidemiology. Perhaps most importantly, it's forward looking, like with the BC, this tool is not a grading system. You're not meant to pull up a published paper and compare it to Matrix and say, Oh my gosh,
30:06
the only address two of the nine asks, so I'm not going to use it. That's not its purpose at all. Its purpose is to begin a conversation between epidemiologists and risk assessors to be able to say, Look, we have these nine asks, Are there any that you think you can build into your study design? Do any of these interest you? Because even if you did one of them, the value of your study for me would increase greatly. Some of these can be done post hoc.
30:31
Once the study is done, a risk assessment might be able to go to an epidemiologist and say, Look, if you share some of your data with me, I could even meet some of these asks and be able to use your study for my risk assessment. So just to summarize, the slides that I showed you early in the talk notwithstanding, it's really been my personal experience that most risk assessors want to use epi data. And that epidemiologists want to have impact,
30:58
but they don't always know what risk assessors are looking for. And so the two tools are meant to offer guidance on key aspects of epi studies that can help with translation and to serve as a foundation for interdisciplinary dialog. And I said it already, but I think it's worth saying again, sometimes here we're talking about very small changes that can have a really big impact in terms of improving the value of epidemiology research for risk assessment. So, just to close,
31:30
I think, you know, we're living in times right now where we have many problems that feel overwhelming and insurmountable. And enhancing epi for risk assessment should not be one of them. I think that risk assessors and epidemiologists can be vocal and work together and be vocal about the need for enhanced training, for additional funding, especially to support some of these additional components and analyzes that I mentioned today. And for a better reward system.
32:01
And I can't close my talk by speaking German, but I thought I would put it on the slides. So thank you for listening.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat