Skip to main content
Loading Explore guides…
Policy Analysis as Problem Solving · Acedra
Home
Explore
Political Science
Policy Analysis as Problem Solving
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
137 segments
Copy
Hide video
0:03
JORRIT DE JONG: Good afternoon, everyone. Welcome. My name is Jorrit de Jong. I'm the director of the Bloomberg Center for Cities here at Harvard University. And it's my pleasure to welcome you to a conversation and a celebration. The celebration is for the book Policy Analysis as Problem Solving, authored by Rachel Meltzer, a professor at the Graduate School of Design here at Harvard and a faculty affiliate of the Bloomberg Center, and Professor Alex Schwartz, who is
0:33
from the New School, a professor of public and urban policy, where he runs the Urban Policy Lab. This is the second edition, new and improved, of the book. I highly recommend it. It's really fascinating, and we're going to learn a lot more about what's in the book and what motivated you to write the book. But we also have a special guest from across the river, Tiffany Chu, who is the chief of staff to Mayor Wu here in Boston. She is a urban planner, but also the co-founder of a civic tech
1:09
company called Remix, and has also been involved in transportation policy issues in California. And she is now overseeing a lot of cross-agency initiatives and strategic policy implementation here in Boston. And we're going to hear from her the real world perspective of how policy gets made and implemented. So we have three fantastic guests, and we'll also open up to you for questions at the end, as well as to the hundreds of people following this event online right now.
1:42
I would love to turn over to Professor Alex Schwartz for some opening remarks, who will then turn it over to Rachel to explain some more about the book. And then we'll open up for a Q&A. Please join me in welcoming Alex, Rachel, and Tiffany. [APPLAUSE] ALEX SCHWARTZ: Thank you. Thank you very much, and thank you to the Bloomberg Center for Cities. It's really a thrill to see so many people here, in person and online, to talk about our book.
2:11
It's something that has never happened before, so I really, really appreciate it. So we're going to give a brief overview of the book and then open it up for discussion. I really look forward to that. So again, this is Policy Analysis as Problem Solving. We call it a framework for thorny problems. So what is policy analysis? It's a general term. And we use it as a pretty open-ended framework, which is similar to and yet different from traditional perspectives.
2:43
It's fundamentally about problem solving for clients, either a policymaker or a legislator. Executive director at a nonprofit organization, usually. It could be for-profit as well and you are addressing an issue for that client. And what makes policy analysis distinctive, I suppose, is that we're using evidence to make an argument to support a recommendation. And as a side note, I would say, we're making the assumption that policy analysis operates under good faith,
3:18
that the client is actually interested in evidence and in a logical argumentation, and I'll leave it at that. So in terms of evidence, you can involve quantitative data, budgetary information, surveys. It could be qualitative data. It can be experiential data. It can be even visual presentations. And policy analysts work under very tight time constraints. They don't have a whole semester, a whole year. Usually it's a matter of days or weeks
3:50
to address the issue, which is distinctive from, let's say, more traditional social science. And policy analysts have to harness a wide range of information for all different steps of the policy analytic process, from problem definition, all the way through analyzing and comparing the different options. We also recognize that information is always imperfect and partial, and you make do with what you have. So this is the cover of the book. It's second edition. I guess, in the book, what makes it
4:28
distinctive from traditional models of policy analysis is that while the framework isn't all that different-- most texts of policy analysis have a multi-step process. But most of them are focused on microeconomics. They, to a large degree, rely on cost-benefit analysis and similar approaches, and really do take this microeconomic approach. And we are more, I guess, agnostic about different analytic approaches. We draw on economics, but we also draw on design.
5:01
We draw on behavioral economics, sociology, and other social sciences. And we also focus on policy analysis as a way of lifting up marginalized voices. So, for example, in the text, we have a couple case studies running through it. One has to do with the child support system, which has really penalized low income parents who are unable to afford child support. And we look at different ways of improving the situation through debt relief, which we'll talk about later.
5:36
And this method, this approach we use, we teach, in that we have applied courses where we advise clients or students advise clients on different issues that they have, and the students use this framework in developing this approach. So here's a quick outline of the book. One chapter is Defining the Problem. Different approaches for thinking about problems. It's often not the first take on the problem. What are some methods of identifying possible options for addressing the problem?
6:11
And here, in addition to more traditional approaches, we also look at design thinking. We look at behavioral economics, other approaches to come up with different ways of developing ideas as well as modifying them. And we emphasize that policy analysis is an iterative process. We have a chapter on criteria, including if some various measures of effectiveness, sometimes cost-benefit analysis, net benefit, but also feasibility, ease of implementation, which is a very important criterion. Cost.
6:45
Whether or not the client wants to know the cost, you almost always have to come up with some sense of it. We have a technical chapter, which introduces students to cost-benefit analysis. We apply a critical perspective on the uses of cost-benefit analysis, some of the conceptual underpinnings, and the weaknesses. We also compare cost-benefit analysis to cost effectiveness. We have a chapter on analysis, where we look at how you apply data and other sources of information
7:17
to weigh the strengths and weaknesses of different options to identify and weigh the different tradeoffs. We have a chapter on the variety of research methods that you can use, should use to work through the policy analytic framework. And we finally have a chapter on the ethics of policy analysis, with examples of actual jobs that policy analysts have. And I think with that-- oh, I'll mention we have a couple cases in the book, and this is actually new for this edition.
7:51
We have one case looking at challenges with water use out West from the Colorado River, and another case looking at whether or not short term rentals, also those Airbnb and other platforms, to what extent do they represent a threat to housing markets, and to what degree should policy analysts address those issues? And I think with that, I'm going to turn it over to Rachel. RACHEL MELTZER: Great. So I'm going to just give you some examples of how we've used this framework and approach to apply it
8:27
in practice. I think what we really try to do in the book and also in delivering and teaching these methods, is to really tie it to application on the ground. We don't want this to be living in some theoretical space. And so we do that in the book by integrating cases throughout. We have one case on child support debt that is threaded throughout the book. And then we have another one related to voting and affecting the engagement and voting. And then we have other examples throughout.
8:57
But the idea is really to make sure that these ideas are tied to actual examples on the ground. So I'm going to just give you a couple examples of how this has played out in projects that we've done in class, both in a live format. So a lot of the content of the book has come out of our teaching this class for a very long time. I won't say the number. But we've had lots of cases on the ground with students and we draw from that not just in the textbook. But here in the examples I'll give,
9:30
we take cases that were live with actual clients at one point that we've recycled into simulation exercises so that students can do the same practice without having an actual client there in that moment. And again, we try to provide resources in the textbook so any instructor out there can replicate it. The first example I want to give is related to a project where originally this came out of working directly with the New York City Mayor's Office of Criminal Justice.
9:56
A lot of these cases were built out of working with agencies, organizations in New York City. That's where Alex is now. That's where I was for a long time before coming to Harvard. And so we worked with a lot of organizations there. The Mayor's Office of Criminal Justice delivered our students with a project. And the students actually came up with this problem. How to reduce the delays in bail payment? That was not the nice, clean, neat problem statement that was delivered to them.
10:23
They were handed a bigger mess of a problem, and they extracted from it this central issue. And I like this as an example, because the students, when they did this live and when we've used it as a simulation exercise, often grapple with the distinction between tasked with solving a bail reform problem writ large versus carving out this very specific question about bail payment right. And so they really on a lot of levels had to do a lot of work to make sure that they were focusing
10:57
on this subset of a problem. So to address this problem, it's an area-- actually criminal justice policy and the implications of spending time behind bars in jail. There's actually a lot of research that shows long term implications of even a few days in jail. So there was a decent body of work out there with quantitative data for the students to build off of to make their case. But there was a lot that they didn't have at the ready to answer this question.
11:33
And so the students spent a lot of time digging into the actual process of bail payment in New York City. I'm going to show you an image in a minute that conveys-- I'm sure you all can guess how not simple it is to do that. And so they really had to understand the different agencies that were involved and the steps that it took. From the seat of a loved one trying to do this for someone who maybe just got arrested and they got a phone call
11:58
that says that they were being bused to a jail in some far away borough of New York City. So really understanding the process of going through that. As I said, there was a decent body of research that they could draw from to understand the implications of incarceration, both in the short and long term. They actually went out and visited the jails. They spent time there putting themselves in the experience of what it would be like to make your way to Rikers
12:23
and try to figure out how to pay bail for a loved one who was arrested maybe hours ago. They interviewed people involved in the jail system, in the judiciary system, and anyone adjacent to this process. And in the end, they came up with a recommendation of an online bail payment system. Again, it seems simple here, but there was a lot of work that went into bringing together this recommendation. Right, and as an example of harnessing all kinds of data. This broad view of data.
13:00
The data is not just quantitative numbers in a spreadsheet, but we're pulling together experiential visits, qualitative understanding of experiences that different stakeholders in this process are going through. And also an example, as Alex mentioned before, of bringing to the table voices of marginalized interests that don't usually have that space. In this example, it was not just people who have been arrested and waiting to be bailed out in that moment, but also the people kind
13:34
of attached to that person, right. And what it means to go through that process and have to deal with the system that you're dealing with. So this is an example of actually a group of students here at Harvard when they did this case for a simulation exercise in the class. And this project really benefited from having a group of students from our Master's in Design Engineering Program. They approached this problem with a systems thinking lens or systems lens. As you can see, I think here, they've
14:06
done a nice job of actually simplifying a process that's way messier and more complicated than it looks here. But there's a lot going on. And they did a nice job of articulating the different phases of this and then used it as a way to zoom in on that node of the process that they're going to try to intervene in. So here's an example of, again, if we approach this agnostically about what lens to use and we introduce a systems approach versus just an economistic one, it opens up a whole new way
14:38
to approach the problem. And it really helped them zoom in on where they could find the solution. And they honed in on this part, final stage of arraignment, where they thought they could make the biggest change. The last example I'll give is another project. This was done a while ago at the New School where, like the GSD, we like to bring different disciplines together. And this was another example of bringing design discipline together with the policy students to try to solve a problem.
15:11
We were lucky. It coincided with a competition that New York City was putting out at the time. I don't know how many in the room here remember when there were payphones on the corner. But when those became no more, we had a lot of infrastructure in the city that was sitting there unused, right. And so this competition was asking, how can we use that infrastructure, right, in a new and maybe useful way? And so we brought together students from the design program at Parsons
15:38
and the policy program at Milano at the New School, and they were tasked with imagining the future of these New York City telephone infrastructure. And this, again, was, I think, a good example of when you broaden the lens, right, the lenses, when you brought in the frameworks of bringing these different perspectives together, they came up with solutions that probably one angle, right, couldn't have done on their own. And so they got together and just did a lot of brainstorming together.
16:07
The policy students were tasked with looking at the service delivery side of things and any policy attachments that would go along with the infrastructure. And then the design students actually implemented the physical, the technological design of the solution. And it did end up being judged by a city wide panel. I don't think they won, but I think they got pretty far. But it was a useful way to tackle it. And this just gives you an example of what they came up with, right.
16:37
So again, the design students probably could not have completed this in full without some understanding of what service this would deliver, what policy it might tap into. And then likewise, the policy students couldn't, right, in the same way have come up with the actual physical and infrastructural design of what we ended with. OK, so the final thing I'm going to end with is what we affectionately call the matrix. And I do see some of my students in the room,
17:04
and they'll know that smirking as they see the matrix. This is really the culmination of all the work that we try to do in the book and the class. And it does a few things. So policy analysis. As Alex mentioned, is a process for using evidence to back up these evaluative criteria that you're going to use to compare the options that you're considering to address the problem that you've identified. There's no perfect solution, so there's always going to be some tradeoff.
17:37
You have to give up something to get something else. And so the matrix is a nice, concise way to show that. Whenever I teach this class, I talk about how you have to assume how your audience has a very limited attention span. Right, maybe you have time for them between meeting A and meeting B. I don't know, maybe Tiffany can speak to this. And so you really have to make your point across in a really concise and effective way. And the matrix is a way to do that.
18:01
So the way it shows these tradeoffs in a one shot is you see on the vertical axis there-- and this is going back to the bail payment example I talked about. On the vertical axis there are the alternatives that the-- and this is the final refined set of alternatives that the students came up with. Along the top horizontal axis are the criteria that they use to compare the options. And you can see that here with the status quo, which is always an option.
18:30
Right, and this is also a tool that makes that clear. That the status quo is always an option in theory. Right, why, it's quite feasible. Right, because you don't have to change much to do the status quo. It didn't achieve their standards of equity that they had established. And the alternative they went along with was lower on the feasibility end. Right, probably was going to take some work to get maybe some coalitions on board. But reached a higher level in terms of the standards they
18:58
wanted to achieve in terms of equity. So it really does a nice job of illustrating these tradeoffs in the end in one table that you can hand off to your client or your audience if they're running from one meeting to the next. And I will end with the matrix. JORRIT DE JONG: Or in the matrix. RACHEL MELTZER: Right, yes. JORRIT DE JONG: Great, so let's go from the matrix to the real world and ask Tiffany Chu, chief of staff to Mayor Wu here in Boston. Can you describe what day-to-day policymaking
19:33
looks like in Boston City Hall? What are the main challenges, and what resonates from the framework that Alex and Rachel presented? TIFFANY CHU: Yes. So hi, everyone. My name is Tiffany Chu. I'm the chief of staff to Mayor Michelle Wu in Boston, and it is such a pleasure to be here and talk about big ideas with people who are excited. I'll just share first and foremost, I think the practice and the day-to-day experience of policymaking is much messier than the beautiful graphs and charts
20:01
that Alex and Rachel just shared. I will say that when we first came in as an administration, I think we were blessed with the fact that then Councilor Wu, who was city councilor for eight years before she was elected mayor, she had already created basically a reputation for being a policy wonk, because she had authored so many policy papers. And so from Green New Deal for Boston to food access and food justice, abolishing the Boston Planning
20:29
and Development Authority, there were many, many different spaces that she had already really shared fleshed out thoughts of what she wanted to see in the world. And that actually made our job way easier when we came into office, because people were so clear about what we wanted to implement and what our priorities were. Basically is number one, housing, number two, climate, number three, youth and families. And so all of those areas she had already put into the world what she wanted to see,
20:57
and it was up to us as staff to implement it. And I actually think the even messier part, not just the policymaking and the decision making, is getting to alignment on how to implement the policy. JORRIT DE JONG: Great. Well, we'll come back to the issue of implementation a little bit later, because that's a very big part why policy sometimes fails or succeeds, right. But I first want to go back to Rachel and Alex. Why a new textbook on policy analysis? There are dozens of textbooks.
21:28
Why did you feel a new framework, a new book was needed? What motivated you to develop it? ALEX SCHWARTZ: So at the New School, we've taught policy analysis a very long time and used a variety of texts. And the framework, as I said, isn't all that different from traditional models. The matrix isn't unique. It's not our invention. The idea of systematically comparing different options. I think what's different is that we try to have a broader view and not
22:02
just look at the desirability, the effectiveness of different options, just in terms of economics, just in terms of looking at the atomistic impacts of a given policy on a population and just tallying up the advantages and disadvantages, the economic advantages and harms, and then tallying them up and coming up with an aggregate figure and deciding which option should be decided on that basis. We just feel that's a very limited approach and probably not realistic in many contexts.
22:39
Which is not to say we don't think that cost-benefit analysis doesn't have its uses. At times, it does. But we felt it was important to be broader, more agnostic in terms of the disciplinary boundaries that should be applied in policy analysis. And that was the main reason, I think, that we decided to write another text. These are a number of them out there. And also trying to provide lots of examples. We had this idea of threading certain cases
23:07
throughout the text to show how from trying to develop, identify, specify the problem that has various nuances and taking the same case, such as voting rights or child support, and threading that through the book was different from the ones we've seen. But I think it's also just being as applied and practical as possible is another reason. But it's not like a giant leap conceptually. It's more of a broadening of the scope of policy analysis. JORRIT DE JONG: It sounds like you really
23:43
wanted to do justice to what Tiffany said. This is messy in the real world, and you need a flexible framework that acknowledges and appreciates that messiness and makes sense of it. Rachel, any thoughts from you? What makes this book different. RACHEL MELTZER: Yeah, no, I think I'll pick up on the messiness theme. And to me, what you alluded to. I think what I felt as I-- teaching the class and when we would work with city agencies and organizations and the problems
24:10
they would give us to solve, mind you, in maybe six weeks at most. Right, these were rapid exercises. For them, the problem is that they couldn't-- I don't know, that they didn't have the evidence ready to go right away. So these were newer, I don't know, newer problems or problems that emerged in new ways. Problems that were less maybe researched or certainly quantitatively empirically. And so this framework became really useful in trying to give still some order to otherwise
24:53
unwieldy problems that did need to be solved. Right, the alternative, it would be, oh, just forget it. Right, we'll put that aside. So I think we try to provide something that could make something that felt unsolvable. Solvable because you had a tool to wade through the unknowns. And so we really wanted to be able to document that in a way that people could follow and learn from so that even if you're thrown something that feels really insurmountable, that you have
25:26
this framework to lean back on. So that the alternative is not to just keep putting that particular problem on the back burner. JORRIT DE JONG: Right, thank you. So, Alex, I want to come back to something you mentioned almost in passing, that there's a big assumption that people are actually interested in evidence and policy analysis. And that assumption may or may not always be true at any given time. So I want to go to Tiffany. What do you do as a policymaker when information and data
25:59
and evidence is contested? Where it is not trusted, where it's not appreciated. Where people have different perspectives on what the truth is, what the problem is, and what solutions might look like. That gets to the messiness, right. But the political messiness of things. So can you describe what that looks like from your perspective? And I want to come back to Rachel and Alex to see how they make sense of that. TIFFANY CHU: Absolutely. So what this makes me think of is when I was flipping
26:29
through Alex and Rachel's book, I was quite taken, because so many of their chapters mirror a template that the mayor had us basically write when we first started on if someone in the city wanted to propose a new policy, how would it get reviewed? What would be the decision making process? And so we have this memo template. It's in a Google Doc. And it's basically like, number one, problem identification. What is the problem? Does everyone agree that this is a problem? Yes or no.
26:59
Who are the stakeholders that agree? Who are the stakeholders that don't? What is the data that backs it up? What is the evidence, both quantitative and qualitative, to make the case to the mayor's office that this is, indeed, a problem worthy of energy being devoted to it to being solved? And I actually feel like that is half of the battle, is to get everyone aligned on a single problem statement. Even though you just put it up on the slide, Rachel.
27:23
Even getting to that singular question or problem statement would take months and sometimes years. And even agreeing on what the scope and scale is, that would require of a huge amount of data and research to back that up. And of course, there's problems of varying sizes. And then you get to the solution exploration space in the template, and then you get to which stakeholders believe in which solutions. And I think all to say when you mesh policy analysis
27:50
with politics and stakeholder engagement, it's, if you're not careful, you often get into this pattern of listening to the loudest voices in the room. Who knows how to reach the mayor's office, who knows how to navigate 311? Who knows how to find the mayor at an event and get in her ear and share that there's a problem? And how do you ground truth that in a way that is actually believable by the most number of people? And you see this happening in neighborhood associations
28:20
all the time, where someone in the neighborhood, oftentimes a leader, will declare that there's a problem. And this happened at this time, and here's a photo of it. But then once you dig in and you unpack it further, it might be some larger situation or it might be a micro situation. And that's why we have neighborhood liaisons who are so crucial to the functioning of the mayor's office. And so we find that the ground truthing, whether it's
28:44
data, both qualitative, quantitative, actually going out into the field door knocking, asking people, hey, is speeding actually a problem on your street? And if so, we can look into solution, like speed humps. That is probably the most important work that the mayor's office does. JORRIT DE JONG: So different sources of information, different stakeholders, bringing them together. But first and foremost, asking yourself, are we really on the same page with regards to this? Taking that step back.
29:13
Let's start with you, Rachel. How do you deal with this question of contested data, different perspectives, and problem definition? This is a rational approach still to policymaking. How to deal with the sometimes more political side of things, or sometimes even the irrationality? RACHEL MELTZER: Yeah, that's a big one. I mean, because there's a whole ethical, right, side of this, obviously. Right, I mean, a lot of people are using data. They're just using it in different ways,
29:49
right, and towards different ends and picking different data to use. I think that on the political side, the point I'd like to make connected to the work we're doing is twofold. I think, one, yes, is the broadening of how we think of data, which that anecdotes are data. Right, I mean, and you don't use them in the same way as you use a census or interpret it in the same way. But that is data that you can use, especially for these problems and issues that aren't deeply researched
30:21
or where we don't have large databases of information to draw from. So I think pulling from those different data sources is not wrong. And I don't think that should be confused with a misuse or mishandling of evidence or data. I think that we view as a reasonable strategy. I think the other piece of it, what we try to do is, again, not take a position on which criteria are inherently more important than others. We have general categories of, you always
30:52
want to be talking about whether the policy is effective. And that looks different, depending on what you're talking about. Right, are you improving student test scores? Are you reducing crime? Right, so the effectiveness looks different. We talk about you might want to have a category of broadly equity, which is really about maybe the distribution of the benefits and costs. So we don't take a stand on which one of those is more important, but that they're transparent.
31:18
Right, that calculation and that tradeoff is made clear. So if the politics, right-- I always say, if there's one council person that you need to get them on board and that's all that matters for this political issue, that should be in the matrix. Right, I mean, maybe an internal matrix, maybe, right, that doesn't get, right, published. But for the decision making process, that should be in that matrix. And it should be applied consistently and transparently.
31:48
The risk is you don't want to take one option and talk about politics for one, and then talk about just equity for the other. Right, they should be applied consistently so that you can make this reasonable comparison. So that's how we try to approach it, is that the politics could be the most important criterion. It just should be applied explicitly and consistently across all the options. JORRIT DE JONG: You're saying embrace it. Embrace the plurality of data and evidence and perspectives
32:21
and make it explicit and then see if you can work with that together. Alex? ALEX SCHWARTZ: I guess, again, that assumes to some degree that a client is interested. And right now, I'm thinking about social cost of carbon, for example. When we revised the tax, we talked about how the federal government has ping-ponged back and forth in terms of what assumptions it makes with regard to social cost of carbon in doing cost-benefit analysis to evaluate environmental regulation.
32:53
And so it was introduced, I think, under the Obama administration. The Trump administration greatly lowered the allowable assumptions regarding the social cost of carbon. The Biden administration increased it. And which has major effects on the chances that different kinds of regulations would be approved or rejected. Federal government requires cost-benefit analysis for any major regulation. Just the other day, it was announced that the Trump administration decided
33:24
to eliminate any consideration of the social cost of carbon. So I think in certain contexts, it's not even clear that policy analysis has any real role in this. It is purely about ideology and politics. And I think it's very difficult for policy analysts to know what to do in that kind of setting. I think that in local government level, it's a little bit easier than, I think, at least in my experience, at least in New York and I think in a number of other municipalities and states,
33:53
there is more appetite for policy analysis. In some degree, policy is somewhat less ideological, perhaps. But I could easily be wrong on that. But right now, I'm somewhat skeptical. There is much space for policy analysis using evidence and argumentation to support different reasons when the client already knows what they want to do. One other point I would make that we talk about in the book is that policy analysis is also distinct from policymaking. We don't claim that policymakers take
34:27
the advice of policy analysts and automatically implement it. It's one factor that there are many other elements. And also even the use of argumentation can differ. So in policy analysis, we value specificity and clarity. But in negotiations and coalition building, ambiguity has its role. And so I think we have to recognize that as well, that policy analysis is only one part of the policymaking process. JORRIT DE JONG: Thank you so much. And also thank you for making a plug for local government.
35:01
Because obviously, we're biased here at the Bloomberg Center for Cities. We want all of you to work for cities, to work in cities, to run for mayor. We need the very best and brightest people in local government. And that's a great place to work, or at least better. So in any case, let's assume for a moment, let's imagine that policymakers are interested in good policy analysis. And many cities face similar problems, whether it's homelessness, or crime, or congestion in traffic, and so forth.
35:36
And there's a whole movement, a whole philosophy around evidence-based policy. So studies have proven that something works in many different contexts. That still raises the question, does that work in my context? And back to Tiffany's point, if you don't agree on what the problem is, you may also not know which evidence-based solution might actually be working in your city for your particular needs. And so, how do you determine what to do under what conditions.
36:14
An evidence-based practice, evidence-based policy might work in a local context. What do you teach your students? What does your book prescribe with regards to that process? And I'll come to you, Tiffany, after that. RACHEL MELTZER: Well, I think one thing that this approach helps analysts do, I think, is break down policies and interventions to their components. I talk a lot about this with the students that I teach. Do we take the really complicated one big policy as one unit and analyze it,
36:53
or do we break it up into the different pieces that make up the whole? JORRIT DE JONG: Could you give an example of that? How to break something down into smaller pieces. RACHEL MELTZER: Sure, yeah. So in the book, actually, one example we use, in the spirit of local government examples, is a program called inclusionary zoning or inclusionary housing, which is attached-- through zoning, you can mandate or incentivize the building or provision of affordable housing
37:21
alongside the building and provision of market rate housing. So it's a way to ensure that any development that happens includes some degree of affordable housing. The way those policies or ordinances are usually designed are made up of components. So it will depend on how many more floors you can build or how much more density you can get if you participate. How affordable it needs to be. To what income level does it need to be targeted? Is there an option to do it on a different site,
37:54
or do you have to do it in the building that you're building otherwise market rate? So there's different components that make up the bigger policy that's called inclusionary zoning. It's also a great example, because it's the kind of policy and ordinance that actually has done a lot of diffusion across places. So a lot of cities have adopted it based on models that they've seen in other places. And some have done a better job than others of adapting it to their local context.
38:20
But when you analyze the effectiveness or outcomes of those individual components, it makes it, I think, easier to translate and also ultimately probably more effective right. Because you start to understand which component is doing what part of that bigger outcome that you're seeing. Right, if you know you're a city and you're dealing with a certain distribution of income that looks different from a neighbor that maybe implemented it, you want to take that into account when you are
38:54
designing the income targets. So when you do that matrix, you want to analyze each component so that you can assess those tradeoffs across each one and then build up. Curate whatever form of the policy might fit best into that context. JORRIT DE JONG: Thank you. Alex, do you want to? ALEX SCHWARTZ: Yeah, I'll just add. Sometimes students get very excited when they see an example from one city. They often call it a best practice, which may or may not be a best practice.
39:21
Most policies don't get evaluated. And one of the things we try to teach students is you have to understand the context in which that policy operated. Is it similar enough to your current place? If it was something in the past, are the conditions then similar to what they are today? Are the conditions and situation in a smaller city in a different political environment applicable to your situation? And those are things that we need to reckon with in order
39:52
to decide to what degree that option could be transplanted or modified. But so the context definitely matters. JORRIT DE JONG: Yeah. Well, thank you for that. And Tiffany, you mentioned that you had a whole template and that the mayor was known even as a councilwoman as a policy wonk. So what is your approach to looking at the whole landscape of potential solutions? Looking at evidence-based policy and best practices, quote, unquote. And then determining what will be right for Boston
40:27
given the specific conditions on the ground. TIFFANY CHU: Yeah, I'll try to answer that question with a specific policy area, which we've been focusing on a lot as administration, and that is on the area of community safety. So some of you may know, Boston experienced the lowest ever number of homicides, the highest rate of just general feeling of safety in, I think, ever in Boston's history. And that was partially due to a lot of policy analysis we had done earlier in our administration
41:00
where, with the help of some experts who had done countrywide work, we analyzed where all the shootings were happening in Boston and realized that 90-- I think it was maybe 75% of the shootings were occurring on 3% of the city's street segments. So if you look at a whole map of the city and you just focus in on the 3% of street segments that were experiencing the large majority of shootings, we realized we could isolate the problem into a very specific geographic contextual solution
41:36
space, as opposed to trying to boil the entire ocean of every single neighborhood across the city. Once we knew what those 3% of street segments were, we would have these coordination meetings across the city, called the Gun Violence Coordination Task Force Meeting. And it would not only be your typical suspects. It's not just the police department. But it's also the public schools. It's also the Boston Public Health Commission. It's also public works. And the reason why was because we realized
42:05
that every single one of those departments had something unique to bring to help make those street segments or those specific parts of the neighborhood safer. If it was cleaning up trash, we'd have the public works team go out and make sure that certain properties were not contributing to a sense of feeling unsafe by picking up trash, by making sure potholes were filled, by making sure street lights were on and functional. And then we'd have our Public Health Commission
42:33
make sure to go out there that if there were incidents of shooting or violence, there was a trauma team that would go out, canvas the neighborhood, make sure that every single resident who experienced some sort of trauma would be available to connect to resources and the city resources that we had. And so that cross-functional kind of coordination, which actually came out of a nationwide best practice. I know this has been done in Baltimore. This has been done in Dallas.
42:59
When we applied it to Boston, we were able to contextualize it in a way that made sense for our city. And I would say that paired with record level of investment in summer jobs for Boston Public School students, record level of summer programming, community centers open, pools open. Something as simple as pools and basketball courts being open make a huge difference to providing activities for young people to play in and be a part of as opposed to resorting to violence, which may
43:26
have been the case otherwise. JORRIT DE JONG: Yeah. Well, thank you so much for that and thank you for the work that you're doing on that. It's great to live in a safer city. And Boston has a history of innovations in this space. The Boston Gun Project in 1995, I believe, was a model for the country, and it got replicated in many other contexts. But it didn't always produce the same type of results. In Boston, it led to a major reduction in homicides within a very short amount of time.
43:55
And then in other cities, it did not work or it worked differently. And so that brings me to implementation. And you brought it up earlier, and I wanted to come back to that. Implementation is not always fully appreciated or understood. Or given the equal amount of attention as the analysis piece. So Rachel and Alex, what do you say in the book, and what do you advise your students and through your students, policymakers, to do to make sure that whatever policy is written up
44:32
is also implementable? And that the intentions are actually ending up being better results. ALEX SCHWARTZ: So one approach to this we take is borrowing from a classic book by Jeffrey Pressman and Aaron Wildavsky called Implementation, which focused on a jobs program in Oakland. And basically, the focus is on how many gatekeepers are there, how many decision makers are there that are required to get approval for a given program or policy. And the more gatekeepers there are,
45:14
the more decision points there are, the more difficult it is to implement to get approval, to get something actually off the ground, to get all the components in line with each other. And that's the first approach we take in doing this, is trying to be cognizant of all the different elements. And to what degree does the client have influence or control over these different decisions? And the less that the client does have control over these different gatekeepers,
45:42
these different decision points, the more difficult it'll probably be to implement. I'd love to know if you think that's an appropriate approach. But that's the one that I think we emphasize the most. TIFFANY CHU: Yeah. I mean, I'll just add. I think we recognize, too, that the degree of implementation detail really depends on the stage you're at in the process. I think a lot of where we use this is maybe often at an earlier stage. Right, because there's different phases
46:14
to bringing something from just ideation to actual legislation. And so the details might emerge through those phases. So I think to Alex's point, it's like, can you have enough understanding of the points of blockage or, right, the amount of negotiation that would have to happen in each phase. So in the case of the Mayor's Office of Criminal Justice, that case I mentioned, it was interesting what came up as a really-- what turned out to be a pretty significant roadblock
46:53
was just the culture of the Department of Corrections. That there was just a culture in that agency that prevented or would make it difficult to implement a lot of the things that the students were working through. So it wasn't a technological blockage. It was just there was an existing culture that would have made it less feasible. But at the stage of analysis, it was still their job to maybe offer up options that did try to fight against that obstacle at this stage.
47:25
And then to Alex's point before, the decision maker, the policymaker, might choose to not fight that battle right now. But it was the responsibility of the analyst to make it clear that if you want to make this change that you say you want to make, you're going to have to deal with these implementation obstacles. And I think the part of the framework, what it also does is enforcing the analysts to make the connection between the intervention
48:00
and the policy and the potential outcomes that it might generate. They're forced to think about implementation. Because to make the connection between opening up swimming pools and seeing a reduction in crime, you have to think about, well, how is that connection made? Is it literally just about opening the pools? Or it's like, how do you actually get youth into the pools? Right, does it matter who's in the pools, lifeguard? So you have to start to think about those details, which
48:30
is inherently about implementation, if you want to credibly be able to say something about the effectiveness of what you're proposing. JORRIT DE JONG: In a way, policy analysis is only rigorous if it actually takes into consideration the implementation, which has a whole different set of issues to consider and think about. Tiffany, last comment for you, and then we're going to open it up for questions. So if you have a question, please think about it now. And there will be mic runners.
48:57
Also for everybody online, I see your questions coming in here on my screen. So I will bring some of those questions into the conversation as well. But Tiffany, on the issue of implementation, what's been your experience, and what do you do to make sure the fantastic policies you develop are also leading to fantastic results? TIFFANY CHU: Yeah, I would say my job as chief of staff is 90% implementation. 10% great ideas and jazz hands and creativity.
49:24
And then 90% is just the grunt work of figuring out how to get it done in a workforce of 20,000 people. And I'll give an example around where I think implementation has been the hardest, but also the meatiest and potentially the most worthwhile. One of the mayor's big priorities coming in was permitting reform. And part of permitting reform and one specific example is making sure that the permitting of affordable housing, she signed an executive order to make that 50% faster than what it was.
49:54
And that, it's a policy. It's documents, an executive order. That was big fanfare, lots of signing. But of course, that is just the beginning to indicate that you want to do it. It's actually all the follow through that matters. And so part of that follow through, and my colleague Maggie is here helping with all that permitting reform, is creating accountability metrics and setting, whether it's quarterly goals, yearly goals, for what that percentage that we're going to hit
50:19
and which department is accountable for what in the pipeline. And my background is in design and startups. And one of the things that we've brought into the mayor's office since we started was this idea of quarterly goals. People might call it OKRs, objectives, key results, however you want to call it. But being able to say, hey, as a city, as a government, we're committing to hitting this goal by the end of the quarter. You'd be surprised the crazed looks I'd gotten from my staff
50:47
when, for the first time, they had these quarterly goals that they had to be accountable to. And that's made a world of difference in terms of how to get implementation from a big splashy press release to actual reality and outcomes. JORRIT DE JONG: Great. So on that note, I'm going to bring in a question from the chat from Paul MacKinnon. And it's a question for all three of you. But let's do brief responses so we can get as many questions in as possible.
51:15
Paul MacKinnon says, considering how universal city solutions are, is the biggest difference between successful cities and not successful cities, mostly down to implementation? TIFFANY CHU: Yes. JORRIT DE JONG: A resounding yes from Tiffany.
51:36
RACHEL MELTZER: I guess I'm just-- JORRIT DE JONG: Academics always need a year or longer to respond. RACHEL MELTZER: Very well. But a successful city is what I'm hanging on to. But yeah, no, I mean, I think I would say implementation. But I wouldn't underrepresent the importance of that first stage of agenda setting and the coalition building around what that problem is in the first place. So, yes, I mean, you have to get it done, right. Especially there are election cycles.
52:14
Right, there's other things at play that this feeds into. But I think that first stage of defining the problem, setting that agenda and getting-- if not getting everyone on board, understanding where everyone stands on that is very important. ALEX SCHWARTZ: I would also say that resources matter. That some cities have much more capability in terms of staffing and just financial wherewithal than others. So I think you have to take that into account as well. JORRIT DE JONG: Yeah.
52:43
So good policies, sufficient resources, and a whole lot of implementation. TIFFANY CHU: Yeah, yeah. JORRIT DE JONG: Great. So let's see if there are questions here in the audience. Yes, on this side here. There will be a mic runner.
53:00
STUDENT: Thank you very much. My name is Geraldine Arias. I'm a career student at the Kennedy School, originally from Nicaragua. I have a question on both planning and implementation and regarding the interface with other departments that might not fill within the unit that's doing the policy analysis. So how do you, both from an academic point of view but also on the practice, how do you manage that? And even on the issue of health, for instance, you also
53:31
have to deal with state-wide legislation and programs and so on. So could you talk a little bit about these challenges? I just mentioned health as an example, but just interested in the overall? JORRIT DE JONG: You start. TIFFANY CHU: Yeah, I'll take a stab. I think health is a really fascinating one, because we have a health commission. It's actually a quasi agency in Boston. There's also obviously the state level and all the hospitals. And so I think we treat them as all key stakeholders
53:55
we have to engage. And I think a lot of the mayor's time actually is spent leading convenings. There's no better convener in cities than the mayor, as Jorrit likes to say in his classes. And so, so much of our time is just getting people into a room led by the mayor to have her share, hey, this is the problem that I'm seeing, and hopefully we get to some sort of problem identification and consensus in the big room. And then with all the different agencies who may or may not
54:22
be within city control, then the next step is after you get a directive big picture, how do we take that and translate that into solutions or exploration or possibilities to explore? But that convening takes a lot of time to do, and we invest in that to build stronger coalitions. JORRIT DE JONG: Maybe either Rachel or Alex. Yeah. RACHEL MELTZER: I would more just from the-- I'm going to defer to Tiffany on the, right, getting it done on the ground.
54:48
But I think in thinking of the importance of that approach, I think everything we've talked about relies actually on the analyst being intentional about going out and seeking those different perspectives and interests. Right, I mean, that's the idea here, again, is to, if you have this broader view, whether it's about the lens through which you're looking at this beyond economics or the data that you accept as useful. Consistent with that is saying, look,
55:17
I'm in the Department of Housing approaching something that's related to health. I should go talk to people in the health department, because they're going to come at this probably from a totally different angle. As long as I'm open to receiving that approach, I think it only makes this better. And I think the tool we have allows for that. I mean, it sets up a way that you can have these different perspectives feed in, and then you can see the tradeoffs.
55:41
Right, the health department might have different priorities than the housing department. And you can see, is there a policy that achieves both those goals at the same time? So as a tool, we think that this is totally useful to try to support that kind of approach. JORRIT DE JONG: Thank you. Alex, let me give you a question from the online audience. Tenzin Gigme asks, whose role would it be to determine metrics or indicators for success for city services or for interventions in general?
56:18
This is an important consideration, he or she writes, given the significance of ensuring that evidence based policies properly accounted for. So how do you define success and who gets to define success? ALEX SCHWARTZ: Yeah, that's a good question. So you're working with a client. I think it'd be a cloud or process who are just determining what the metrics should be for determining the performance, the success of the option. I think it really would depend on the characteristics
56:47
of the alternative that you're recommending. But I think it would definitely be a collaborative process with the client. JORRIT DE JONG: And in Boston, who defines what success looks like? TIFFANY CHU: I would say the voters. JORRIT DE JONG: Ultimately, right? TIFFANY CHU: I think the mayor and the mayor's office have a lot to do with it. The cabinet and department heads have a lot to do with measuring success. But ultimately, it's the voters. RACHEL MELTZER: Can I just add one thing?
57:14
Because I think this is where we learn that, actually, design approaches, behavioral science gives some good insight here to really actually pay attention to what is often called the user. Right, who's on the other end of the policy? So we spent a lot of time when students do projects and in the book saying that, right, again, in that matrix, those users should be represented somewhere in those criteria. Because you may not be able to measure precisely in the end what they are talking about,
57:47
but I think that's an important metric of success in the end, is something that represents the people on the other side of whatever policy you're putting out there. JORRIT DE JONG: Let's see. In the back. Right underneath the clock. Yes, over there. STUDENT: Thank you, panelists. My name is Emma Bowers. My background is working in the Australian government. Great to hear your insights today. I think a number of you mentioned one of the barriers
58:15
to public servants implementing some of these really great innovative tools is that bureaucrats are not always known for being the most innovative and open to novel approaches. So I'd love to hear, especially from Tiffany, some practical examples of when you mentioned barriers to people adopting OKRs and things, what have you seen that's worked to break through that resistance and encourage people to be more innovative in government? TIFFANY CHU: Yeah.
58:51
I've learned this magic word in local government, which is called validators. I did not know this word before in my previous world. Which is, if you're going to say you're going to do something, who's going to come out and say that they're in support of it? And figuring out who our validators are, who our supporters are for these new untested risky new ideas has been a huge job of our job as the mayor's staff to be able to say, hey, if we're going to try this new academic policy, how
59:22
do we know that people are going to support it? And we're not going to know until later because you have to do the policy implementation. But to kick it off in a good way, we have to line up five plus validators from respected organizations or neighborhood leaders or community organizations to say, hey, we believe in this, and we're going to support it until it's successful. JORRIT DE JONG: Great. Any other thoughts? RACHEL MELTZER: I mean, the only thing I'd add and maybe ask Tiffany is,
59:48
I wonder, too, if the strategy of incrementalism helps here, too. Again, there's different ways to make change. You can do things at the margin incrementally that over time, right, can accumulate certainly to big change. And then there's the transformation. And so I wonder, too, if maybe the approach here is incrementalist. Right if you can get people on board with smaller changes. Again, I think the example you gave of just open the pools extra two hours.
1:00:19
Right, or open the parks earlier when it's lighter out, things like that. Don't really change the normal operating procedures in a big way that people can maybe warm up to these bigger changes. TIFFANY CHU: Yeah. And incrementalism I feel like sometimes has a negative-- RACHEL MELTZER: Yeah. TIFFANY CHU: Oh, you're only making a little bit of change. But it's actually the whole theory behind pilots. This is why governments love piloting, because it's an experiment. It's branded as an experiment.
1:00:49
If it doesn't work, that's OK, because it was just a pilot. So do as many pilots as you're politically able to. JORRIT DE JONG: OK. Right here in the middle. Hang on one second, and then the mic will be coming to you. STUDENT: Thank you so much for a wonderful panel. My name is Nina Nagebo. I am a fellow at the Harvard Kennedy School Center for Public Leadership, where we talk a lot about addressing interdisciplinary-- or using interdisciplinary approaches to address public challenges.
1:01:19
And one of the public challenges I wanted to ask you about is with regard to youth psychosocial adjustment in schools, particularly thinking about gateway cities in Massachusetts, if you have any familiarity. But where the challenges are often both school based but also community based, because communities like gateway cities. Holyoke being an example. Also Chelsea, Haverhill, Brockton, other communities in Massachusetts because of the social economic barriers.
1:01:49
So how would you evaluate in the matrix youth testimonials for addressing the challenge, both school based and community based challenge of psychosocial adjustment in schools? RACHEL MELTZER: Thank you.
1:02:07
JORRIT DE JONG: Rachel? RACHEL MELTZER: Yeah, I'll take a stab. So I think the way that I would understand that is so the youth testimonials I would view as the data, the evidence, right. And so the question is, towards what criteria or priority are we trying to understand? So it would help inform what matters to them, what outcomes might affect them. And the testimonials might be data that we could use to project that in the options that we're assessing.
1:02:42
Knowing not as much about the topic that you chose here, so I can't give very specific examples. But I think here in the matrix, it would be trying to say, is there a criterion there that reflects their interests and priorities? Right, is there something about, right, that we think matters in the broader idea of effectiveness? I mean, I like to stay away from these generic terms, but that's where it would fit, right. So for this policy to be effective,
1:03:07
it has to speak or impact this group in some way. Right, and the research would suggest that. So the testimonials, if you have them and if there's a body of testimonials you can draw from, that could be used in a sense to understand, well, if we did this intervention with a school, what did the testimonials suggest in terms of how those youth would respond to that intervention, if that makes sense? ALEX SCHWARTZ: The challenge with that is you want to be able to compare different options.
1:03:38
And of all the testimonials that are at one particular course of action and you don't have anything about the others, that becomes more difficult. But if you could, for example, talk with the key stakeholders with the students and get their sense of the different approaches and that could inform a criteria in terms of student support, for example, or to compare different options. But just having testimonials for particular option, that could inform--
1:04:07
be part of a broader gauge of support or feasibility. Political feasibility, perhaps. But it would be difficult to use it just by itself, I would say. RACHEL MELTZER: Yeah. And the only other thing I'll add, which we haven't talked a lot about the status quo, again, as a thing or as a viable, again, alternative in the range of options we're exploring. So this could help, too, in just setting the stage, laying out the landscape of what we're dealing with. Which is important work here.
1:04:37
We didn't talk a lot about it. But before you even start digging into what the options for change are, you really have to understand what's going on in the first place, because that's a really-- first of all, it's the status quo is usually what you know the most about because it's there in front of you. And so options can build off that or aim to disrupt that. So the testimonials might also be helpful in establishing where things are now and what looks like opportunities for change, what
1:05:05
is working, what isn't working. And as a jumping off point for the policy changes that get suggested. JORRIT DE JONG: Great. Well, that brings me to the final question of the conversation. And then, of course, you can all stick around. There's food, and the speakers will be around for a little bit longer and you can have more conversations. But for the public part, what do the best public problem solvers have in common? What type of skill set and mindset are most needed in today's public sector?
1:05:40
Do you have any advice for the students here who would like to become professional urban problem solvers? Let me start with Alex, and then Rachel, and then Tiffany, you have the last word. ALEX SCHWARTZ: I guess I would say being open-minded, being open to different both theoretical and analytic approaches. Yeah, I'll leave it at that.
1:06:07
RACHEL MELTZER: I would say-- and again, I say this to my students all the time, that it's not about being necessarily an expert in one policy, topic, or area. In fact, the opposite. If you can get really good at being thrown any which topic or issue and being able to grapple with it, that, I think, makes you really equipped and ready to go in and do this kind of work, which I'm sure you'll attest to, right. Especially if you sit in the mayor's office.
1:06:34
I mean, you're thrown all different kinds of issues, and there's no expectation that you can be an expert on all of them. But you have to know who to go ask, right, about that expertise. How to drill down on, right, what is the core problem? So being able to parse a lot of information, a lot of noise and pull out the signal is a really good skill to have. And I know that sounds abstract, but the way you get good at that is actually by just practicing it.
1:07:02
And that's why we talk a lot about cases, doing simulations. The more you do this stuff, I always say, it's like you're strengthening these muscles. These are muscles that you can make stronger, and then it just becomes almost intuitive. So when you're throwing something, you don't know you can fall into that exercise pretty easily. So even the exercise of, which T line will I take to work? Right, those decisions all have decision points. And you're weighing tradeoffs all the time.
1:07:36
So the more you do that, even on those micro scales, actually really sets you up to be able to make these decisions often really quickly. Sometimes, right, time is not on your side. Right, you often have to make these decisions quickly. So that just comes with practice. JORRIT DE JONG: All right. Tiffany. TIFFANY CHU: My advice would be, don't wait. Don't wait to get involved. Don't wait until you have a formal degree or a graduation diploma or your next big career title.
1:08:08
Just get involved in your local community right now. You can practice, as Rachel was saying, by logging on to your next neighborhood association meeting and speaking in support of a housing development in your neighborhood that probably many other people are against. Or you can go sign up to give public comment at a city hall meeting. Or you can show up to any variety of public discussions that are happening very robustly in the Boston region about any topic.
1:08:41
And I would say that's your best bet to learning more, because the more you practice your arguments, your policy positions. Even looking someone in the eye and being like, is my narrative being compelling to you? Are you hearing what I'm saying? Do I need to adjust the way that I'm presenting my topic? I think that'll make a huge world of difference. And we need more voices in our democracy. JORRIT DE JONG: All right, well, I couldn't wish for a better ending and a better plug.
1:09:07
Because if you are interested in this work, the Bloomberg Center has very many opportunities to actually work with people like Tiffany and Mayor Wu all across the country and even around the world. We work with many mayors and senior leaders in city government. We have summer fellowships. We have postgraduate fellowships, where we place people in cities for two years to do this work and to support cities in doing this important policy analysis and implementation work.
1:09:37
Also, on the notion of cases, there's a large collection of new cases available that we publish on our website. There are QR codes in the back. So if you pull out your phone, you can get connected with these opportunities and resources. But most importantly, the resource that I want to plug here is the book. Again, Alex and Rachel, congratulations on the book. And Tiffany, thank you so much for sharing your perspective. [APPLAUSE]
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat