Skip to main content
Loading Explore guides…
Platform Engineering & DDD · Acedra
Home
Explore
Computer Science
Platform Engineering & DDD
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
94 segments
Copy
Hide video
0:10
Have a warm welcome for Gregor. Oh, thank you!
0:17
Yes, and welcome to the last sessions. The heads must be almost full. So my goal is to add a little bit extra. So when I attend conferences, I get the impression that we have our own kind of hype cycle. Like, in a way, when the topic is new, everybody talks about definition. What is it actually? And then once we have the baseline down, then every person comes with their own point of view, kind of puts their own label on it.
0:44
And then a little bit later we get the talks about what can actually go wrong and the experience reports, which I find very, very useful. And then when we think we are done with a topic, one person always comes and says, hold on, you guys got it all wrong. Let me set you straight. So my talk is a little bit of all of that. And in particularly I want to connect platform engineering with domain driven design. There is a topic we talk about a lot, but it has an interesting history
1:15
when Eric Evans, the book came out almost the same time as Enterprise Integration Patterns in 2003, and it kind of people talked about it, but it was kind of sort of puttering along. Then about ten years later, Vaughan's book gave it a big boost. But really, DDT came to life when we started doing microservices, right? It was sort of at the time when Eric wrote the book, that was not conceivable, right? There was no notion of microservices.
1:41
So it's kind of interesting how something completely different actually gave his work such a giant boost. And I think platforms actually also should give Didi another boost or vice versa. We should see a lot more DDT inside platform engineering. But let's rewind the movie and think about a little bit where the whole platform story and the interest in platforms comes from, and the realization is that we always see our relationship with technology
2:13
as we are in the driver's seat, and we push the technology right. We make all the new stuff, we make the cloud, we make Clanton, we make AI, right? We are sort of the masters of the game. The interesting part, though, is this is actually a two way street. We push the technology, but the technology also pushes back, and particularly it pushes back with complexity. If we make something that's hard to understand, it will be difficult to adopt. We gonna make mistakes, right?
2:43
It's gonna be basically it shows us where our own limitations are. So it's not just we push take to its limits. It also pushes ourselves to our own limits. And in recent years, funnily, we made it worse because we had the great idea that the software delivery lifecycle is fraught with handover and friction points. Right? Somebody writes down what the requirements are, then somebody else codes to requirements and somebody tests to make sure
3:11
that these are actually the requirements that were written down. And by the time the thing is actually released, we find out that this wasn't really what the users wanted. So we were like, doesn't work very well. So we had the great idea that we can avoid all this friction if we just push everything left, right. And this is where things come from while you build it, you run it, but it's also you build it, you secure it. And in the end, we expect a average developer
3:38
to be a product owner, a business analyst, a domain driven design expert, a security expert, an operational expert. And when they have any time left, they may be writing a little bit of code. So cognitive load has become a big term because how are you gonna find these people now in it for never shy of solutions? Right. So the solution to this problem sounds like this platform. Right. And it's going to make all this cognitive load go away. It enables it doesn't restrict. Right.
4:11
Sounds really good. But how does it actually do that. And that's why I want to dive into a little bit. Right. How does a platform achieve all these things that something before perhaps hasn't achieved. Right. There must be something fundamentally different about platforms, because how else can we solve this magic puzzle that apparently we weren't able to solve before? And the insight? I like simple pictures. The inside comes from a very simple picture of a pair of pictures.
4:43
The idea of having a common layer is by no means new, right? We don't write software from scratch. We have operating systems, we have cloud platforms, we have libraries. We have frameworks. Right? We've always been building on a baseline, but the driver most of the time has been cost. It would be more expensive to build things from scratch. So we use what is actually there. And the broader the basis, the more of this we reuse, the better the economies of scale and the better off this model is.
5:15
Now, for me as an architect, when I look at a model, I always try to think about what assumptions are baked into this and what am I going? Jokes is whenever you see a pyramid, you should be a little bit nervous because in real life we stop building permits about 5000 years ago, yet in IoT they're everywhere, right? A little bit suspicious. So there's an assumption behind here that may not actually halt. And the assumption is you can only do this if you can anticipate
5:51
everyone's needs, because that's how the tip of the pyramid gets so small, because you anticipated exactly what people want, and now they just need to sort of flick a little switch on the underneath the logo here and the website in green or blue and then they don't need to do anything else. Now this obviously comes with some problems. The first problem is, well, how smart are you going to be to anticipate everybody's needs. And B, even if I could I don't want to anticipate everybody's needs.
6:22
I want people to have needs that I didn't anticipate because that's called innovation. This is people making new things. If I anticipated everything, I don't have innovation. I get painting by numbers. So the platform picture looks different. It's not trying to make this little tip of the pyramid. It's not trying to anticipate all use cases. It wants diversity. And rather than just reducing costs, it wants speed and innovation. So it's this doubled pyramid.
6:57
Now the pyramid is always appealing because the economics sound so nice. All you just need to do this little thing on the top. But the base assumption is flawed. So this captures really the essence about what is different now. Funnily, in it we always feel like we invented everything, right? Because we're doing new stuff now. Luckily that is not true. So we have examples to fall back onto. And I'm somewhat famous for car metaphor, so there is a car metaphor for platforms.
7:30
Automotive makers have been using a platform strategy for half a century or longer, and they've followed the same guideline. The goal of the car platform was not to make fewer car models. It wasn't about guardrails and harmonization. The goal was to have more models. I wanted more diversity. Volkswagen is a great example. It started with a golf and the Jetta. On the Passat and the A4, right. They got the hatchback and the trunk. Small variations, but it was very clear
8:05
they wanted more models in an economic fashion, not fewer models. And that is probably the most important reminder for platform builders. It's not about restricting and narrowing and making the tip of the pyramid. It's about opening up more opportunities and having more innovation and more diversity. And that worked really well for the car manufacturers. I once attended a very interesting top from a gentleman who consulted BMW. I forum probably 2 or 3 decades, and he explained this very easily.
8:38
He said when we were kids, BMW had essentially three models. It was A35 on the seven. There was a little bit oddballs, a six and an M1 and so on are different things here. And there, but basically they had three model lines. Now they have about 30 or so. Right. There's 1234567 x1, 23M all the electric ones, the mini. It's 30 plus different model series, each of which comes in a million configurations, different engines, etc. and his summary was very easy
9:08
without a platform strategy that would not be economically possible. So you get more models, more diversity thanks to a platform strategy. And if people can do this now with nuts and bolts, we should be able to do that with software as well. So I like tests, you know, sort of binary conditions. So he has a good test for whether you build something that should actually be called platform, because I we heard in an earlier talk like Hannah mentioned.
9:37
Also one of the big dangers is people rename existing stuff to platform because it sounds better, right. So here's a test that quickly debunks this. And that is if you really build a platform, the users on top should have made something that surprised you, something that you did not anticipate. If this doesn't happen, probably the platform didn't really make the top half of the pyramid possible. So this is what you should aim for.
10:06
The temptation is always there to shrink the top half of the pyramid, because the top half is kind of, oh, it's still custom development. We still need to make a new car model. We still need to build a new application. So there's always the temptation to shrink the top half of the double pyramid, because it seems like extra work. But that is a very slippery slope. And the common effect scenarios already validated this for us, that this doesn't end well. And that's called batch engineering.
10:38
In the US, folks had the idea of all the platforms are great, but we still need to make a new car model. So isn't it much cheaper to take an existing car, put all the options in, and just stick a different label on the trunk? And this is called the Cadillac Cimarron effect. Nobody remembers the car is a horrible car except for the fact what they did is they took a Chevy Cavalier, put all the options in the big engine, the nice radio power windows, the nice metallic paint, blah, blah, blah.
11:10
And then they said one. Now this is a Cadillac. The problem is all the customer says, no, it isn't. It's just an overpriced Chevy Cavalier. So the reminder is what sits on top of the platform is equally important to what's inside. Don't try to eliminate what's on top of the platform. That gives you the innovation, that gives you the diversity. That's what you're actually after. And if you take this away, imagine the picture. If you shrink the top half of the double pyramid, you end up
11:40
back at the pyramid where you didn't want to be. And this has been proven also half a century ago that this does not work. So with that as framing, now it's time to look a little bit closer. What does this mean for it? For in-house and for development platforms? I'm an architect by trade and my passion is architecture. So one of my key insights, or the key points I like to bring out is that to me, platforms are reified architecture. So using a little bit of a big word right.
12:13
But this is where architecture comes to life. A lot of architecture lives on paper as a reference architecture. Design patterns. Right. They're nice pictures I like pictures. Tomorrow we'll do a whole workshop on drawing pictures. But pictures are just pictures. Platforms are able to bring those architecture decisions to life in form of running software. So if your architecture guidance is that you should have microservices resilient, easily scalable kind of things, right?
12:46
That's what a lot of architects say. Well then your platform should support this. It should have the runtime environment, the communication patterns, the observability, the deployment pipelines. Right. And your platform should support exactly what your architects like your software to be. So that's how architecture comes to life. As a piece of running software. And that is much more powerful than just having a piece of paper. Sadly, I see this connection still relatively weak
13:19
with many of my clients and is one of the first things I do. I go to the architects and say, show me your pictures, and then I go to the platform. People show me your platform. And most of the time they're two different things. I'm like, how can this be right? So they should be linked. There's another challenge with platforms, and that is when you draw a high level picture, it looks like a picture we've seen before. Well there's the application developers and underneath is the platform.
13:47
Well, in the old days the picture looked the same except the label was slightly different. Now, obviously this doesn't really make platforms look very exciting. The problem here is not with the platform. The problem is with the diagram. What have we done? We use a static model. We use a box model to express something that is about a speed and velocity and innovation, and that doesn't work. A static model cannot express the advantages of something that's supposed to accelerate something.
14:20
There's no movement here. So if you want to understand platforms, you need movement. And once you plot the software delivery cycle across these boxes, then the difference becomes pretty apparent. Right on the left hand side, you have the classic throw it over the wall and throw it back cycle, right? We throw something in production and then when it doesn't run well, they throw it back to development and that gets pretty old pretty quick. A platform team is not in the loop.
14:51
The example I always give is if you deploy a serverless function on AWS, nobody at AWS does anything pretty easy to imagine because right there would be very busy. Nobody does anything at the time you use the platform. People do capacity management. They do support, they build new features, but they're not deploying the function for you. And that's how you should think as a platform team in-house as well. You're happy to help people and sometimes maybe things are not 100% done.
15:22
Now you're happy to engage. But if you are in the loop, that is an anti-pattern. And if you need a metaphor for this right, you will the axle. You're the grease. You're the low friction thing in the middle that makes this wheel spin faster. And that expresses the big difference. I heard a really nice analogy. It's between being the waiter that's like the ticket based system somebody orders from a restaurant versus the vending machine. The platform is the vending machine.
15:52
You push the button and the thing comes right out. The only difference is in this case, the vending machine serves really tasty food. Now come on, it is, you know, containers, runtimes, observability. So it's a high end vending machine. Now, the double pyramid also means that platform teams serve multiple customers, right. You serve corporate if you wish. So your sponsors I.T leadership, your risk and compliance people, governance, etc. so they have certain expectations.
16:24
But then you're also serving the development teams. And one of the failure modes that I see is platform teams selling the wrong thing to the wrong customers. A application development team doesn't care about reuse. You care about reuse because the more people use the same thing, the cheaper it gets. And total. But not for the project. They don't care if it's faster and cheaper and better for them, they will use it. The reuse argument doesn't matter to them. This is easy to mix up.
17:00
And then people say, oh, nobody understands the value of our platform. But this happens because you sell the wrong thing to the wrong target audience. So be very clear about you have a little bit of this schizophrenic lifestyle, right, where you have the economies of scale underneath and the economies of speed on top. But it also means your value proposition is a different one in the two directions. And don't mix those up because you will just be frustrated
17:29
that you know, people don't agree with you. The other thing, the picture that we just saw with the two boxes means is there is a temptation to rebrand your existing infrastructure and operation teams as platform team, right? Because a it sounds better and b they know the existing infrastructure very well. They know the cloud, they know how to run a Kubernetes cluster. They know the network configuration. So they seem very well qualified, right, to make a platform
18:00
and unfortunately that is true and is wrong. They are well qualified from a content perspective. They know all the pieces that a good platform makes, but they're unfortunately also the worst people because their mindset is a completely different one. A platform team is customer centered. It makes a product, it's self-service, it's transparent, it enables. That is not what traditional infrastructure does, right? You file a ticket, maybe you get the server. It's all about governance.
18:33
It's all about, you know, boxing people in. So this is a tricky maneuver. You want the skill set from those people, but you kind of don't want their mindset right. And you can't split people into sort of two parts. So this is a delicate affair. When you build a platform team, probably you want to mix a little bit from folks from the application side with other folks on the infrastructure side and somehow get this to work together, because otherwise you fall into this, into this trap, right?
19:03
You get the old mindset with the technology that you want. And this obviously then doesn't give you a platform and the way this often shows is that people promise the Gordon path, the superhighway, the high velocity. But in the end, all the budget went into the guardrails because the guardrails you can sell to your sponsors. Right. Remember the the bottom half of the pyramid, that's where you can sell governance and compliance. And because those are the people who fund you,
19:33
what happens is you build giant guardrails. And unfortunately, there was no more money left to actually put the pavement on the so-called Gordon Path. And then don't be surprised that the application development teams are not that excited about your platform, because it didn't actually make their life easier. So if you need another car metaphor, because we use the word guardrails so much, I always remind people guardrail is a last resort safety mechanism, right? When you hit a guardrail,
20:06
you were happy the guardrail was there, but it's also a failure. Your car is wrecked and the guardrail is wrecked, so we don't like hitting guardrails. God, it's under the steering device. Nobody follows the road by scraping along the guardrail. So the metaphor is guardrails are good, but what you really want is lane assist. You want people to stay in their lane so they never actually need to hit the guardrail. The guardrail is a last resort. What does hitting a guardrail look like?
20:37
While it's a late security or architectural review, the project has been going for like nine months. It's ready for launch in a month or two. Then this big security review comes and people say, well, that thing is never going to launch, right? Because it doesn't comply with any of our security guidelines. That is hitting a guardrail. That project is now wrecked, right. How are you going to get this back on track? So the point of lane assist is that teams get early feedback.
21:04
They sit on top of a platform which already incorporates a lot of these mechanisms, so they never hit the guardrail. So instead of guardrails or in addition to guardrails you also want lane assist. That's what modern platforms look like.
21:20
Now we need to think about how people interact with this platform. It all sounds great right? We got the wheels spinning, the axle. You know, the lane assist a lot of nice metaphors, but you need to design the interface to your platform. And we want abstraction because abstraction reduces cognitive load. But abstraction is one of those things right. Sounds easy in reality, not easy to do. Now the main hurdle we have is as engineers who build stuff,
21:53
we have a very difficult time finding good names for things. Right? And another car metaphor is how, you know, if the engineers had named the automobile, it would be called the piston crankshaft, gear, wheel assembly, singleton factories, something rather.
22:09
And that happens because we are so proud of all the things we put in. Right? We tend to look at things from the bottom up, from the inside out, but that makes the worst naming, the worst abstractions. Now, luckily, somebody had the foresight to understand that this thing moves itself so it becomes an automobile. But for us, it's really difficult. As engineers, we want to look at the value that it provides, not what it's built from.
22:39
And this has happened to me many times when we named some of the enterprise integration patterns. We also named them by how they built, not by how old they used. And we later fix that. I know very well how this happens because I've fallen into the same trap myself many, many, many times. I right. So one test of a good abstraction, is it shouldn't leak the implementation, it should define what you do with it, not how it's actually built.
23:11
Sounds very simple and obvious, but again, easily to trip up because you know how it's built, so you need to forget the internals. Classic example is in the US, this thing is largely called gas pedal. Right. And that is a pure abstraction. And that's easily tested by electric vehicles. Right. How much gas is in your electric vehicle? Well, hopefully very little if there's gas in the batteries. That is a very, very bad thing. So the proper name for this is accelerator, right.
23:44
And actually in German, the official name is power pedal metal. So people do know the proper names and it makes sense. It's the thing that makes the car go, not the thing that opens the throttle. So more evaporated gasoline and gas form can go into the cylinder. All right. So this is an easy test. It's the use that it has not the ingredients that go into it. But it's I said very easy to fall into this trap. Another trap that we fall into. And this is related to Gordon Parsons saying default.
24:21
We feel that taking pieces away is inherently making things simpler. That is not true if you look at cloud services. So basically a lot of platform teams go and say, hey, DynamoDB, do DynamoDB has a little bit too many settings. So let me just like default most of them. And didn't I do a great job? Now the cognitive load is less. That is not necessarily true. And my metaphor here is right. If you think a thousand piece puzzle is a little bit too difficult
24:51
for the kids, taking pieces out is not going to make it easier. And it actually makes it harder. You know, the settings have implications on performance, on cost. They're not independent. You can just sort of make some of the defaults disappear. That is not actually an abstraction. It's a convenience, right? It makes some settings easy and you don't need to think about it. But per se, it is not an abstraction. And I see this with the cloud providers a lot.
25:19
One of my favorite so AWS things is where they had, solution patterns and they actually called them abstraction. And the name of the pattern was Lambda SQS Lambda. I'm like, how can this be an abstraction, right? Because if I don't know what SQS and Lambda is, this thing means nothing to me. That's gas pedal, right? 100%. So if abstraction uses the underlying language per definition, it cannot be an abstraction, right? It's just the content repackaged may be convenient, but it's not abstracted.
25:52
So the key thing is abstraction requires judgment. You gotta omit some things. You got to define a different language. Now we have the convenient word essential, right. And of course what is essential? That is up to you as a platform designer, as an architect, when I still believe that is largely human judgment. Right? What is most important to you? And that will vary by your industry, by your location, by the type of business, by the type of teams you work with.
26:25
What is essential and what is not essential will vary. And I think that is a good thing. This is your decision, right? This is your judgment. What do you consider essential or not essential? So if you overdo this and you make things simpler than they really are, you end up creating an illusion, right? You're sort of portraying something to the developers that actually doesn't hold very tempting to do because you made it simpler for them. You made it easier, but it doesn't match reality.
26:59
So that comes back to haunt them. That will not last. Classic example is our PC. And you know, now we have gRPC and I think RPC has become a little bit more acceptable. But in the early days of RPC it was an illusion because the name already says remote procedure call. Well, it's just like a procedure call, except it happens to be remote. The reality is the remoteness changes everything, right? The remote properties, each a call has all the properties that a local procedure code does not have.
27:34
It has high latency, many different error error kind of conditions like server unreachable. It has partial failures. It has inconsistencies. It has retries its item potency. Right. Basically a whole new universe opens up. That is not a small variation on a method call. So this is one of these kind of illusions where we tried to make it simple, but then it came back to haunt us. And getting this right is quite delicate. When I worked at AWS,
28:06
we debated about this a lot because we want to make things simple, but I always upheld said some things just aren't that simple and trying to make them simpler than they really are, I don't feel is doing the development teams a good service because you cannot tricking them, right? It's like, oh look, this is all easy, but in reality it it isn't. So it's a little bit of a Goldilocks game, right? If you don't take much away, if you don't define a new language,
28:34
we are you're not reducing cognitive load enough versus if you have sort of the one magic parameter I always joke about. Right. The ultimate abstraction is one method. It says do the right thing and you just need to get the boolean parameter correct, set it to true, and magically everything is done and people feel like, oh, that's the best abstraction ever. It's like, no, because nobody knows what's going on inside, right? That is actually not right either.
29:01
It gives you some sort of illusion and the latest at runtime. You run into issues when something goes wrong. I have a famous saying that says failure does not respect abstraction, right? When something goes wrong, all the reality bubbles back out and then nobody has any idea what is actually going on. So this magic method kind of thing is not where you want to be, right? You want to be somewhere in the middle where again, it's your judgment,
29:29
what you consider essential and what you don't consider essential. There's one easy trap to fall into, and that is when people abstract something away. Developers always feel like they're not telling the whole story. Right? Like if I take something away, which I need to do in order to get an abstraction, we feel like, are we kind of cheating a little bit? We're not telling people everything. We're hiding something. But what's really important to keep in mind, a good abstraction
30:01
allows people to see things that they otherwise would not see because you take the noise out. So taking things away actually allows teams to see more. And I think that's very important for developers to understand. So we're not stuck in this hesitation of like, oh, this isn't quite telling the whole story. You know, getting the noise out brings clarity and people actually see more and understand better the decisions and the trade offs that they make.
30:33
So in this case, a little bit less is actually more because you bring clarity, right? You have less noise, less vocabulary, but more clarity about the kind of systems that people really built. But careful.
30:50
Again this is sort of the temptation of the single method. You can't measure cognitive load by the size of your API. Right. And then the classic example is do the right thing. You believe that has no cognitive load because it's one method. None of the cognitive load is actually huge because you have no idea what you're actually deciding. Maybe this thing is crazy expensive. Maybe it doesn't do what you need, maybe it doesn't scale. Maybe it doesn't run on the infrastructure that you need.
31:19
So don't mix up cognitive load with the size of the API, right? You don't want it to be oversimplify it. That's where the illusion comes from. But you want to make it intuitive. These are two different things. And as I said at AWS, we debated about this a lot because I've worked on the serverless team. So it's distributed asynchronous event driven systems. And my general manager always said we need to make it simpler. I'm sorry, it's not simple. Distributed systems are not simple.
31:52
I want to teach people about versus push versus pull and queues get full and time to live. And all that pressure. Like I need to get people to understand these concepts. Otherwise they cannot design a distributed system solution. So is a constant argument that we have. And I feel you should strive for something that's intuitive, something that makes the complexity approachable rather than pretending that this complexity doesn't exist because that's where you end up in the illusion.
32:24
The idea of not measuring our cognitive load by the size of the API is actually not new at all. We didn't have the term at the time I wrote a chapter in there, and it was, you know, basically, the chapter was called API design is not a convenience. It's not an elegy. I called it right. People said, oh, it's more convenient if I have just one parameter. And the semantics of the parameter changes sort of magically. If it's like a little string, then it takes the parameter.
32:56
And if it's a file name, then it reads from a file. Isn't that convenient? Because I only need one parameter like no, this is like the worst cognitive load that you can have. Because what if your file has a name that sounds like a literal string is like you are in a world of hurt. So some 1520 years ago, we already knew reducing the number of methods and reducing the number of parameters can actually dramatically increase the cognitive load.
33:23
Because you end up with this magic settings like do something one right or two right, you have no idea. The cognitive load is actually very high. So in platform design this plays out very, very much. We are so tempted to make it simple even though it isn't. And then we fall into this trap of magic parameters and magic arguments that nobody actually knows what what they mean. I heard one real life story where a platform team had a lot of checks
33:54
about what is actually compliant to be deployed, but it was in transparent. So the development teams were trying to actually deploy something. It would come back. It's like, oh, this is not compliant, right. So in the end, but it was completely transparent. So the cognitive load is off the chart. So what ended up happening is developer teams just ended up putting random parameters in everything until the damn thing just somehow deployed and don't care at all. That's what you get.
34:20
And I would not even blame the development teams for this, right? You have work to be done. If there's something in your way that you cannot understand because it's undocumented and transparent, he just fiddle with it until the thing goes through. Of course, this is exactly the opposite of what you want from a functioning platform team. So expose the inherent complexity that is there now coming to the domain aspect. And and you can already tell we're dealing with a fairly complex domain.
34:50
It's a domain of distributed systems, the domain of cloud services. Right. The domain of deploying software. That is a domain. When we think domain driven design, we generally think about our customer's domain, which think about banking and shipping containers. But our technical domains have become equally complex that we actually should really think in domain driven design about our own terms and our own technology. And as you'll see in a second,
35:21
I believe that holds the key to good platform design. You need to understand your domain in order to build a meaningful platform. So the key challenge I posed to clients on teams is don't try to invent a better DynamoDB. Don't second guess the cloud provider, right? They're pretty good. They're pretty smart. They have a lot of customers, right? Second guessing their API is a losing proposition. Plus it evolves. So what you should do is you should make services that the cloud providers
35:55
fundamentally cannot deliver, because that is value out, right. You do something that they cannot. And any say, well Gregor smarty pants, that sounds really clever, but how do I do this? Obviously the cloud providers have a lot of people, a lot of money. Right? How can I make something that they cannot make? And the answer is your domain, right? You gotta understand your domain. So if you consider your platform just sort of another cloud API,
36:23
if you consider domain of the platform to only be the technical stuff, you can't do this because you just be second guessing the cloud provider. So you need to bring your business domain together with the technical domain. And then you can make services that the cloud provider cannot make a great example of sort of lifting something up, making a real abstraction. Finally, are operating systems right. Operating systems did not give us same defaults.
36:55
They didn't say, hey, I make reading this this record easier for you by defaulting the drive letter to zero. Well, that's not how operating systems came about. Operating systems give us a different vocabulary. They give us value added services on top of the infrastructure. They give us streams. We read from sockets, we read from file streams. And, you know, the disk has no idea what a directory in a symbolic link in the stream is. Right? That's what operating systems
37:24
and library do is like, get next byte right, fetch next line. Those are real abstractions. They give you something that the underlying layer fundamentally does not have. So if we bring this back to our in-house platforms, the services that the cloud provider cannot make, our services that are specific to your domain. And so, for example, you can have a database that is configured to host customer data, PII, personally identifiable information.
37:55
The cloud provider can give you all the bells and whistles on how to configure that database, but only you can decide what that configuration is. What encryption policy do you need? What replication policy do you need do restrict it to certain regions, right? Can you have consistent or inconsistent reads? Can you have dirty reads? Do you have to be consistent? All of these kind of things you define and the end result is not a better DynamoDB,
38:24
but it's a database that is suitable to hold customer data, and it will be more expensive than the other database that you provide, which is a database that cannot hold customer data or real customer example ledgers. Database a database that tracks all the changes that are being made and that is a value added construct. It's not a better database, it's a different kind of database. And what do you notice on the vocabulary here?
38:55
The names of these things use terms from the business domain like a ledger. That is a financial construct. This was an insurance company who did this right to them. Ledger means something. Ledger is sort of an add on a record right. Of everything that happens, we kind of well, our technical term is change data capture, but the platform calls it a ledger database because they bring in vocabulary from the business domain together with a technical domain.
39:23
A database is a technical domain, ledger is the business domain, PII is the business domain, database is the technical domain. So the platform lives at this intersection. And that's where the cloud providers cannot live because they don't know your internal guidelines, your regulations for your country and your industry and your preferences. This is something only you can do. And this adds real value. All right. So here again
39:50
so there's the the business domain I call this the business technical domain. You're not modeling the business itself. So this is not about shipping containers and banking accounts. But it's where the business domain meets the technical elements. So it's a customer and a database. It's a ledger and a database. And that requires you to understand your technical domain and your business domain, and to understand domain driven design and domain modeling. Right.
40:19
You need to now express these kind of things, give them names. And to me, this is very much dee dee dee. And without this I see platforms just being sort off. I call it the grim rapper, right? People just raps out of the cloud, calls it doesn't add a lot of value, is difficult to maintain. So in my experience, this is where you want to play. You want to play at the next layer up for your business domain and your technical domains intersect. And you will find this is a very rich space.
40:50
There's a lot of stuff going on here. So your platform can add a lot of value, as opposed to just sort of second guessing the the database API. There's like a handful of footnotes to this. The one thing I mentioned is failure does not respect abstraction. All right. So let's say you make this magic customer database. It has a lot of certain settings. And it maps to a certain stuff. Now if something goes wrong the failure will always occur in the runtime.
41:23
Underneath right is the cloud services that you provisioned. Your abstraction is kind of gone when the failure happens. So I built something that we call it cloud compiler, something pretty clever where you design your serverless pipelines in a high level language. But of course once it's deployed, it's all EventBridge. SQS has an S on that, right? Because that is the runtime that is basically the virtual machine you have underneath.
41:48
So if something goes wrong, the error will happen in one of those elements. Now luckily this is a problem we've known for a long time, right. Basically every compiler has line number mappings. Now look at Java right. The Java runtime also creates code that you don't write. If you don't write a default constructor, well the compiler will make one for you. And it's rare that the default constructor fails. But if it would, it has a line number link back to the class.
42:19
So make sure you abstractions work both ways. You can do this with tagging, for example, right is not easy, but you must understand that by the time something goes wrong in operations, the abstraction is no longer there because it's all just cloud services. So you gotta have a link back to the abstraction. Otherwise people don't know what came from where. So this is like most compilers do it. And I actually blogged about this some some time ago.
42:48
So platform teams also need to think about both ways. The other trap that's easy to fall into is the logical abstractions cannot overcome physical limitations. I have that argument a lot. There's a lot of sort of cross cloud kind of things, right. And portability frameworks. That is really nice. But then it comes to service availability in different regions, feature availability, availability, cost quotas. What if you exhaust a quote on that cloud provider but not on this one?
43:18
No abstraction is going to help you, right? So you need to be very careful what you can abstract and what you cannot abstract and cloud providers are very much about physical limitations throughout this cost, this quarters. You don't get all services in all regions. No abstraction will help you. So keep that in mind. And the last one is again a very slippery slope. And that is your platform will naturally grow because you want to be
43:44
customer centric and your customers will come to you and say, well, I could use your platform if it only had this feature. It will happen all the time, so the platform will grow and that is dangerous because you can easily lose the cohesion. You can lose the composability. So always kind platform teams to pay attention to how easy it is to put something together as opposed to you. How much is in the platform. The natural reaction is to overindex on size and compromise the composability.
44:19
And that is very, very hard to undo. I call this the legal principle. The capability of one Lego block is very minimal. It basically does nothing. But the composability is awesome, right? You can make anything out of it that you like. And I think this is a good takeaway for platform teams. Don't be tempted to just pile more and more stuff on it. Make sure that people can actually put the pieces together, because if it's not composable, more capability won't help you
44:48
because he can't put it together anyway, right? It's like having more colored Lego bricks, but they don't fit. Doesn't help you, right? So protect the composability of your platform because you always be tempted to slip the other way. So only two closing thoughts. And the one. Of course, there cannot be any talk without AI, right? Otherwise nobody would have shown up. So the question is all this abstraction, all this thinking, all the stuff I talk about doesn't just journey.
45:16
I do this for me, right? Just like it generates all these APIs. Even if my API is not so great, you know, it just generates the code to use the API. And don't all my problems go away? There's certainly a lot of value in the code to sustain the compiler notes, but I was very clever here. I actually asked AI this very question and it says, well, basically does Jenny I replace the need for platform engineering and API design? And I itself gave me a very nice answer and says, well,
45:46
you know, it's useful, but it still requires human expertise. So I agree with Jenny, I in this case, I think this was Claude who gave me this answer. So I agree with Claude that he has used the tools as a booster. But domain driven design and abstractions defining what is essential means is very much a human activity. So I don't think Jenny is going to take that activity from us. And then the only last answer I have to give you always, don't always believe you eyes right.
46:17
This impossible object, this kind of abstraction is actually real, right? It just plays with your senses by sort of making it look outside in when it is actually inside out. So some illusions can actually be fun, but not in software engineering. You don't want illusions. So with that, I thank you for your attention. If you haven't read the book, a lot of these kind of ideas, the stuff I talk about either on my blog, in the book, it's a holiday season.
46:47
So if you want the coupon for the e-book, you're welcome to use this as well. And I thank you so much for your attention.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat