Skip to main content
Loading Explore guides…
Software Eng · Acedra
Home
Explore
Computer Science
Software Eng
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
136 segments
Copy
Hide video
0:00
Hi everyone. My name is Patrick Akil and if you're interested in Mastering Software Engineering as a craft, this episode is for you. Joining me today is Joris Kuipers. CTO and hands on architect over at Trifork, and he has over 25 years of experience in this industry, more than yours and mine combined, potentially. He walks us through what to focus on when you're early in career or already senior, with regards to how to master new topics,
0:25
which environments to be in, to thrive, and much, much more. So enjoy the soft coding. In essence, you've been in the industry for a lot longer than me, and I think for for a lot longer than most listeners as well. What still gives you energy when you're creating software, when you're delivering things?
0:44
What I one thing that I still like, and that's also why I like being very deep into a tech stack, is to actually have a feeling of mastery, where if you do get a new problem, you don't feel overwhelmed or it's like, oh, man, we were going to do this. I have no idea. But you actually really control, you feel in a way that you can't say, okay, I understand what you want. I can plot out an alternative so I can think about the different pros and cons of different alternatives, then make it work.
1:11
And then when you do that and it actually works, that's still something that I find very rewarding. I always joke at work that, my methodology is, guy. But first time. Right? I like to build something that just works rather than have to reiterate and fix and fix and fix. No, this is not the way. And you can only do that when you when you have a, like, a deep understanding upfront of what you're going to do, what you're going to encounter, what the pros and cons are,
1:34
because in the end, what I, what I like to do is on the one hand, I like to build custom software. That's what I've been doing for a very long time already, and I still enjoy doing that. But I also like to do it in a in a way that for customers, sounds like, a real benefits rather than something that just costs a lot of additional money because they couldn't buy something off the shelf. And that's very often the perception of the enterprise custom software.
1:58
It's oh, yeah, it's just going to be expensive because it is expensive to do something that is completely custom. Yeah. So that's that, feeling of mastery where you can quickly show that you can deliver real business value and at the same time, know that the way that you've done it is not like taking shortcuts or doing a hack, but to do something proper that you will be able to maintain for a longer time as well. For example. Yeah, that's still something I really enjoy.
2:23
I feel like that like the idea of mastery. Okay, I love that. But also in software engineering, there's so many new things that like come in and out. How do you still like have this confidence of mastery? Because even for me, I've done this for like eight years now in the industry. I don't feel like I've mastered this craft. I feel like new stuff always keeps us true. Yeah, it's what I like in general as to have to drive the direction of where I need to learn and where I need to evolve is to to
2:51
to have real, real work or real problems that I need to solve. So this has been a problem, for example, for myself in my previous job when I was working for Spring Source. Yeah, as a trainer and consultant. And this would always be, when it wasn't training, the consultancy was typically short assignments, couple of days with a customer that had some particular issue that they were struggling with, or they wanted to learn about something that I didn't understand yet, and I would help them out.
3:15
But when you do that, and you do some trainings and then every now and then you have like a week where there's downtime, you can just start studying stuff. There's nothing really to tell you. Okay. So out of the 5000 things that I could start to research now, what is it that I'm going to do? Because well you have your personal interests, but it doesn't really make sense to only have your personal interest drive your professional development. It should come from professional requirements.
3:41
So what I really like about my current job is that, most of the stuff that I researched is actually driven by real requirements. So if I'm, for example, I'm currently working for, and they lost a lot and that does a lot of research for a couple of years already where we are building a big integration platform. So the typical concerns, there are stuff like, making, robust integrations to external systems where we have AI impotency and retries and circuit
4:10
breakers and observability and all of that stuff in place. It's about, orchestration, making sure that the when we have business processes that span multiple backends and something goes wrong that we can correct it in a way that everything is still consistent and that automatically that focus gives me a way of, okay, so this is what I need to learn. I need to dive into, what tooling is available to make sure that we have stuff like, circuit breaking in place.
4:35
I need to look into observability to make sure that not only do we have logging, but we have metrics around all of this, and we can do dashboarding and we can see what's happening because we have over 40 backends that we're actually integrating into this platform. So it's huge. And that yeah, that sort of automatically gives you a way, a way forward. We're running stuff on Kubernetes. So that means, okay, I need to learn about containerization, how to effectively do that with my applications,
5:02
the way that you package up a spring boot application, for example, in a, in a Docker image, makes a big difference in how it actually performs. And I would run, and I would build. So also you don't want to like, have to rebuild an entire layer just because of a single code change. So yeah, I usually try to let the, the, the requirements of, of real projects, of real customers drive what I'm actually getting into. And. Yeah, that has been working out quite well for the last 20 years.
5:30
I've been, I've, I've been doing something similar where if there's a certain problem at a customer with specific technologies, that for me is the opportunity to dive deep in those technologies specific. Now, I don't have this inkling or this interest to go deep by myself to just various technologies, Excel, or if I'm going to apply that similar with me. And sometimes I that could also be tricky, right? Because you're only learning the stuff that happens to be in
5:52
just like a chicken egg problem. No. Exactly. So for example, recently I was a colleague. I was invited to, an event or a seminar around quantum computing. And this was one of those areas where it's like, oh, this is completely foreign to me. Yeah right. I need to I don't know anything about this. And I know people also in the Java space, that has been looking into what is quantum computing. They've tried to make it digestible and explainable to, two people.
6:18
And I find this sort of interesting, but it's indeed not the thing that I'm just by myself. I'm going to be spending a lot of time on because, I am very focused on, professional engineering. That's always what I've been doing. Before I got into college, I didn't want to study computer science, okay? Because I thought it's just going to be sitting behind a computer, and it's probably not for me. Yeah, so I did, I, but I did I in 1994 to 1999. So it was 20 years too early. You're an AI expert.
6:46
I was most certainly I'm not anymore. No. That this was traditional AI. Mostly it was at the Free University in Amsterdam. It was expert systems, that sort of stuff. At the end, there rebranded it to multi-agent systems, but it was still expert system like that. So that's what I did. And then, my first job actually was, the R&D department, for a show for engineering, geek. And I thought, I will be nice. I will learn some more server engineering then I learned in college,
7:13
and then I'll go back into AI. But I never went back into AI. But it means that my my drive to do software engineering to learn about it has always been professional. It's never been like, oh, I'm just really into computers and I want to learn all the languages or I want to learn. It's always been, okay, we're going to solve this issue, and I want to do it in the best possible way. And for that, I need to understand a lot. And when you start with enterprise Java in 1999,
7:35
you have to learn a lot of stuff just to build a very simple system. Yeah, I think that that's also part of it. Right. I've always been in this situation where you need you need to have like this super steep learning curve. And then only once you get past that you can start to do something. So that's been becoming sort of a habit for me. It doesn't matter if I need to learn a lot, I'll just dive in until I really understand and feel like that.
7:56
That skill, that essence is what is going to make people successful. Software engineer. Yeah, I think it has to. You have to. It's a combination of being curious but also being persistent. Yeah. Right. Yeah. So a piece of discipline and also kind of comfort in being uncomfortable. Yeah I like that you highlight that like disposition that you have of mastery that gives you the comfort. But it doesn't take away that there's a lot of technologies that you have to learn.
8:19
Yeah, yeah. And that's consistent. That's never going to go away. No. Exactly. Yeah. Yeah. What would you advise for people that are like trying to master their own craft? Because I feel like if I were to start out now and it's very hard to relate, but there's a lot of technologies. Yeah. I do think that like, software engineering is a very good core to have. And if you can be a really good software engineer, by essence of you being responsible for creating software,
8:40
you're going to think of observability, you're going to think of infrastructure. You're going to touch on topics like data. And I in cloud, like I feel like software is the center of it. So it could be a good start to start with. But then how do you start and what do you master first or. Yeah, I think I think the the start needs to be with the basics in terms of the language that you're using. You should just understand the language very well.
9:01
I still see a lot of people programing in, in, in languages where it's clear that they don't really grasp a lot of the basics still. And that means not only that you're limited in what you can express in your own code. It also means it becomes a lot harder to understand other people's code. So one thing that taught me a tremendous amount of stuff was reading other people's code. And not just anyone, but for example, the open source frameworks that you use
9:25
that you know, have been authored by experienced people that have been taught. So spring, in my case, is not only a framework that I know very well because I use it. I've actually, read a lot of the code, to understand how it works. But then that, that taught me a lot of patterns also, and this is the way that they approach is, hey, this seems to be the similar type of problem, but in a different domain. But I see that the approach that they take to solve it is similar.
9:49
So you learn about these patterns, these use cases. I would start with that, because it's going to be overwhelming if you're going to try to if, start as an architect that needs to think about everything. Right? You cannot think about, okay, this thing needs to be coded. But I also need to think about the build, and I need to think about the CD because it needs to be deployed to an environment. And then I need to think about how will it actually run in that environment,
10:13
and how are we going to make sure that it is that it is open, that we can observe it? Yeah. And then I need to it's just too much. So I think a very natural way for most developers will be to start with just developing the functionality and leave all of that other stuff built, runtimes, operations, leave that to other people. And then over time,
10:32
in my case, not even on purpose, but just how it goes, you will you will learn about that stuff because you need to pick it up because someone else is not there to do it for you, or because you find that you are indeed interested in it. In my case, I'm sure if ability has been something that over the last seven years I spent a lot of time diving in just because I really enjoy doing it. It turned out that stuff like building dashboards based on metrics,
10:54
is something that once I learned that it could be done with, because this is not something that was really available ten years ago. But now with all of the tooling that's there, it's it's quite straightforward. Yeah. I find it really enjoyable. So it was again, a combination of okay, I see that this stuff becomes important. It it provides real value for customers. You can also see things also business metrics, not just operational metric. But it's also interesting to learn.
11:17
And there is a lot that I don't know yet that I think I can get into. So yeah, just start start with the basics, learn it properly and then take it from there I would say. But learning it properly as I think is indeed something that I find really important. Yeah, I do see a lot of people skipping because they, a lot of people I think are also asked to be not experts in one thing, but to, do a little bit of everything. And when you start as a junior and you don't learn anything deep,
11:42
but your attention is just spread across everywhere, it's really hard to, to do this. And you see that with people, for example, who are working in, two people, teams at some startup. But if you start there as a junior, you're, you're supposed to do everything. But, so you do learn a lot, but you learn only in a, in a very broad sense. Yeah. You don't ever go deep. That's how I started. Yeah. Like I started in ops, and I got to touch on a lot of systems, and then I moved
12:09
more to software engineering, but it was also front and back. It was immediately kind of the whole stack and cloud. Yeah, exactly. Thinking of a lot of things. And that's I think now if you want to do that nowadays, it's just you cannot go deep. You're going to be overwhelmed. Yeah. You're going to be overwhelmed. Just look at I mean, front end nowadays is even that in itself to keeping up with what's quite a beast. It's huge. Yeah, it's huge and it's happening. It's happening so quickly also.
12:34
So just catching up, making sure that we had a, discussion yesterday. Someone told me that, they were somewhere where they were still working on an AngularJS angular one. A well, for example, that's the sort of thing like, from that is no different in that sense. From backend. You have legacy there as well, and stuff moves really quickly and you need to understand why you're doing things. But again, with AI, it's also easy to just say, well, you know, we're using some stack, it involves react.
13:01
And just give me something that seems to work. Yeah, it's a quick win. It's a quick win for now. Yeah. It's and it depends a lot on the one of the other things apart from like mastery. One of the things I think that really defines how I work and what I find important is, is also pragmatism. You need to understand always the context in which you're working. And, sometimes we work or actually quite often we work on big systems that need to, support mission critical processes
13:29
that need to be, developed and maintained for many years. So then a lot of this stuff is important, but you also need to recognize that maybe now I'm building something that will be replaced in year. What for? Now, we just need a solution. You shouldn't apply the same principles then. I think that's also very important. How do you handle those conversations? Because I've been in conversations with regards to pre-sales and customers
13:50
that I have people where their discipline is like cloud infrastructure. And I can see they look at the problem through their lens. Right? If it's AWS, they go DynamoDB. We can run it there and like those immediately. Those infrastructure components come in. And then I come in with regards to, okay, do we have metrics on which users use which components and what's the traffic. And then can we go just with this component and then live as fast as possible and and build from there.
14:13
Like I have a very different approach. And then we have the conversations. If you get defensive and I call people out when they get defensive, I'm like, I'm not. I'm not trying to attack your judgment or whatever. I just think this is what we need and then this is good enough. And they're like, well, but this is robust and scalable. And if you like, those conversations for me are always challenging. It is challenging, but I think it is the way to do it. You need to have that conversation.
14:33
I was not my previous job, but the job before that, I started to work as an internal consultant for the, the Nate a bank. Yeah. So that's in English. That will be like the Dutch bank. But they're not an actual commercial bank. They are actually a government organization that governs all of the banks and other financial institutions in the Netherlands. And this was one of those traditional government organizations where there would be like every other week, someone would celebrate
15:02
their 25th anniversary within the company that people had been working there. Unbelievable. Forever. Yeah. And that was true also for a lot of the developers. And I started to work there when I was around 25 years old. Okay. And very fresh. Yeah, very fresh and basically my job was to tell those people how to work or how to start working in a different way. So that, that felt, that felt exactly the same, like, okay, so now I need to start to persuade people
15:27
with 20, 25 years of experience of what they're doing in listening to me. And, and again, the way I think that that I managed to do that because it turned out that it was possible and actually quite successful to do that is to, make sure indeed, that you don't just tell people we're going to do it this way, we're going to do it that way. You need to convince them, and they need to also be on board. But you do that with arguments. You always have to explain your reasoning.
15:52
You always have to be able to say, I'm thinking about these and these and goals. Therefore, I think that these and these things are important requirements to take into consideration. But these other things are not, based on this, I think that we can go this or this way and also have a way to verify the results. So what what you're saying, for example, what I really like is get something out as quickly as possible. If you have that, it's much easier to iterate on something that works
16:21
than to work for a very long time on something where you don't really see the intermediate results. Right? This is why big projects fail. Yeah. Very well. Still in it. Everyone just says, yeah, yeah, we're 80% there. And for months. For months. Yeah, exactly. Incredible. It doesn't work. You need to have something else. You need to have something working. And you can almost always do that. I think. But you need to think about how to do it. So then you can show people that stuff works,
16:46
and then you can gain their confidence and they get on board and they become enthusiastic because they see, oh, actually, this is a much nicer way of working than what I'm used to. But this baby steps. Yeah. And, yeah, you need to, that sort of consultancy aspect of getting people on board and, open to change is tricky, but can be very rewarding if it works, I think. Yeah. I feel like it's it's a skill that is quite interesting
17:09
because you need to be able to communicate properly to different audiences, upper management or your peers, software engineers specifically, and maybe even more so, people in tech have this bullshit detector. So if you're selling them something that they don't believe in because you don't have that depth. Yeah, then it's also a problem. You can't just be great at communicating. You need to have deep knowledge. But that's that's one of the reasons that I like my, my current job
17:29
is that I get to do all of these things at in a single position. Yeah. Because, I like developing, but I don't want to be a full time go. Yeah. No, I want to be a hands on, but I don't want to do it all the time. I want to have also be involved with, customer discussions. I want to have, like, an account manager like role with my customers. I want to be able to do this sort of, mentoring and coaching. I like still delivering trainings as well.
17:54
And, Oh, my current job allows me to actually combine all the all of that. I do all of that. Yeah. It's I don't I don't get to choose. And, that's also laziness. Maybe sometimes I don't really make very deliberate choices in my career. That's never been the case. It's always just happened sort of to me. Yeah. But what I really like is that it does allow me to combine all of these different things and a, sometimes I do actually interact with each other
18:18
and then answer each other in a very nice way. But if you if you're used to delivering, a lot of trainings, it means you are used to not only explaining things to people, but also just to speak in public. So it becomes easier to become a public speaker. When you do that, you'll learn that it's actually important that you know what you're talking about. But you don't need to be an expert at everything, right? Very often people first start with public speaking. I think, like, oh,
18:41
I need to be a super expert in order to share something useful with people, and otherwise they will just know I'm bullshitting.
18:48
But this is not the case at all. But that's the sort of stuff I think that if you do a lot of that different things. Yeah, you you'll build up a broad, but also something that people talk about this deep, real far. I actually thought about this before going into this podcast, like because you had a question like what? What sort of defines how you work? And I think it's not my not my goal because I don't think it's a deal of my ambition, sort of is to say instead of like a D like profile,
19:11
I actually want to have the the top of the team, but then then it needs to be brought in to be both broad and deep at everything. Yeah. And you can't, but that's why I think it's so nice that you can get focus from the work that you do right now. I don't need to know everything about everything. I need to know everything about the stuff I actually use on a day to day basis. Yeah. And even then, everything is probably a bit much. But, now I'm still.
19:35
It's very often people will come to me before they Google. Right. That's, that's a good sign. Yeah, that's a good tactic. Yeah. Yeah. For me. So the the depth is always something that I, I think is interesting and I can go deep based on customer problems. And then I always have this kind of internal struggle because should I need to know this in advance or is this finding can I go deep. And then what is time that is too long before I reach out to colleagues
19:58
and like ask for help and stuff like that? Yeah, that's also something I like to, explain people. Or when I coach developers. Indeed. It's, always spend some time for yourself because that's typically where you learn the most, right? If someone just provides you with an answer, do this and it works, then yeah, you don't. You are learning why or what you could have tried, but you also don't want to have someone, especially when they work remote, for example, who just struggle for a day at home.
20:22
And it turns out that there was some small thing that that you could have solved with them. And like in 15 minutes. So I think it's always good to get feedback and ask questions, but always make sure that you, you know what you're asking, right? You don't just go to the first error, or something doesn't work and you immediately say, can someone help me out? Now you first look into it and you think, oh, maybe I can't understand it or I can't.
20:44
And then once you have an understanding of the problem but, you know, not fix it, that's the time, I think, that you can ask for for help, but always try to make sure that it becomes a learning experience where ask for help instead of just someone saying, but it could be stupid things. It's all right. I had, even my, my CEO, who is also very technical, actually. Yeah. Just before his holiday, he wanted to get something fixed and he got to me on slack. It was like literally his last day
21:07
before he left and he said, something is off, but I don't see it. Maybe you see it. And you needed to change your get URL from SSH to Https and you forgot to replace it. Go along with the slash and yeah, that's the sort of thing I can take a half a day if you don't see it and you're just struggling. And that's that's why you need help with that, why you need to ask someone else real. You shouldn't be working on that sort of stuff for for too long.
21:30
But if it, yeah, if it really comes to, yeah, understanding the technology that you're using, it's worth spending some time on. And that's what I find really important. So also when I have people on my team, that are junior or mid-tier but not senior yet, I always give them some time for this. I always tell them, don't stress out. Right. If you don't get something to work or if there is, it would never end there. Just spend some time on it.
21:54
Just try to figure it out and then explain to me what you've done, and especially if they're able to explain it, and you can see if they learned anything. Yeah, yeah, I like that. You put a lot of importance on also people's own kind of personal development and growth within a team. I feel like that's very important, not just to be part of a team that is kind of continuously learning, but also for there to be room and space. Yeah, but that last part, usually when there's high
22:17
pressure, people are looking towards business outcomes faster. You might not be even know optimal team size. And then yeah pressure gets high. Yeah. But there needs to be kind of a fine balance between learning growth. Yeah it depends a lot indeed on on how your project is run, how it's being perceived to run. So yeah that's true. And I'm in the lucky position, that the customers that I'm working with right now have, a very deep trust in what we're doing.
22:42
And working more like a partnership relationship in there with them rather than a vendor supplier, sort of that, role. That's the best. And that's. Yeah, that's ideal because you can have open discussions. You don't have to, upfront explain everything that you're going to do just so that they can trust that you're doing the right thing. It also means that,
23:02
I think from six years ago, or maybe even more, we stopped doing estimations for, for stories, for individual stories, of course, we estimate like, oh, we're going to try something. I ask you, how how big is this budget high over. Indeed. Yeah. We had to, for this integration project. So we started traditional scrum. So we did, the whole planning poker thing. And in the end, someone, from a partner company actually said, this doesn't make any sense.
23:24
You're just spending a lot of time talking about stuff where you don't know what it's going to be anyways, because integration is a really tricky domain. You're always depending on other partners on other companies that need to set stuff up. Can we even access this external system already? Do we have credentials for it? Do they have documentation? You're going to run into problems all the time. Everything gets blocked and then you need to pick up something
23:43
while you wait for that to become unblocked. And then in the end, every estimate becomes like, yeah, well three points, it's, it's it's it's waste. And, in the end, what you want to do is you want to reduce that waste. Of course. So, we have a very well, I think agile is sort of a, that work, but we have a very flexible way of working as work on the lifestyle, where we do take some of the aspects of scrum that we like. Like, every three weeks we do three week sprints or iterations.
24:10
We still do, presentation and then the demo, just so that everyone is aware of what it is that we're doing. It's also a nice point for us to show to everyone in the team what we have been doing, because not everyone might be aware, we do daily stand ups as well. But we don't do stuff like in the estimation, we don't have burn downs. We also don't worry about. Oh, we thought we would have finished this in three weeks, but. Well, it turned out that it got blocked and now it's not finished.
24:37
Okay, then it's not finished. It will begin when we get ready. Yeah, we'll keep going. And, I really like that way of working rather than the, the real traditional way of doing scrum, because I've seen that go wrong also on our, some of our own projects so often that's a I don't really want to do that anymore. I feel like it's, it's similar to what we discussed with regards to kind of being really good at communicating and then having depth, knowing what you're talking about,
25:01
because the communicating part, a lot of agile coaches that I've seen and perceived, they have that down. And then there's the depth missing on why are we. Oh exactly. Points in the first. Yes. And it's it's not just an IT problem. I think it's a problem that's happening everywhere. That's for example, there's not really like in how what he writes and what he thinks about is the bear to bear in the Netherlands. He also has a in English, social media accounts,
25:25
that he publishes on a lot on various things. But one of the things that he's talking about, and I see this in it, but I see this in other things, is that within government, for example, there is no more expertise. Government used to be an expert in a lot of things. For example, in, all of the bridges that needed to be maintained at the dikes, they would have people themselves that knew how to do this. And over time, all of that has been outsourced, has been going away.
25:51
And now they don't even have the expertise anymore to verify the results of the work that they outsource. That's a problem. And that is a real problem. You're at. But it's, I think something that has been seen in many different areas. Right. It's in governments. But you saw the same thing in health care. My, my mother has been working in health care for decades, and she saw that same thing happening there where managers were just thought to be generic process managers.
26:15
Right. You don't need to understand the process you're managing. You need to just understand management. And that happens in it. It happens in healthcare. It happens in the government. It happens everywhere where there's this notion of, yeah, you have to understand what you're doing and be good at it. It's just something that is perceived as, yeah, but we can just outsource death, right? We just need to manage contracts and then have regular goals about are we still on schedule?
26:38
And that's the management part of it. And that doesn't really work in it. I think, that's also why I think as a manager, I'm in the empty of, of our company. Yeah. You don't necessarily need to be like the best experts, at everything. But you definitely need to have the actual hands on experience of what it means to be in it. Otherwise, I don't think you can. Same thing with people who are in sales, right.
27:03
It can be tempting to think that, well, if you're a good salesman, you can sell everything, but then indeed, you're going to end up like you're saying with these people who have their bullshit detector. And that's, oh, they're just trying to sell me. They don't even know what you're selling, especially if you're trying to sell custom software development, which is a really, as we found out, that really tricky service to sell.
27:23
Because people are not looking for a company who can just build software, people are looking for, oh, I have an insurance problem or have a problem with, lotteries or I have a problem where I need to have a, a system that checks if you, can answer all of your driving license, your exams. Correct. Right. That's that's the what, what the customers see. But in the end and I think that's happens quite, quite quickly in the last couple of years,
27:48
in the end, companies tend to realize, oh, but that is actually not a problem. Yeah. I feel like slowly and especially with newer companies, the people that are in leadership teams or managers, they they're not necessarily experts anymore. No, but they definitely do have expertise. Like there's a lot of value in having managers and also expertise on what they're managing for the people that are doing the actual work. Yeah. I see the same thing.
28:11
For example, with, my product owner from, does lotteries, he's actually someone who can buy himself, analyze, production logs if something is off. Yeah. Make, an estimated, guess on what is happening and then after that comes to me and say, can you have a look at this? Because I think we need to fix over there, or maybe someone else needs to do something over there, but he's basically already done the analysis and that is that helps so much, because it means that you can also
28:40
get to focus a lot more on the work that you want to do. New development. Interesting. Rather than just putting out fires everywhere or analyzing things for other people. And again, because this is the business I'm currently in, integration this this can very quickly run out of hand where you're just spending half of your day explaining stuff. Still other clients or to, downstream systems that you need to integrate with. And in the end, it turns out that was, it had nothing to do with the
29:09
thing that you were doing.
29:11
So that's another reason, I think, that it's important for people indeed to, who are involved in these sort of things to, to know what they are doing. Yeah. I feel like if I were to start my career and I wanted to do software engineering again, there's a lot of integration problems that need to be solved. And the part of that is kind of resiliency and observability and security. Absolutely. But a lot of engineering work is going to go. And I think also in the future more into integration work.
29:35
Yeah, we have now a lot of software as a service companies, but also AI providers. And everyone needs integration. You also see that those companies that are for example, their core is an AI model or their core is chatbot AI something they are hiring people on staff. They call them forward deployed engineers, which are basically integration engineers. And they come into an organization kind of as consultants, they do the integration work. And then that leads.
30:00
Yeah, I feel like a lot of people are. Yeah, kind of looking for integration. It's also a lot of work that we do tends to, either be centered around integration, like my own project, or even if it's not an integration project, still, a big component is always integration. Yeah. Because, yeah. Especially in enterprise. Right. So is everywhere. Yeah. So you're no longer working on isolated little islands that just run by themselves on a machine that is, under someone's desk?
30:26
Everything. Everything needs to be integrated into it. I feel like if you're, like, if you get really good at the integration work, then you've solved a lot of problems, if you like. And when it comes to learning, for example, I think this is after learning like basic self-development and languages and different stuff. Learning integration patterns is actually a great way of becoming more of, senior developer or architect, right?
30:48
Thinking about what it means to do stuff synchronously versus asynchronously. What are the pros and cons there? What options do you have on a technical level for integration? Is it going to be based on the Http websocket? It's going to be based on message oriented middleware. What are the patterns for using message oriented middleware? What's the difference between point to point versus pub sub? What technologies do exist there or what is actually suitable for stuff
31:13
like edge computing where you have, a huge amount of the client sending in information versus just doing point to point between two big applications. All of that sort of stuff is is around integration knowledge, basically, and indeed is quite important. Yeah. And it's also much more available nowadays. One of the things I really like as as an architect, about the fact that so many clients, organizations are moving to cloud computing, is that, using something like measuring the middleware is just
31:44
something you enable. Toggle. Yeah, it's just toggle. There's already a credit card there. You can just pay for it. Where I've been with a lot of clients in the past, where they were just running stuff on their own data center, and then the small operations team responsible for things where you would go to them. You would say, you know, I have this use case now that we're building, for example, with the, the Dutch Financial Times, we have once built their entire website. What?
32:09
They were still using their old, content management system. So whenever the content management system would either publish a new article, news article, or make an update to an existing article, we needed all of our, application instances to know about that. So that could drop their cache and refresh it. This is a classic pop up message use case. Yeah. So we said, okay, what do you have for messaging system. We don't okay. Can you just install something maybe RabbitMQ active queue whatever.
32:38
Because we would really like to have a little pub sub system set up so we can do this and say well no I, we're not going to do that because well, we're three people. We're already struggling with the stuff that we have right now. We're not just gonna introduce an entirely new piece of middleware because you ask us. To which it makes sense. Yeah. I mean, from their point of view, I can completely understand that. But it means that, well, I cannot use the best technology for the job.
33:00
And now I need to, like, an idiot, manually do, like, individual requests to all of the endpoints that all of a sudden need to know, whereas I don't have to deal with anything like this. And with cloud, this is not a problem anymore. And everyone has, either something like, if you're on AWS, you have access and this or they're running some form of Kafka, which is really popular nowadays, or there is RabbitMQ or there is something else because it's not important what we're thing there is.
33:29
It's just important that there is such a technology in place. And then there's there's abstractions and adapters everywhere nowadays. So integrating something like this in your application is peanuts. Yeah, I feel like I really like that story because it highlights, I think, a really good thinking in the first place with regards to the total cost of ownership. And if we don't have it should be really added. If there's a really good use case then like this. Yes.
33:50
Because that burden, yeah is going to be somewhere in the first place anyway. And I feel like now it's a good thing that infrastructure is becoming more accessible. And it also has that dark side where, yeah, I feel like people can over engineer systems and resiliency where it's not necessarily needed. Like that conversation of cost of ownership, even on the cloud should still be a conversation. Yeah. And then yes, it can be easier to add to it. So a similar thing where with microservices
34:13
is there still conversation there like okay, how many do we need and how big do they need to be. Yeah, I remember something like three years ago, not that long ago, we had, go to Amsterdam, that's, a conference that we used to organize where there was a, speaker there, who did a lot of, microservice cloud native architecture with go.
34:32
But one thing I still remember very well, because it was very different from how we used how we work, not even used to work, how we still work. Is that, to connect to a database? They would never just open up a database connection from their go service. They would actually build, like a dedicated, adapter service, okay, in front of their database that they would expose over gRPC and then over that gRPC, you could actually make the calls to the database.
34:57
And it sounded completely weird to me. Yeah. But then he started to explain and he said, yeah, well, actually we found that, at that time, at least the, the, the database support and go isn't actually that great when it comes to things like connection pools and, validation. And we found that, well, some systems like Java is actually really good at that. So just building a JavaScript for that, for the gRPC for London and everything else go,
35:20
so that I found that very interesting because it shows a very different way of thinking about granularity and how big should service be and what does that mean? And there is no right or wrong way. But they but you do need to think about it indeed, and see what are the tradeoffs when we start doing this. Yeah, that's why I think it's super important that the people that build the software, they also are responsible for like running it and owning it and also in production. Yeah.
35:42
And I mean, I agree and sense that I don't think that people need to be always like uncle for their own software, for example. But one thing, for example, that way that very much agree with is whenever we have someone interviewing with us, for example, and they come from banking or insurance, like traditional sort of enterprises, I always ask them, were you able to just even see the log files on production for the software that you were building? And it's still so often I get.
36:09
Well, no, not directly, but if there was a problem, we could find a request and someone would make an export of a part of the logs, and they would ship it to us, like, yeah, if that's your if that's the way that they, that they do, they do trust you to build the software that runs our production. Yeah. They don't trust you to actually look at it running strange. Right. It's completely insane. They actively break a feedback loop and you have no idea what what your staff is doing.
36:33
And, that's another thing I try to cultivate in people indeed, is, make sure that you once you actually build it, that you look at it and make sure that you can see what it's doing. And if you find out that, well, you cannot see it properly, think about what you need to do to make sure that you can see it at logging, at metrics or whatever. Because yeah, I think that is super important for people. Once you move beyond junior is you need to you need to build that feedback loop
36:57
for yourself where you can understand what the stuff that you build is actually doing in a real environment.
37:03
And if you don't have that, if that's missing and there are a lot of companies where this is missing, people just don't see what their stuff is doing. You're never going to move beyond a certain level. No, I agree, yeah. You mentioned that you wouldn't necessarily put the people that create the software also in the role of being on call duty. No. Why why is that? Because I think a lot of, a lot of problems that we typically see that people that are on call need to deal with are not about
37:28
there is a bug in the software that we need to fix immediately. And rollout. It's about other stuff in the integration business that I'm in right now. Typically it's not us, it's, downstream system that is struggling. But then, that means the queues are, filling up or maybe, some thread pool is running out of threads because some system is just super slow. Now everything starts stalling, and that causes certain things to happen. But it's not that often the case that,
37:53
that the developer will be able to efficiently deal with that. And they might also not have, like, the complete overview of the entire platform rather than the system that they are responsible for to make these sort of decisions. Yeah. So I think there is still a value in separating those roles. And I still see a couple of years ago when DevOps started to become a term, people made fun of companies who had a DevOps team because, the whole point of DevOps was that
38:18
you didn't have a DevOps team anymore, but that it was development and operations doing stuff together rather than saying, oh yeah, you're not. You're now to DevOps here. But I think that's changed. DevOps, the meaning of DevOps itself has changed quite significantly. Actually. DevOps now means way more, things like infrastructure as a software, doing stuff with Terraform, automatic provisioning of, of entire, environments. And again, it makes sense to say, oh, yeah, that's not something
38:45
that every developer can be expected to know about and to be an expert in. So we need people for that. And that's DevOps at least as far as I'm concerned. A lot of that is what we now call DevOps. Yeah. And it makes a lot more sense to have those type of people on call. And I don't need to all to be experts at the system that they are operating, that they just need to know, like the basics of what is more than the metrics that we need to watch and the health checks, and what should we do
39:10
if certain non issues arise. And only see like a third line, you go to a developer and you say, well I think this is actually something in the software that you need to fix. Yeah. It's interesting because I followed this training from Greg or hope it was a couple of weeks ago. Yeah. And he was in a company responsible for a team of architects, and they had done something which needed to have kind of an ankle duty for and their Ops team didn't want to do it.
39:34
So then he was like, okay, then we'll we'll do it ourselves. Yeah. And he had, you know, he had enough political credit to be able to do that in the first place. And then his team was like, oh, do we really want to do it? And his mindset was, we are responsible for kind of architecting this thing, and we also should have the confidence that it should run flawlessly. No that's fine. Like build it flawlessly. Exactly. You have easy, uncle. That's why indeed why you should have the feedback loop.
39:56
And this is an even stronger feedback loop if you get out of bed. Of course, if you could just see what's happening. But. Yeah, but that makes sense. That is important. But it's also true. I think that a lot of things are not really about, is indeed about what you expect from an engineer from Cerner, if it's just about the software that they develop, that's a different thing than the entire system that you run and maintain. And you can do both. Amazon does it famously, right?
40:20
You build it, you run it. Yeah. But I don't think it's necessarily the best model for everyone. And that's not saying that it's, that you shouldn't do it. It's saying that I think for a lot of typical, let's say, enterprise use cases. That is the business that I'm in. I think it still makes sense to need to separate some of these responsibilities. Yeah, sometimes also for like regulatory or security reasons. Right. Maybe you don't want your developers to be able to see
40:44
and do everything that is possible in a production environment, right? I get that. So, yeah, we we see that a lot actually. We have customers also in financial industries, dealing with credit cards, payment gateway, for example, where they need to have, very strict regulations around for AIS principles and do what needs to be audited. And then if you're going to say, well, every new developer on the team, the first thing that we're going to do is deploy to production, right?
41:09
That can be a really nice practice, but scary for people. Yeah. And in this sort of situations can be tricky. I feel like in in whatever you do there's going to be risk. And then it's just how do we mitigate that. Oh yeah. Is it a principle or is it something else where people are still tracked but they still have that flexibility of going to production? Yeah, that is the conversation. And this is also, I think, very much why it's so important that you
41:34
that you know what you're doing because then you can properly estimate that risk. And I think it's a really good point. I had I did a keynote. I'd go to myself, a while ago where I said, I think a lot of people overvalue, the return on investment that you get from testing on non-production environments. Obviously, you need to do test. But a lot of people think that, okay, if we do this, if we do this in the right way, we will never encounter some problem on production.
42:01
And if we do, then it will be very hard to fix because, well,
42:07
once something reaches production, it's always a hundred times or a thousand times harder to fix than if we find it during development, for example. And in my experience, this is not necessarily true at all. I actually just yesterday, I did, my final production deploy of this big integration platform before my holiday starts. And, I was checking if everything worked as expected, and I saw some unexpected errors there. And it turned out that actually introduce an error myself somewhere.
42:34
This was causing a problem in people during the day when they would try to see the, result of a ticket. Like what did they win? So there's no nothing breaks in the sense that there's no data issue or anything. It's just that one small piece of functionality no longer there. I think from the time that I spotted the error in the production logs to getting a, production fix outs with defects. 20 minutes. Yeah, tops. Perfect.
43:02
And if you know that you can do that, if you know that you have the experience, but also the, CI CD capabilities and you have the access rights and everything else in place that you can actually get these sort of fixes out that quickly
43:17
then. Well, how much time would you actually want to invest in making sure that something like this can cannot happen by putting all sorts of guardrails into the process, everything in place? Yeah. It's just you have to look at the return on the investment and sometimes it's just not worth it. No, it's interesting that like speeding things up might make things more resilient, like on the long term, that's which is a weird mindset, right?
43:40
If we deploy to production faster, then what about all these risk kind of affordances that we have in place, like are those going to be kind of sub converter? Are those not going to be there? And then when you're talking about this feedback loop, it's so tangible, right. Yeah. You see an issue. You can immediately fix it. Sure. Maybe if we deploy faster we create more issues, but we fix it faster and those issues will still be there if we deploy less fast. Yeah. And this is something
44:03
that is also very hard to to explain to both, I think customers, but also regulators, for example, within within that's lotteries. We had an audit and
44:15
what this was actually a while ago already. We have several these. But one thing that they want is, whenever we deploy a new version of a service, which is something that we could basically do all the time. Yeah. They required us whenever this service was involved with, for example, the casino part of the of the lottery, that we would have, a hash or something of this thing and that we would report what version was running in production. So we've done that, and they didn't want it just as text.
44:44
They wanted screenshots. Okay. One of our DevOps engineers is actually automated. The process that whenever we go to production now, we're gonna send screenshots of all of the stuff that is in our Docker registry and show that it actually runs a reduction, but it means absolutely nothing. No, it's it's completely pointless, but it's just because these companies are used not to regulate custom built, enterprise software that gets deployed on their very frequent basis.
45:10
They are used to regulating products, and these products are maybe going to have two major releases every year and three twice a year. That's. Yeah. And we need to be absolutely sure that the thing that they claim to have delivered is running in production. So you need to be able to prove that. But yeah for us it's it's just pure overhead. Yeah. And what we see it with customers as well that sometimes they are indeed very scared to to put new versions of something in production because, well,
45:38
the thing that we have now actually works and, there's a lot of people using it and who knows what happened, but
45:44
yeah, you need to really educate your customers on this, that the longer you wait and the longer you accumulate change, the higher actually the risk is going to be because, well, if you're doing it regularly, you become better at it because you do with variables, and the amount of changes that are inside of everything will become a lot smaller. And people will still remember what was actually in there. I think the last one is really important, right?
46:06
If I'm going to deploy something and it has changes from a colleague who's not even on my team anymore that they made like six months ago, and now it finally reaches production, something is wrong with that. No one has a clue now. So then we really have to dig. Yeah, we'll have to dig. Have to reverse engineer what happens. Whereas if we're deploying, every week, every day and something is wrong, then. Oh, yeah, I still remember what I worked on. That must be that thing.
46:29
And then indeed, my experience is that getting fixes out, then this is way, way, way, way quicker I agree. Yeah, it's interesting that with requirements like okay, we need to have evidence of what we deploy. Yeah, that's a requirement. And then sure you can think okay, maybe that's strange and I don't really agree with it. But changing the system can also be hard. And it requires time usually. Yeah. And it's such an engineering mindset to then be like
46:52
but we really like fast deployments because of all these benefits. Right. Exactly. Quick changes. We know exactly what we deploy. We can kind of triangulate problems faster and get the fixes out faster. So we automate kind of this process of them giving you the evidence. Yeah. And maybe that's not what the people want because they expect kind of oh maybe two screenshots of like twice a year deployments and then get overwhelmed. But yeah that help is going to change the system then.
47:15
Oh yeah. That's that's really good. Yeah. And fortunately I think for me, I don't have a very, very large amount of experience in these regulatory environments. So this is the exception rather than the rule. For me, what I've heard from people, a lot of people who, do more work and flying it, for example, do they indeed they they're dealing with this all the time. And, there have been some nice, talks about it. They Farley, for example, is one of those people,
47:40
he invented the, continuous deployment, a term I think I wrote the book on it together with, just humble. But they Farley has written also a lot about, how can you actually work in, iterative, fast, often deployable fashion in the regulatory settings. And it basically means you have to work with the regulators to, to say, okay, we understand what you're asking now, but that's not the real thing that you're after. That's just your current way of
48:11
getting the information that you need for your ultimate goal. So what is your ultimate goal? And once we establish that, we can find out how we can actually meet that goal in a different way and get it to work. But this isn't it not this is not my area of expertise, because fortunately, I haven't had to do this that often. But I think it's it is very interesting because, yeah, if you just say we cannot do this because of regulations and it's not good enough. No, it's no good.
48:36
You need to that depth. Explain it to me. Like especially with engineers, they need for me, it's really hard to do something without understanding. It's not good enough for your engineers. But in the end, it's also not good enough for it. Well, your customers need our systems, of course. Yeah, you need to. You need to be able to work in the way that you want to work and still meet these requirements. And, it's possible, but it's it's real work. Yeah. It really depends on the environment.
48:57
I think especially for your career, if you want to accelerate, you need to be in environments where people have the patience and also the respect to educate each other. With regards to, okay, this is the knowledge. Yeah. And try and create that shared understanding. There needs to be enough interesting problems also for you to learn and grow and not just work on kind of surface level and not being able to see logs in production. For example, oh. What you build.
49:18
And I think that's really going to make or break your career acceleration, you can coast and coasting is fine. Not everyone needs to kind of aim for the stars with regards to growth. It depends on your environment, but if you're in a really good environment, it can really help your growth. Yeah. That's mainly yeah. Yeah yeah. I think it's also you can tell very often when, when you have people that have that drive and that it's, it's important also that they are able to do this
49:41
because it becomes like a self accelerating effect. Yeah. If people see the results of that, a lot of their efforts and then it sort of just bootstraps itself from there. But you can see the same thing happening with people who indeed are there. They're they're young. The answer is yes they can. They want to learn things and want to do things, and then they end up in the situation where, oh, yeah, this is not really possible, or yeah, by the time that you're working on something now,
50:07
it won't be running in a production environment for half a year. And then that maybe not even because someone changed their mind. Right. You see that sometimes you product development companies where you can be working on stuff that never makes, makes it into the light of day. Yeah. That's of course, not very motivating. So, yeah, it's also important to so that people actually stay motivated to, to continuously improve themselves and do the work that you don't get motivated by doing the work.
50:33
You get motivated by seeing the results of doing the work. And if you don't see those results, then, yeah, it's it's really hard to just keep going. Yeah. I mean, it makes me sad that people can get stuck in an organization like that and think, okay, this is it and this is it. And the same is on the other side. And I mentioned interviews before, like I always ask these questions to people from banks, insurers, companies. I've had a couple of interviews also with people
50:55
coming out of these sort of organizations where I told them at the end, well, you know, we're going to make you an offer you can accept or not, but whatever you do, please leave your current job, because the way that you're explaining what you're doing, this is just not a good position to be in. And yeah, well, we can be a company that can do better for you, hopefully. But there are many other opportunities that are better as well because sometimes people just don't know. Right?
51:17
If you're if you're in your first job and this is what you're used to and this is what you do, you think that. Yeah, this is just the way the things work or, but then you learn by talking to other people, like what is actually possible. And how could it also be? Yeah yeah sometimes changing jobs is the easiest way of making things better for them. That's why I really like when people come on like you on podcasts like this to share their perspective.
51:40
Yeah, because otherwise it's just if no one shares their perspective, you only know kind of what's within your organizational boundaries. Yeah. And that's true. Yeah. Thank you so much for coming on, man. You really enjoyed this, good software engineering mastery. I feel like it's a long path. It's a hard path. There's continuous learning there as well. But once you have really solid fundamentals, you can really excel and deliver. Yeah. And also, once you actually go beyond the basics,
52:05
you start to learn to better understand the, the underlying, abstractions. And then it actually becomes easier. Right? If I need to learn the new programing language now, I don't it's a very different experience from someone who is just into computer, engineering who needs to learn the language. Right. I can just yeah, that's from there. That's from there. I definitely have that. You know what to expect. So it's, it gets better. It's like. Yeah. But there's also the message to people struggling.
52:32
I think it gets better over time. Yeah. Yeah definitely. You have to get through that hurdle if you're still here. Thank you so much for listening. Let us know in the comment section what you thought of this episode. Leave a like if you like the episode and otherwise we'll see you on the next one.
52:45
This is all coding.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat