Skip to main content
Loading Explore guides…
Becoming a Great Software Engineer · Acedra
Home
Explore
Computer Science
Becoming a Great Software Engineer
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
80 segments
Copy
Hide video
0:00
If they understood the problem, they wouldn't be hiring you. They'd fix it themselves. Simple is complicated enough, especially at scale to a lot of people trying to adopt new technology just for the sake of it. And that is something that a good engineer doesn't do. The good architects are usually the ones where magically everything goes well and nobody knows exactly why. What is your advice for people that really want to become great software engineers? What they need to do beyond coding
0:25
What is your advice for people that really want to become great software engineers? What they need to do. Increase your breath. There's a lot of focus on that. Early on in your career, which is important. And there's an advice that perpetuates which says, focus.
0:42
True focus. But at the same time, don't dismiss everything else that's happening. Increase your breadth. Personally, I optimized one thing that really got me far, which is having the ability to learn effectively in very short periods of time. And that's why whenever I tackle a problem, I don't have a problem digging very deep into anything new I encounter. From quantum mechanics all the way to any technical problem that I face on a day to day basis. I'm a, you know, I mean, if I say drop out,
1:17
I never finished my computer science education, but that did not deter me from picking up that education myself and really digging deep into all sorts of theoretical topics, but also practical areas and topics. And I'm a perpetual learner. So in my opinion, the next phase where we're heading is going to require people to be able to learn really fast, get out of their comfort zone really fast, dig deep really fast, and attain high levels of functional proficiency in any new area.
1:47
And then attaining mastery is not always required in everything. So just pick a few topics where you attain mastery end, but also increase your breadth. And if we are talking about the skills that are for the future, you're saying technology accelerates quite quickly. What are some of the skills that you think people should focus on specifically for their career, but also for like peace of fulfillment and looking towards the future? The ability to learn?
2:10
First and foremost, none of the tools I work with I learned, I learned as free as people. Back in the day, some at some people in the audience will be cringing right now. SPSS taught myself, to scrape Twitter back before it became shitter. And to to model the 2013 14 Syrian refugee crisis. I hated Putin before I knew where Ukraine was on the map.
2:37
But from I went into Python and now Python is the end all be all. But I'm from an era where people were still asking like, are you sure? Should you be learning SAS because SAS is better in the market? Right now. So God knows what we'll be using in 10 in 10 years, unless all the all the limbs start overfitting on on just languages that are already in use. We might actually see a stagnation in the development, in the development of coding languages.
3:06
Now, since Claude and Gemini will not be able to adapt to new languages as much as produce more Java whores, meaning we will be stuck with Java until the he death of the universe, I'm pretty sure. But what I'm trying to show here is that those tools, those techniques, cloud platforms, they change so fast that you can't focus on learning a technique. You gotta learn a method to learn new stuff that is the most single and most important skill in the entire tech field.
3:37
To be able to learn, not even master, but just work with new tech. Anytime I have a new project. Oh god, it's a new cloud platform I haven't worked with. Okay, well, at AWS Google Azure I'm pretty sure it has most of the same functionalities. Jade. They just call every damn button differently and that's how they vendor lock. So so you can go go to the other cloud platform because the names is different,
4:02
which is less of a hurdle than you would think. And then there's of course nuance, like the billing models are different. The scaling is different. The type of servers you can run it in is different. You cannot. Local host I mean, sovereign, this this version, you remember that people sovereignty is the new local host. You're just running your own damn model on your own damn computer. But you don't call it local host. You. You call it sovereign. The client will love it. Thank me later.
4:33
So. So it's. And that's the other thing. And I mentioned that before understanding the words because buzzwords will come, but they usually mean something that wasn't new at all. Again, sovereignty is just local host but new. Everybody screams with screaming AI. But we mainly meant just deep learning on steroids with a lot a lot of GPU and compute. Big data was just data that you store on multiple different servers, because you don't have a single server that can contain it all.
5:05
But nowadays, my laptop has more storage than a whole complex of servers a decade ago, meaning big data is dead long, long live big data. The core problem here is the core problem of humanity, which is that sometimes we say the same words, but we mean different things, and sometimes we say different words, but we mean the same thing. And all of miscommunication stems from this. And in our field, it is important to figure out not just
5:35
what words people are using, which is what sales heavily focuses on, but what their actual problem is and what they actually mean, which can be which can be completely different from the words that are actually using. Because I've had clients that that ask for AI, and it could mean a random tree, a random forest, it can mean you have some kind of image recognition model, it can mean some type of speech synthesis. It can mean producing synthetic data of bananas using 3D models.
6:05
It can mean so many different things, but they all use the same word. And it's our job to figure out what the heck they want and then to build that, not just to receive orders and build something, because that job will be outsourced and automated to hell and back by whatever cloud copilot thing they, they buy and or consume. But the core skill to figure out what the fluff they actually want and need, and what is the problem behind the problem,
6:35
because if they understood the problem, they wouldn't be hiring you. They to fix it themselves, but they do not understand the problem, do not understand the solution. It's our job to figure out that solution, then do that. And to do that, we need to be able to learn new tech and learn new stuff and that's why learning is key. What have you seen in how architects execute that makes them really good architects for the people that actually go on that career path? Yeah, I always say, I mean,
7:00
first comment would be it's not a sort of binary choice. So I don't think that being an architect is determined by what's on your business card. So I think it could be a technical product person with a very architecture mindset, right, for me. But let me start the other way around. The the bad architects are easier to spot. The good architects are usually the ones where magically everything goes well and nobody knows exactly why. The bad architects are easier to spot.
7:28
It's like people who spew out a lot of buzz where it's, you know, it's like, oh, everything must be cloud native or loosely coupled. I would say, I don't need an architect to tell me that I can just put a post on the wall. They are. Then why? Why do I need this? This person or people who believe they should have all the decision power is like, oh developers, I'm here to guide you. The I should make three components, right? No more, no less, right? Those I find the not so stellar architects, the,
7:55
the mantra that we have in the workshop and that I very much follow is that architects shouldn't try to be the smartest people, but they should make everybody else smarter. That's an amplifier. You don't want to be a kind of oracle. People come with the questions and look for magic answers. Oh, what should we do? People come to me and I'm like, well, it's kind of charming, but how do I know it's your project? It's your application, right? How can I make all the decisions for you?
8:24
But what I can do is help you make a better decision. And I think to me, that is one of the characteristics of a of a great architect. You can absorb the context, things that people explain to you, but you can uncover blind spots or maybe help people see different points of view, different angles, distill trade offs that they might be implicitly making, but they're not aware of those kind of examples. I think that makes a great architect, and it makes a nice architect
8:54
if you wish, because you're not trying to be the ivory tower big boss. Yeah, but he actually an amplifier to the team. Any other foundations or fundamentals you want to highlight that distinguishes great engineers from not so great ones?
9:11
If I look at, what is happening in the picnic context, again, what what really strikes me is that, our, our engineering culture is relatively, uniform in the sense that when we started, ten years ago, we have a very strong, developer platform that was, developed from the get go.
9:34
And they're what I see in some other companies in other, contexts. So there's a lot of, people trying to adopt new technology just for the sake of it. And that is something that I feel that a good engineer doesn't do. Right. So if you adopt new technology, if you find out about the new technology, you need to understand, whether it's actually an improvement over work, you have, make very clear also, why did this the case, and not just jump on any, any bandwagon.
10:08
And that's something that I feel that's very good engineers are good at. Right. So they there's there's a science. I'm not sure if you know, the boring technology desktop. I don't know it. That sounds very funny. Yeah, yeah. We should have asked this, for, use what works. Use what is proven. Use, and then use it in a very, effective way, as you mentioned, sir, that that is, I think with both YouTube and, discerning between. Okay, but what new technology is that interesting to jump on to, etc..
10:38
That is super hard. Of course. And also maybe a bit more of an art than the science sometimes, but precisely. That's kind of, gut feeling. I would say that that the knowledge and the wisdom that you have as a, as a, as a craftsman, as an engineer that sets you apart from people who, yeah, who don't have that taste or who don't have that knowledge to, to make the decision. And then sometimes you end up in situations where there's, companies that are much smaller than.
11:08
And then the company I work at that has like dozens of different technology stack soon, because, well, people, decided that in some kind of context, it was better to use, rest or, teams go or to and then you end up with this fantastic collection of technologies that nobody really knows how to manage anymore, really how to operate anymore, who really knows how to how to bring us together and, yeah, that's I think, one of the hallmarks to go back to your question of a good engineer
11:38
is to actually also, limit this and, and stick to, yeah, proven things and what works. Yeah. Is this then your advice as well. So to stick with what you have, start with initial design that is simple. And then stick with it until you actually reach certain limits. And you can kind of based on your usage forecast what the next iteration would be. If I was a startup founder or a CTO of a startup, right. I would never go and start building for 100 x scale, even if I'm a
12:07
like a firm believer in my idea and know that it's going to reach the masses and it's going to be global, I would never design for that. I would design for getting 100 users or maybe even a thousand, right? And I would just run everything on a single VM, to be honest. Maybe to
12:23
make it a bit more available. I would not go and build massive database clusters. Yeah, just start with one, maybe two nodes. Stops replication. That's it. And this will like tell you really, really far to be honest. Once I attain or I'm get, I'm like close to 80% of these limits of what my resources can do. And by the way, vertical scaling can take you a long way. Like you have no idea. We have large scale nodes that have hundreds of CPUs on them. So you don't really need to go
12:58
and start sharding and doing all sorts of fancy stuff. When vertical scaling consists of you and right now with cloud native solutions, you can go pretty far, like you can get a one VM with 120 CPU cores, right? You can get terabytes of memory. Yeah, and that can do a lot for you. You don't need to start doing horizontal scaling before you get you really start hitting these limits. That's what I've seen where it's challenging for people, where they say, okay, I'm a little bit of a perfectionist.
13:28
I value quality in what I do on a day to day. My job is basically part of who I am, and if I don't deliver the highest quality that I can in this aspect, like I'm doing myself with this, just this. And it's very like if I lay it out like that, it might sound selfish, but those people have no clue that what they're saying is selfish. They genuinely think it's best for the organization. It's just some misinformation and misunderstanding because there's a dogmatic aspect of software engineering,
13:53
and it was perpetuated in the previous era until 2000, 2000 I when I ventured into working professionals in 2007, eight nine, there was a lot of dogma, like there's a lot of this is how you do things and this is how you should do things. And if you do things any other way, you're wrong. Yeah. And part of that is elegance of code. Code cleanliness, code quality. You were all raised on this, these these concepts. Right. And they're important.
14:18
And everybody always complained about bad code and horrible code because it was difficult to maintain. Nobody enjoys that. You're going to make it very difficult for the future engineers to solve these problems. But sometimes, you know, you don't have future engineers to solve these problems. Sometimes nobody's going to touch this code for decades to come. It's okay if it's just it just works. And then we ended up on the complete other end of the spectrum,
14:43
where we tried to clean up everything that we started adopting. All sorts of design, overcomplicated abstractions that are are just horrible to me. Horrible to reason about. It takes you decades to become proficient in just reading that type of code. And then when you write it, it gets even more complicated and it's very difficult to maintain, and big tech. And I always tell my colleagues in the juniors that report to me, simple is complicated enough, especially at scale. Yeah.
15:12
So just write it in the dumbest, most simple way possible. This is why I also love gold. For example. That's all I do, right? Yes. It's amazing. It's dumb. You don't have to
15:24
think too much. It's just all in your face. Whenever you want to understand how something is built, you just trace it wrong. It's there, it's there. Fantastic. I love that, and in the situations where we're building at scale, I don't want to reason about abstractions. I don't want to think about edge cases. I don't want to think about complicated memory allocations and complicated memory structures. You know, I just want things to be simple.
15:50
Memory leaks happen often, and they are very catastrophic at scale. I don't want to think about these things. I don't want to think about how my garbage collector is going to, you know, shut down the whole world and start causing multi second latency on my API requests, which at scale again become a gigantic problem for me. I just want things to be simple, easy to reason about
16:12
and that's it. What is then your perspective on building systems that can evolve? Because some people you can say, okay, this is the system design that we have and we're going to continue on this until we reach a certain level of scale, and then we're going to revamp and say this would be the next iteration. But some people also say, okay, we have a system now, and this is evolve able to whatever scale we need. And then they put in the extra effort to make sure
16:36
that that is there from the beginning in our industry,
16:40
very few technologies last for like a decade plus, and the rest is changing often. And that's why I personally don't invest, or I wouldn't design a solution that would last like ten, 20, 30 years, unlike what we did in the 80s or 90s. That's what that's how that was the design mentality. But things were way less complicated than they are today,
17:03
and the scale was different. So I would always come with the mindset that I'm just going to design for the next order of magnitude, and then I'm going to evolve it from there, and that's going to require another round of investment. There's going to be continual investments in this and that business people are not going to love this, because that makes accounting much harder. That makes strategic finance much harder because they cannot predict
17:25
or anticipate the costs that might come in the future. They cannot extrapolate from today. You know, they don't like this fuzziness software. It's all risk. Yes. So and also like they cannot calculate like their bottom line or projections for the future, which are things that are very realistic, like publicly, publicly traded companies want to be able to do these projections, and they want to be able to say that, hey, we're not going to be spending more here than on evolving our software,
17:50
but we're going to make much more revenue and increase our bottom line and margin. That's what businesses look for. So it's understandable. But from a software perspective, there's needs to be regular investments. And I think the only approachable way, which is something we do at GitHub, is to continuously revisit these investments on a quarterly by quarterly half a year or a year basis. Right. We can make projections for the next three, four, five years,
18:16
but the confidence in these projections is less and less and less. Yeah, more time goes. And I think a lot more companies need to start doing the same. We're always working in teams. Also, from a business standpoint, how would I, for example, say we're going to design for this until we reach scale and then we're going to need to revamp. And this is what that's going to entail for people that don't understand how to build software, understand the business constraints.
18:37
So they're never going to be able to convince the business folks about how something technically there's just it's too abstract, even if it feels too intuitive for us, for them, it's magic. Yeah. And even if they try to understand how things work, they're never going to comprehend the magnitude of problems. And they're never going to have this intuition about how software is and how not hard, you know, how how evolve abilities. So it's our job to get closer to their side.
19:08
Some people say no we want the business to understand. They're not gonna it's trickier. It's easier for you to acquire or assimilate some of the not everything some of the business constraints and discuss them from that perspective. And that's why a lot of engineers who grow are very good at understanding the business constraints and then discussing the technical solutions within those constraints. So for example, I used to work in the container terminal industry for a while.
19:35
We built a huge container terminal. We furnished everything from the fiber optics layer all the way to the three tier data center. With all of the enterprise applications running and running on it, and the operating system for the container, and it's one of the most massive operations you can ever imagine. One of the most hazardous as well. So the folks there as far away from tech as possible, you're not never gonna find the terminal operator who's going to understand what it means
20:01
that your database is running hot and you cannot support the Is. All they care about is. I have a ship here, was docked. There's, like, 10,000 containers on it. I need to empty them into my yard. I have a very fixed window of time. And if I don't do it in that window of time, this is going to cost me money.
20:19
We were operating at 12% capacity. That terminal was generating about $50,000 per hour,
20:27
so 2%. So you can imagine if we get delayed, every minute of delay is actually money that's disappearing. So this is how you need to discuss problems and investments and solutions. You need to understand what the impact is, what the impact is on the operators. And sometimes it's life threatening impact, right. If your software fails when a SDS when the crane is offloading a container, this can be a very life threatening situation. You cannot really have your software malfunction at this point.
20:59
So sloppiness of the software sub millisecond latencies, all of these things matter, right? So you need to approach the discussions with the stakeholders. Yes. By understanding them first. So what I've done, for example, in the industry is I've literally sat on a truck and went there and I sat with every single individual who works in that terminal to really understand what they go through on a daily basis. So a truck driver who's driving a truck for six, seven, eight hours,
21:25
and it was in Saudi Arabia in scorching heat, it's not going to understand like the button that they clicked is just is so
21:33
slow. Doesn't matter. He doesn't care. Night. He just wants it to just work. So you got to understand all of these perspectives. And then when you do you can start approaching discussions from that vantage point. And this applies to banks. This applies to hospitals to any other industry. So understand the industry understand the business constraints. And then any technical solution you want to discuss and investment, you can approach it from that vantage point
21:56
and you're going to have much more success in these conversations. When you were speaking about hard skills, how much of an enterprise architecture or being an architect in essence, has changed nowadays because I feel like I get bombarded with tools. There's a lot more possible specifically with large language models, but in essence, like the Phantom sketch artist, everything underneath kind of still holds true from your perspective. Does architecture evolve quickly?
22:24
Because I feel like tooling does at a more rapid pace. Definitely. The environment in which you operate has changed. So there's a couple of assumptions that architects used to be able to make to make their lives easier, that no longer hold. Right. The one is like snapshotting like the world is moving too fast to be able to pretend. It's like, okay, let's just assume nothing changes. Well, that turns out to be a very poor assumption in today's environment. Yeah.
22:51
So that, for example, means exercises like trying to draw a giant ID landscape, you know, or cataloging all the applications you have and putting that into some tool, taking a few months for that. Yeah. And then a few months by the time you're done. This has already changed. So for me, the assumption of being able to snapshot and being, I call this the cartographer, being the person who makes all the great landscapes, that is very useful.
23:18
But it's no longer practical because stuff changes too quickly. So one of the metaphors I use, which captures the changing role for enterprise architects, is rather than trying to be the cartographer who walks around with a giant map of all the pieces, you need to be much more like a scout. You have an objective in mind. You want to cross the river or beat the enemy, or like, move somewhere or like whatever, right? You have some objective, you send some scouts, right?
23:45
They're going to go look at things. They figure out what works, what won't work. Right. And they come back to you with a very simple map, but a very timely and very expressive map. Right. Do you want to cross the river? Is I going to use a mountain? Yes. The enemy. The bridge is gone, but he is not. So DB, we can cross there. We can go right. Whatever it is. Right? But they have a very situational. They have a sort a purpose driven map.
24:08
The map is not trying to be like the perfect map of all the exact landscape. They only depict what's relevant to your move. And I think that is becoming a more viable metaphor of what enterprise architects should do. It's no longer plausible to, like, maintain this giant map where everything is on and everything is up to the head. Rather, you need to have the purpose. You need to have a direction in mind like l'Alliance Jean AI is a great example, right?
24:39
Okay, the enterprise architect, what's our strategy for this? Where should we find the best first use cases? How do we integrate this with the rest of our systems? We know LMS have super high rate of evolution. Yeah right. So how do we keep that change out of other systems? Right. What are we doing with with agents and a genetic right. Is that just another form of workflow integration or is that something novel? Right. Give me a point of view, but is very purpose driven.
25:07
And as you will quickly see, you can only do that if you understand the subject matter right. You can sort of, wave your way through that. Right? You need to have, you know, the hard tech skills as well. But for me, it's very purpose driven. What should we do? Do we have a point of view? Right? Do we have a question in mind? Too many architects try to find answers when they don't have a question. It seems like, oh, here's the thing that answers all possible questions.
25:34
I'm like, that would be nice.
25:37
Easy. Let me. Would that be easy? That doesn't work anymore. You gotta start with a question in your mind and then like the scale, I just like whatever. Can we cross the river? I said, okay, come back with the answer to this. It's got to be situational and timely. Otherwise it becomes hard work and we might like our ad work, but obviously we're not a museum. No. The it. Yeah, it's it can't be a historian. Yeah, exactly. We can be a historian. We're not collecting modern art. Right.
26:03
Our diagrams serve a purpose, right? They help us make better decisions, get better decision, transparency and discipline. So aid the decision is, is clearer. And everybody's on the same page to what the decision was. Actually what drove the decision. What trade offs are we willing to make. That's what I think the value and that the picture just has to be a means to that end. If that end isn't there, the picture is also worthless. Yeah. And then it just becomes modern art.
26:32
Some boxes and triangles, some kind of kind of things. From a career perspective, people are saying very early on topics are getting automated now by agents, which means that the people that are educating themselves and that usually that's a step for them to build up the skills to do very, very cool things that help society. That step is now changing and people are asking, well, what do I do then to make myself not obsolete? And we're talking about anything with regards
26:56
to communication and soft skills and learning. I agree with all of that. And then there's still a tooling aspect like I don't know what I should advise people with regards to.
27:05
Okay, how do you go from junior to actually in the end, becoming a senior because it's such a gray area that I don't know yet. I feel like we're really figuring things out, and the only advice that I have is be as open minded as you can. Try and familiarize, familiarize yourself with a lot of things. Try and build as much foundational knowledge as you can because tooling does change. I'm wondering what your perspective is on that. Don't be a fool with a tool.
27:29
That is one of the core pieces of advice. I got started my career some over nine years ago. Got
27:37
don't be a fool with a tool back then we refer to only knowing are only knowing Python or only back then. Do you remember RPA people? Do you remember back when RPA was, was a thing? I was using a Python package back in the day to to scrape Twitter. I forgot forgot the name selenium. Yeah, yeah, I was learning selenium because that was one of the first Python based browser to automate the browser and click down. Back then, those were bots. Yeah.
28:09
But I knew people who doubled down on only learning selenium and all the ins and outs, but selenium was open source and it was great. You could do everything with it. But then the blue prism came in, which was basically the, the company, or the company organization equivalent of it, but it was reliable because you could sue somebody if it didn't work. And suddenly the whole selenium market disappeared because people went for it for larger infrastructure tools instead.
28:37
And the guy who knew everything about selenium was stuck. And the people who learned, Blue Prism were laughing at him. Nowadays, we laugh at the people who only know Blue Prism because the cycle continues. It's not about learning any one tool. It's about being able to learn in general. And that involves learning the ins and outs of tools. I think it, we are switching from an explicit knowledge economy to an implicit knowledge economy, and our education systems haven't adjusted to that.
29:10
Yeah, that the worth of knowing stuff in your head, memorizing it, that worth is only going down. But the ability to look up stuff that was introduced, like what, 25 years ago when Google hit the market, before we had that, we had, start start by
29:29
that has been introduced during our lifetimes, but the value of that of finding stuff in large amounts of information that is only going up, especially in the age of disinformation, because we are drowning it in the information. But truth and wisdom has only increased, invaluable, and has become relatively more scarce, because now there's more bots and more fake stuff flooding in the internet than actual real stuff. And the only way we can still distinguish truth from fake,
30:02
because I see so many deep fakes now of Zelenskyy, of booting of F-35s that crashed in Iran, and we could only tell they were fake because there was still a human in the cockpit because they forgot to prompt it removes the human from the cockpit. We can only tell truth from fake because of human error and that too will end. So what we need to learn again, because learning is important, is to triangulate information to figure out what's real. And this is why I, I'm frustrated with modern media
30:31
and journalism that thinks that just regurgitating and repeating what no lie or say is somehow journalism. I forgot the quote, but somebody once wrote this if one source tells you it's raining and the other says they're sunshine, your job as a journalist is not to report both, but to open a damn window.
30:52
And that's what I'm missing. What is your perspective on AI tooling in this building frame? Because if you're looking towards end goals, if my end goal is to build a product or an e-commerce website, I can I can generate a lot of that stuff. But would I then learn how to actually do that? And I would achieve my goal in a different manner with people from your perspective that start learning now, should they leverage AI tooling or how can they use it to learn effectively?
31:17
This is a great question, and we're still finding that out. I think, yeah. So so my take is, yes, you can use it to, to set up a lot of things for you. I would more use it as, a tool for validation rather than something for, for you can generate things. Right. So if you are not able to actually explain what comes out of this model, and then if you are not able to understand why the code is there, then you will have a very rough problem also maintaining that and extending that.
31:50
And of course, and you can say, okay, well then I have some agents,
31:55
who does silly. But it doesn't need there, there is, there will be a gap of understanding where the model tripped up and you are not able to get it, catch it. So use, use AI models to validate your understanding. Try to, generate maybe a few things and see. Do I actually understand what's happening here? If not, ask the model that. That's great. But don't focus on getting, a product out of it and then getting sort of the the
32:24
yeah, the products there in the end is really, a tool that you can use, especially if you're like a taxi driver who, who's going to and doing coding. It's a tool that you can use to to build your knowledge. You can also use very much to change of course, and to just so I step all of this hard work of trying to do a sentence. But I don't think that will, in the long run, help you.
32:47
beyond coding
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat