Skip to main content
Loading Explore guides…
LeKiwi Mobile Robot · Acedra
Home
Explore
Engineering
LeKiwi Mobile Robot
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
105 segments
Copy
Hide video
0:13
Hi, I'm Ilia and today I'm going to talk about this mobile robot. It's called LeKiwi and it was developed by SIG Robotics and the University of Illinois at Urbana-Champaign in collaboration with the LeRobot team. I will talk about how to start with this robot, how to assemble it, how to control the robot remotely, collect datasets and train autonomous policies. So let's get started. This is the main character of this video. It's called LeKiwi and my version is not the default
0:48
one. So actually it's pretty heavily modified. I use some custom cameras. I use a custom gripper. My front camera is rotating together with the robot. I also added this top extra layer that allows you to add one more top view camera and it can be attached on top of the robot like this. And you will see it in this video many times. So sometimes I will use the more default version with just two cameras and no top layer and sometimes this upgraded version with extra top view camera.
1:25
But regardless of all these modifications, it's still the same robot and everything that I will be talking about is applicable to any version or any modification of LeKiwi. This robot consists of two main parts and it's convenient that they have different colors in my case. So the top part is just SO-ARM 100 or 101 and simply speaking it's this robot. I have a big video about it describing all the steps that can be useful to start experimenting
1:59
with this robot. So you can just follow the link and check it out. I would definitely recommend you to watch this video because everything that is applicable for this robot is also applicable for the LeKiwi and I will try not to repeat myself too much in this video. So if you haven't seen it, I strictly recommend to do it right now. And the mobile base is basically what distinguishes this robot from the arm itself. So it allows it to move in any direction or rotate in
2:30
any direction and you can do much more different tasks using this robot because your arm is no longer static. It can move in the space. This base has three omni wheels, has three motors that are the same as servo motors in the arm itself. So, it's very easy to control them and also it has in my case Raspberry Pi that actually controls everything and a battery over here. This base will cost you extra maybe $300 on top of the cost of the arm. I think both wheels and motors
3:11
are a bit pricey, but I think there are already some options how to get cheaper wheels or even 3D print them. So you can search in the community and maybe find some alternative options. Overall I would say there is much more variability of components of this robot than in the original arm. So you can be much more creative and try to do a little bit different things. Before we move to the robot's actual capabilities, let's quickly talk about 3D printing and assembly
3:45
of this robot. The mobile base is mostly 3D printed and uses the same motors as the arm. It took me a couple of days to print all the parts, but the actual time will depend on your printer and how you distribute the printing. Some parts you likely need to adjust for your setup. If you're using non-standard cameras or batteries, you'll need to print custom mounts for them. There are several component variations available. You can use different wheels,
4:13
different batteries or different single board computers to control the robot. Just follow the BOM listed on the repository and find the best parts available in your region. I will also share a repo with my custom parts in the description. After everything is printed, the next step is assembly. Just follow the instructions in the original repo and it took me a couple of hours to assemble the robot. Pay attention to the small parts required. Unlike SO-ARM,
4:42
the LeKiwi needs extra screws and nuts that aren't included with the motor. I initially missed this and had to improvise to fix it. Even if you plan to run it using a battery and single board computer eventually, first I would recommend you to test it using a cable. Just connect it to the power supply and your computer. It's easier to debug if something isn't working at this stage. The manual control of the robot is called
5:20
teleoperation and the most default way to do it is by using the keyboard of your computer and the leader arm. You can use a keyboard to move the mobile base in any direction and also rotate it. And you are using the leader arm like the one right here to control the actual arm of the robot, the follower arm. So you can see whenever I'm moving my leader arm, the follower arm is making the same movement. So you can control the gripper, you can control any joint of the robot by simply
5:52
moving the leader. It's actually completely the same as controlling the simple SO-ARM. So again, you can just refer to my old video where I discuss more teleoperation options. Also, there are not so many default ways to control the robot, but especially after the recent update, LeRobot allows you to integrate any teleoperation device into the library pretty easily. And there are a lot of community efforts in this direction and different people actually
6:25
integrate different teleoperation controllers into the library. One very common thing is to integrate different gamepads and controllers. But another example that you can see right now in the video is what I developed during the LeRobot hackathon that happened a few weeks ago. And this is a simple Android application that allows you to control the robot remotely. And also all the video from the cameras is sent directly to this application. So you can see it on the
7:00
screen. You have joysticks that are similar to joysticks in video games. And you control both arm and mobile base using this device. To be honest, I would say that using the leader arm is still the most convenient way to control the arm itself. But I think for the mobile base, there are better options than simply using the keyboard. By the way, if you want to try it, all the code for this application is open source and I will also provide the link in the description.
7:39
The robot uses Rerun that allows you to see the stream from cameras during the teleoperation. So in this mode I can see what's happening with my robot, can control it and can also move the arm itself and immediately see what's happening and what's changing in the cameras. You can also see all the actions and all the states here on the chart and yeah it's very convenient to control a robot like this and I would say it's a very good exercise and good practice in the beginning
8:16
when you just assembled your setup to try to control your robot using only these cameras. It's not strictly necessary for data collection but it's a very good heuristic because if you cannot control the robot using cameras yourself, how do you expect the policy to control the robot? You can see my top camera is slightly shaky whenever I move it. It is shaking but yeah I will fix it later. When you control your robot using cameras only, you will also be forced to
8:51
do some exploration. For example, you see if I move the camera like this, then the wrist camera becomes pretty useless. But if I move the camera like this, it's a nice front looking view. So if I would control my robot without looking at this video, at these images from cameras, I would probably not care about the position of the wrist. But when I control it using cameras I'm kind of forced to do some exploration and find the best position for the camera to have the best
9:26
view on the environment. And the policy when you train it will learn how to do it autonomously. Your policy will know what to do with the cameras to get better view on the environment. Yeah. So this is how it works. Quite an enjoyable process to control the robot like this. So let's see what the mobile base is capable of. So first of all, of course, it can move forward as any remote control car and it can move back and also it can rotate in place again as many
10:02
robots or RC cars. But what is really nice is that this robot also can move sideways. So it can move to the right, it can move to the left. And you can combine any direction and any rotation at any moment. So for example, if you simultaneously move forward and to the right, you'll just move
10:26
along the diagonal and you can move the same way in other directions. And you can also combine rotation and side movement. So for example, here I move sideways and I rotate at the same time and in this case the robot rotates around some point in front of it. If I rotate to the other side, it will start basically rotating around some point behind the robot. So you can see these very strange but nice and fun movements and this makes this robot very capable of
11:04
moving in hard environments and it's very easy to control and it gives it very high maneuverability. I already mentioned that this robot uses this special type of wheels called omni wheels. So let's have a closer look and discuss how these wheels work. Okay. So first of all omni wheel is a wheel. So if you take this wheel, if you connect the motor start rotating this motor, this wheel will move forward as any other wheel and there is nothing specific about it. What is specific
11:40
is that this wheel also has a lot of these rollers everywhere that are very easy to roll without much friction, much resistance. So the idea is if you push this wheel from the side it will freely go sideways without even needing to actually rotate the motor that is connected to this wheel. So it means that at any point in time this wheel can have velocity that consists of two components. Component number one is along the wheel dimension and it's basically generated
12:19
by the rotation of the motor attached to this wheel. So when the motor rotates the wheel is moving. Component number two is not generated by this motor and it has a direction that is orthogonal to the main direction of the wheel. And if you have one component directed this way, one component directed this way, then eventually your wheel will start to move diagonally and it can move in absolutely any direction like this. So you just need to combine the rotation of the wheel
12:54
with this sliding thing to the rollers and you will get omnidirectional movement of this wheel. This is why they are called omni wheels and they allow to move the robot in any possible direction in any combination of velocities both linear and angular when you just have these three wheels and control them by rotation speed of three motors. So let's dive deeper into the kinematics of the robot that has these three wheels attached as in LeKiwi. Let's try to derive formulas
13:36
for the forward and inverse kinematics for this robot. Let's say that our x-axis is
13:44
looking forward and corresponds to forward movement and y-axis is orthogonal to it and
13:53
positive numbers correspond to movements to the left. So in our case then the positive direction for rotation will be counterclockwise. And also we can describe the position of the wheel by just providing the angle between the mounting like this line connecting the center of the robot and the mounting point of the wheel and the x-axis. It's important to mention that if you want all formulas to work for all wheels and be universal, we need to pay attention to the
14:32
direction of the angles. Okay. And finally, let's denote the velocity of the center of the wheel as V and represented by this purple vector. And this velocity as we discussed have two components. So the one component is the actual speed of the wheel rotation that will exist for any type of wheel. And the second one is orthogonal to it velocity that comes from the rollers. And this velocity, this part of velocity we get for free. And yeah, these two vectors,
15:07
the orange one and the green one, they sum up, we will get the total velocity of the center of the wheel that is mounted to the robot. And yeah, finally we can see that the angle between this roller related velocity and x is the same as alpha. And then the angle between this green velocity of the main velocity of the wheel and the x is just shifted by 90° because there is 90° between these two vectors. So it will be alpha minus 90.
15:48
And also you can look at this triangle and you can see that if we know this total velocity and we know the angle between the green and the purple lines then we always can compute the velocity with which the center of the wheel should move in this main direction by just multiplying v by cosine between v and vw and we will use this formula for all the derivation in the next pictures. Now let's consider a few simple examples. Example number one when we want to move the robot strictly
16:32
forward without any side movement and without any rotation. So here by this red line I just show the direction of robot movement. So this vector is not really equal to the actual speed but it just shows the direction. But if our robot, the whole robot is moving forward with some velocity then every point of this robot including the center of the wheel will move with the same velocity forward.
17:02
So it means that the center of the wheel should also move strictly forward and it means that our vx equals v. And then we can just put this vx into this formula to get this velocity of the wheel, the green one that we want to find. So we just put it there and then just put all the angles that we know and we can get that velocity that comes from the rotation of this particular wheel should be simply equal vx multiplied by sin alpha. Then let's say we want to move to the right
17:40
or to the left and we have some vector vy that represents this movement of the robot. And again the logic is the same. So all the points of the robot will move with the same velocity in this case this green velocity of the wheel should be simply equal vy multiplied by cosine alpha. So now we know the solutions for these two simple cases. And the third simple case is when the robot is not moving forward, backwards or sideways but it's rotating. So it means that all the points are
18:17
just moving around the center of the robot without the robot actually shifting in the space with some angular velocity omega it means that the velocity of every point of this object should be orthogonal to the line connecting the center of rotation with this point. So basically it should be tangential to this robot circle and in our case it simply means that there is absolutely no component that
18:49
is related to the rolling of the wheels sideways and the whole rotational velocity is the same as the velocity of the rotation of the wheel itself. So in this case the formula is very simple. So our green vw is simply omega multiplied by r where r is the radius of the robot. But now let's look at the more general example when we have some movements in all the directions and so we move forward or backwards and at the same time the robot is
19:25
moving sideways and at the same time is rotating. And the nice thing about it that we can simply sum up all the vectors to find the final vector that is describing the direction and the speed of the center of the wheel. So we take the component that corresponds to the movements forward along x-axis. Then we add the component that corresponds to the movement along the y-axis. And finally we add the component corresponding to the rotation. And if we sum all of them up, we'll
20:01
get the final vector representing the velocity of the center of the wheel that we should have if our robot is moving at the same time moving in all directions and rotating. So the nice thing that we can also simply sum up all the components and if you want to get the final value for our total VW then we should simply sum up all the things that we derived in the three previous simple steps. So if our movement is complex and consists of both x, y
20:36
movements and the rotation then the final formula for the required velocity of the center of the wheel will be just sum of all these three formulas that we wrote. But we usually don't care about the linear velocity of the center of the wheel. What we care about is the rotation speed of the wheel because this is what we can control by sending some commands to the motor and but it's very easy to convert this velocity into the angular velocity
21:08
of the wheel rotation. We just need to divide everything by the radius of the wheel. So if we do it and if r is this radius then this will be the final formula and again as I said in the beginning the difference between these three wheels is only the angle at which they're connected to the robot. So formulas for other two wheels will be exactly the same. We just need to replace this alpha with different corresponding alphas. And if you write it down in the matrix form,
21:48
we will get this simple matrix that converts the velocities of the robot into the angular velocities of the wheels. I use numbers 7, 8 and 9 in this case because first six numbers from 1 to six I used to refer to the motors in the arm itself and these 7, 8 and 9 are officially used as numbers of the motors corresponding to the wheels of the robot. So I use these numbers. Okay. So this is called inverse kinematics and this is what we just derived. So our
22:27
inverse kinematics again in the matrix form looks like this. And we can for simplicity call this 3x3 matrix A and if you want to derive forward kinematics we don't need to use any geometry. The only thing that we need to do is just to solve this system of linear equations and
22:53
we will do it by simply inverting the matrix A. So if you invert matrix A then we will get formulas that can transform all the angular velocities of all our motors into the actual velocities of the robot and you can solve it analytically. So if you solve it analytically this will be the solution.
23:22
But for example if you look at the code of LeRobot then they don't I think even solve it analytically they just use NumPy to invert this matrix and it works just fine. So it's pretty fast. So you don't even need to solve it but I solved it anyway. And I think it's more straightforward and uses less computations if you already solve it. And another thing because I already mentioned the code of the robot I think in the robot in the library they use slightly different notation maybe
23:54
slightly different coordinate directions. So the actual formulas that they use are slightly different from what I use here but they are actually identical. They just again a little bit different definitions of how they compute angles and how they direct all the x and y axes. So idea is completely the same there even so formulas can look slightly different. And finally for our case where we know actual alpha for all the motors we can compute the
24:31
actual numbers all these sin-cos things and this is what we get and when you look at these things you can actually find some simple things for sanity checking. First of all you see some symmetry in the thing. So we have two wheels that are right here kind of symmetric, the two front wheels. So whenever we're talking about movement forwards they should move in the opposite directions such that both of them are actually moving forward. So this is why we have this one
25:08
has negative sign, one has positive sign but the absolute values of these two numbers are the same. So this is because of the symmetry. And then if for example you look at this zero then this zero says and then the corresponding zero here. This zero means that there is no impact of the back wheel, wheel number eight on the speed or velocity along the x-axis. That also makes total sense because the back wheel is completely orthogonal to X-axis and all the movements of
25:42
this wheel along this axis are only because of the rollers. So you never need to rotate this wheel to move forward or backwards and vice versa. If you strictly move forward backwards, the back wheel is not rotating. And this is the meaning of this zero. It also makes sense. And another thing that also can come from the symmetry is that this column and this row they have the same numbers. This is because this column and this row they are corresponding to the
26:16
angular velocity of the robot to the rotation of the robot and all the wheels are symmetrical
26:24
respective to the center of the robot. So the actual rotation should always be the same. There is absolutely no reason for them to be different just because our robot is completely symmetric. So yeah these are just some sanity checks that you can do when you look at the numbers like this. But overall I think that this is a nice exercise if you want to better understand how all the kinematics of different types of omni wheels work. It's a nice thing to try to
26:58
derive it yourself. Also, if you want to make some robot with unusual setup with unusual location of the wheels or some different wheels and so on, you can always use the similar logic to derive
27:14
all these equations. Yeah, that's it. This is how this robot moves and how we can connect the angular velocities of wheel rotation and the actual robot movement velocities.
27:30
Now when we know how to control the robot, we can start recording the dataset. So dataset is simply recording of you teleoperating the robot to complete some task. In this example, I want to just show a very simple task where my robot grabs this Boba tea toy and to teach it to do it autonomously. I recorded around 60 episodes in different parts of my condo in a little bit different locations with a little bit different lighting and different
28:09
backgrounds. And yeah, you can see some of them in this video. For this exercise, I use only two cameras. So I don't use top view camera. Here you can see it's disconnected. I operated the robot completely remotely. So I was sitting in a different room. So I couldn't see what's happening with the robot from the outside. I was just relying on the video from cameras from the robot itself and was controlling it using the keyboard and the leader arm. When you collect
28:46
dataset to train the policy for the arm only, it's usually enough to collect 30 to 50 episodes for a simple task. In case of mobile robot, we want it to be more robust. We wanted to generalize better for new environments. So this is why you need to collect more data in general. So for this simple task, 60 was more or less enough, but I still feel like I needed more to get even better quality.
29:12
After you collected the dataset, you can train the policy and your robot will learn how to do this task autonomously. Policy is simply the machine learning model that controls your robot. It gets the signals from the cameras and predicts what should be the next action. And there are a lot of policies that are already implemented in the LeRobot library and you can use them out of the box. I also developed my own policy around half a year ago. It's called DOT policy and it's pretty
29:47
simple and small policy that works pretty well on simple tasks. So I decided to give it a try and try to use this policy for LeKiwi. I had some concerns before I started to use it because one of the things that I assume in my policy is that what we predict are actual positions of the motors but in case of LeKiwi you also need to predict the velocities and there is slightly different logic in how we aggregate the predictions and so on so
30:20
forth. So I wasn't sure that it will work but eventually it worked and it's pretty
30:29
nice policy that works with LeKiwi as well. Here you can see the actual policy rollout. I simply trained it with mostly default parameters using the 60 episodes that I collected. And
30:56
I would say that it learned to complete the task. Maybe not as good as I would want it to do it. This video has the actual speed. So it's not sped up, it's the real speed. And I would say that the success rate is maybe around 50% in the locations where I actually collected the training dataset but it's lower when I move it to new locations. So it maybe goes to 20 or 30% when I move it to locations with new background that the robot didn't see before.
31:36
So I guess that for the actually generalizing strategy we need much
31:44
more data and more diverse data. Also you can see here that yes quite often the robot fails as I said the success rate even in the known locations is around 50%. But I would say even 50% is pretty good after 60 observations for this robot given as I said that I didn't even expect that it will work well with velocities and it seems it works pretty well. I'm pretty sure if I collect more data and also try to optimize the policy a little bit maybe by
32:26
adjusting some parameters and so on then I can get much better results for this task because both parts of the logic of the process seem to work. You can see that the robot can actually move towards the target pretty well and when it's in front of the target it can grab it. So sometimes some issues happen but probably it's because of some strange location that wasn't represented in the dataset. You can even see that it can adjust its position.
33:06
It's not just moving forward backwards. It can move sideways. I didn't see it rotating much to be honest. It's probably because I didn't have too much rotation in the dataset. It
33:19
didn't learn how to properly rotate towards the target. But yeah, overall you can see that it works. It works pretty well. It works pretty fast and definitely DOT policy can be used with LeKiwi. Another important thing to discuss is performance and when we want to use LeKiwi as a mobile robot. It means that we can actually run the policy on the robot itself or we can run the policy remotely and then the robot should just send the observations to some another computer,
34:03
another server. We make predictions on this server and then send them back to the robot. I tested three different options. How to control the robot? How to run the policy on the robot? The first one is using Raspberry Pi that you've seen is installed on my robot. In this case, I run the policy on the robot itself without using any other computer, any other server. The second one is when I run the policy on my Mac. So, I have MacBook with M1 processor. It's three generations
34:43
behind the latest processors from Apple, I guess. But it's still pretty good. So, yeah, I don't have any GPU on this laptop, just this Apple processor. And finally, also pretty old, also three generations behind. I have a computer with GPU. It's 2080 Ti. Maybe you know 7, 8 years ago it was a powerful one. Now it's not considered powerful for sure. And what I wanted to test, I wanted to run three different policies on these three different
35:23
devices. So I used my DOT policy that has just 14 million parameters. I use ACT policy that uses 52 million parameters and I use Diffusion policy that has more than 200 million parameters. For all of these policies I use the same inputs. I use two images 640x360 and all of them predict 30 steps forward. So it means they predict around 1 second because my FPS is 30 for both the cameras. The only exception is Diffusion because of some parameter limitations. It needs to predict
36:06
32 steps but it doesn't make much difference. And for the first test I used n action steps or in case of DOT policy it's called predict every n parameter set to one. It means that I try to make the prediction of the model after each frame that I receive from the cameras and you can see the predictions per second or actions per second in this table. So you can see that when I run it directly on Raspberry it's very very slow. So I make one prediction per second for DOT,
36:51
even less than one for ACT and it's basically zero for Diffusion. It takes forever to make prediction on Raspberry for Diffusion model. Again I just use the model with the default parameters. On Mac it's much faster. So when I run it on Mac it's already pretty good. So 20 predictions per second is good enough to really run the policy in real life. So DOT policy works very well. ACT is pretty close also reasonable predictions but yeah
37:22
both of them we don't have enough performance to predict to make predictions for every frame and finally Diffusion is still pretty slow. And then when I run it on my computer with GPU, it's almost 30. Close to the case when you connect your robot directly to this computer. So the time that takes to send images and receive actions over Wi-Fi network is pretty negligible in this case. So it works very well. But Diffusion policy still stays slow. We make almost one prediction
38:03
per second. That is still not so good. But actually all of these models they have different optimization for this case specifically for this case. And one of the most common optimizations is instead of making predictions for every frame make predictions once in a while few steps forward and then make these steps, make the actions and then predict the next
38:32
steps. And usually it's controlled by this parameter n action steps. So I ran another experiment when I set it to 10 and you can see that results improved everywhere. So if you look let's go backwards this time. So on 2080 DOT and ACT basically work in real time. Diffusion is still pretty slow and it's very shaky. It's not nice how the robot is moving in this case. But probably with more powerful GPU or maybe with some optimization
39:10
of parameters you can make it work because it's already started to be in the reasonable range. Then on my Mac also ACT runs very fast so it's actually very usable. Diffusion probably not so usable. So if you want to run Diffusion models I would recommend you to have some GPU. And on the Raspberry it's again it's better than one prediction per second but it's still pretty slow. And also you can see that DOT policy gets smaller boost than ACT policy. In DOT policy
39:46
I use multiple past observations. And this is why you still need to make predictions for some past
39:55
images from the cameras and the improvement in speed and performance is not as big as in ACT. So when you run it on Raspberry Pi they actually are very very similar. Yeah. So the first conclusion is that if you run it directly on Raspberry Pi it's actually pretty slow. So maybe it's not the best option. So there are other single board computers out there. For example Jetson series from Nvidia should be much faster for this type of problems than
40:28
Raspberry but I don't have any latest generation Jetson at home so I didn't try to test it even so I've seen there is official version of LeKiwi using it so I think it can be a good alternative if you want to run it locally but the good news is that even over Wi-Fi, even over my home network. There is no extra delay and I can run the policy either on my MacBook or on my computer with GPU and they work pretty fast. So it's actually cool. It means that in these cases at least in
41:05
my case of experiments at home, I don't really need a powerful hardware on the robot itself. Up to this point we're talking about the basic things. So how this robot works, how to assemble it, how to teleoperate it, how to collect dataset and train your policy. You can learn about all these things from the official LeRobot guide. So I think they're very nice. I would definitely recommend you to just go through them and read them and do all the things
41:34
one by one. But I think the most interesting part of robotics starts when you finish the official guide and you start experimenting. So when you start to change your robot, when you start to try to invent some new hardware, new teleoperation devices or maybe write some new code or try some things that are not available by default and I think yeah I think it's very interesting part and
42:01
I would definitely encourage you to try to do some extra experiments and try to go just beyond what's described in the instructions or beyond what other people are doing. And yeah, I think this is when you actually learn and also this is where most of the fun is. And in the end of the video, I just want to quickly show a few of my experiments of what I changed, what I tried and so on because maybe it will be inspirational for you and you will
42:34
come up with new ideas of what you can try. First of all I already mentioned it but just want to discuss it a little bit more in details. I use pretty modified version of LeKiwi. So some of these modifications I already covered in the previous video. So for example I use this gripper that is different from original SO-ARM gripper. It's symmetric. It's kind of transparent from the side let's say. And for some tasks it's much more convenient than the original gripper. But also
43:12
this one has some problems. So I think eventually I will replace it too. I will either develop a new gripper or will just find something that's developed by community or just sold somewhere. Another thing I experimented a lot with is camera mounts. So again in my old video you can see some other ways to mount this camera. I printed numerous versions of this camera mount. I tried to put it higher, lower. I experimented with different angles. I experimented a lot with
43:45
camera lenses. So sometimes for some tasks it's more convenient to use wide lens. For some tasks it's more convenient to use narrow lens cameras. So yeah, I did a lot of tests just try to teleoperate and try to train the policy or did some other experiments with different cameras. I replaced the mount for this forward looking camera. In the original version of the robot. If I remember correctly, this camera is mounted directly to the bottom platform and it's not
44:19
rotating when the arm is rotating. And I think that this setup is much more convenient and gives you much more information when you grab some stuff, especially when you grab it from the floor. So this camera that always looks in the direction of the robot arm is more convenient than the fixed one, at least in my experience. The only problem with this right now is that I cannot rotate it too much to the left or the right but again it's not a problem for mobile robot because this
44:56
robot can just rotate completely if it's needed. Then I also did some small things. So I actually
45:06
printed a little bit different brackets for wheels but these are small. I don't want to talk about it. I printed different bracket for the battery because I just use another battery. So all of these are small and yeah the big one, the big addition that you may see and also discussed is this next level. So when I was experimenting with the robot with the arm itself, I found out that the top view camera is very useful and really policies
45:34
work very well when you have this top view and usually I used another stand with top view camera in front of the robot. But for mobile robot it's important to attach it to the robot itself because
45:51
then you are not limited by your current position. So then it means your robot can go anywhere and it will always have the camera at the same place. So I developed this next level and put camera here. Again as I said before it's a little bit shaky because I printed this level very very thin. So it's not strong enough. I will just reprint it later and it will be much better, much more stable. So I think I will definitely improve it so it will be more strong and
46:25
less shaky. Again my robot is in the state of constant prototype. I always change some stuff
46:34
every week. If you look at this robot, it looks a little bit different. So at some point, yeah, I will make everything a bit more reliable. And yeah, another thing that helps a little bit with wire management because currently I have this camera with pretty short USB cable. So if I want to extend the arm very far, then it's fine when cable goes here, but it's not enough when cable goes directly to the Raspberry Pi. So having it here, having some USB hub here is very convenient.
47:08
And yeah, so I think these are all changes that I made for the robot at least at this point. I also experimented with some different things. But yeah, this is what I did and I think doing these changes, making them, applying to your own robot helps you understand much better why some of the design choices were made by the original creators of the robots or you can find some better design
47:38
choices or you can even come up with some extra improvements that maybe other people didn't think about and you can even contribute back to the community, contribute back to the design of this robot. So I really really encourage you to try to do some things to try to do maybe simple things or maybe a bit more complicated to modify the robot. By the way I share all the files for 3D printing that I use in one of the GitHub repos and I will leave the link in the description below.
48:17
LeKiwi uses omni wheels as we discussed. But there is another type of wheels that allow you to move in any direction and they're called mecanum wheels and they have very similar idea. But the main difference of these wheels is that the rollers are not orthogonal to the wheel but they have 45° angle between the roller and the main direction of the wheel. And generally in most of the cases you use these wheels in a four-wheel setup. So you have four wheels,
48:50
actually two pairs of wheels because two of them have rollers like this, two have rollers that are 90° rotated compared to this one. And then it has very nice stable kinematics of the robot. But there was a question, there was some discussion in Discord of the LeRobot whether it's possible to make three wheels like LeKiwi with these wheels and I also I had these wheels at home so decided to give it a try and you can see a short clip right now of how it
49:23
works in practice. So I would say it's kind of possible but the results are not so nice because
49:32
when you derive all the formulas you ignore all the slippery, all the friction and so on and then eventually you make the non-symmetrical setup, all of these problems become very significant. So you can see that it's not very nicely moving in all the directions but I would say it's a nice exercise. So if you want to understand better how kinematics of omni wheels or mecanum wheels works, try to create some very unusual non-standard setup of your robot,
50:08
try to write down all the formulas and derive what are the actual kinematics formulas and then try to implement it and see how it works in practice. So, I think yeah, even so, I wouldn't recommend anybody to really use this type of wheels. They're not supposed to be used in the three wheel setup. But as a fun exercise and some learning exercise, it's actually pretty cool. And if you already have these mecanum wheels at home, just give it a try. I think it's fun.
50:41
The last experiment that I want to talk about shows that in the age of LLMs, you don't even necessarily need to train your own policy and collect your dataset to make your robot work autonomously. I already talked about this project. I have a long video about it. But briefly speaking so I developed MCP server and AI agent that can control the robot by using some LLM. I found out that in my experiments Claude works the best. So I want to
51:17
demonstrate how it works with this robot and yeah when I made my previous video it worked only with SO-ARM but I updated it. So now it supports LeKiwi. Now it can even directly run on the let's say Raspberry Pi of LeKiwi itself because it doesn't require any computation and you can create the really autonomous robot controlled by LLMs. So currently it still controls only the arm. I didn't upgrade it to control the actual mobile base. But I will update it soon. So I think it's
51:59
not so hard to make it also move the robot and let it roam around. So yeah, let's see how it works. Let's look at some demonstration. Let's run the command line interface of the agent and see how it works. So this interface allows me to simply chat with the robot and let's say ask it to grab a toy. And you can see that it's communicating using Claude in this case and it can request or can run
52:34
some simple MCP tools for example to request some extra instructions or to request the position of the robot and the information from cameras and then by analyzing this information, analyzing the pictures from cameras it can decide what to do and then send actual commands to the robot. I didn't train it on any tasks. It's completely zero shot. It's just pure LLM controlling the robot based on just one simple request that I wrote in the chat that you've seen. So yeah,
53:15
let's see if it will be able to grab it this time. Looks like pretty close.
53:25
You can also see all the thinking process right there in the chat. So it's thinking where,
53:37
providing you detailed information of what's happening, what should be the next steps and so on so forth. And let's see it's almost here. Just need to grab it.
53:57
Okay. And let's say lift it up 20 cm. Okay. And drop it.
54:23
So yeah, you see that you can control your robot using the simple interface and command line that communicates with different LLMs. And it's cool. So I think it's on the other side of spectrum of possible ways how to control your robot and yeah I will also keep experimenting and will try to do something in between like simple policies and LLMs that are completely controlling the robot. But yeah if you want more details again just watch another video where I explain how I did it,
55:02
where I explain exactly how it works, do some code overview and give much more details about it. This is it for this video. I hope you enjoyed it and learned something new. Thank you for watching. If you have any questions or any thoughts, don't hesitate to share it in the comments. As for me, I will keep experimenting and share my experience and results here in this channel. So like and subscribe if you want to see more. And yeah, that's it. So see you in the next video. Bye.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat