Skip to main content
Loading Explore guides…
Robot Motion Learning with PD Prior · Acedra
Home
Explore
Engineering
Robot Motion Learning with PD Prior
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
100 segments
Copy
Hide video
0:05
So without further delay, let's get started. Today I'm going to talk about my work on robot motion learning with physics-based PD prior. So first let me introduce like what is the problem of motion planning and uh motion planning is a way for an uh agent to coordinate its behavior from a given start to a given goal while satisfying all the desired constraint. So whenever a dynamical system or a robot need to move they need to solve this motion planning problem. Okay. And it has
0:39
application or these are the application that my lab focus on like ranging from uh full body motion planning. Here there is a mobile robot mobile robot. It is moving uh object from a given start to a given goal. Then we also solve this problem in dynamic environment where we dynamically place a obstacle in front of the robot. Another area that we are very exciting interested uh in is in reactive manipulation. While robot is solving manipulation planning problems under
1:09
constraint, they may experience some disturbances. How robot dynamically adapt to those disturbances and continue to solve the assigned task. Lastly, uh we are also exp applying these uh motion planning tools to uh reactive multi-agent planning where you have large number of agents, they are performing their assigned uh task and then they will have to reactively avoid collision with each other. So spreading uh across all these problems the the core issue is how to
1:41
solve these motion planning problem. How to come up with tools that are efficient and the focus my lab has is on like developing these tools that are real time that enable real-time coordination of robot movement in unstructured constraint environment with minimal pre-training or trial and error. I have been trying to solve uh or achieve this objective for quite some time. During my undergrad, I started with uh sampling based uh motion planning method and my focus was on coming up with adaptive
2:15
sampling techniques so that these sampling based method can construct the tree as quickly as possible. Then one branch of that tree is your path solution from your start and bowl. And then during uh my PhD I moved toward uh extending these methods to uh datadriven approaches. So the issue with a classical sampling based techniques work they were computationally very slow. As the robot dimension increases like let's say you are planning uh motion for a humanoid robot. It's a huge degree of
2:46
freedom for those kind of system their computation time became very slow. So we could not do many real-time application because of the slow computational speed of those method. Then I moved toward datadriven approaches in in 2018 or 19. Before that many people were trying to solve motion turning problems with these uh neuronet networks but they were missing one key piece and in this these papers I gave the recipe that just having neuronet network is not sufficient like you can
3:19
do imitation learning and still it won't learn. You need stoasticity in these models and you can incorporate stoasticity in different ways. In our work we use dropout. Then people moved took up that idea moved toward using variational autoenccoder. Now you use the diffusion model. They they have some sort of stoasticity. They start with some noise and then den noiseise it over time. So there is some sort of randomness incorporated into the process that helps these models plan motion. So
3:50
the core recipe is you need neuronet network then you also need some sort of uh stoasticity in the process so that because these neuronet network they are approximating the solution they can get stuck into local minima these den noising process or starting with some multiple uh seeds you basically help these neuronet network come out of those local minimas. Okay. But my main issue with these models were like the training cost. So offline you we had to run these classical methods to gather
4:24
all the data and then you were training these models on that data and yes during inference it was giving fast inference but if we think about it were they really solving like uh overcoming these slow computational uh speed of these method like because if you add the training time or time spent on getting the data and then combine it with inference time you may think that it's not really beneficial. So when I started as a faculty at Purdue, I started looking at some other ways to train
4:57
these neural networks without expert demonstration. And there are three features that my lab focus on like one is inference efficiency. I want to develop method that allow immediate execution. They infer plans as quickly as possible. Then second feature that we look for is training efficiency. We want to reduce model training cost. We want to eliminate expert need for expert annot annotation and this is solitary transferability. Like right now if you whenever you talk to someone who is
5:27
doing imitation learning or training very large model the second question come up can they transfer to a different environment or unseen environment because the issue is when you move to a different environment you may need to reather all the data but if you have a model that's whose training cost is very low and that eliminates need for expert demonstration then you can transfer that model very easily to to a new domain. Third feature which is very important
5:54
like we want to design these models so that they adapt to high degree of freedom very complex terrains unknown environment or uh very complex constraint like involved in uh manipulation problem and they should also scale to large number of agents so that we can deploy multiple robots operating in uh in a given environment then they can solve some task in a coordination or or in a decentralized manner. So just to give a overview like where we stand in term of these features. So we
6:26
have optimization based techniques and these techniques they have high inference efficiency. We have very nice optimization based tools that can find a trajectory uh very efficiently. They have very high uh training efficiency. You don't need to re pre-train your optimizer but they do not adapt to very complex scenario. they are prone to uh local minima and they can often get stuck there. Then we have like classical uh tools for solving robot planning and control problem. We
6:59
have like um uh discretization based approaches you may know AAR you should and then there are sampling based approaches. There are other classical tools within control like random shooting method and uh techniques like that. These methods they have low training effic high training efficiency. You do not need to train them but their inference efficiency is very slow because these method they need to either discretize your entire space or construct a graph and then uh find a path through that
7:33
recent advancement. I I should acknowledge that people are exploring like how can we use massive parallelization within these method to speed up their performance. But it is we still have to see how these methods scale to complex uh kinematic constraints. Then we have datadriven approaches like imitation learning or reinforcement learning. We know they have like uh very low training efficiency but once trained their inference efficiency is very high and they also adapt to very complex
8:03
scenarios. But can we have a method like I have mentioned all these three categories. They were missing either one or more of those features. Optimization based they could not adapt to complexity. Sampling or classical method they had like uh low uh inference efficiency and then datadriven they have like very low training efficiency. Can we have method that have all these three features? So over answer is yes. like if you incorporate some uh priors that are available to us from physics you can
8:37
achieve all these three features and that will be the focus of my uh rest of the this talk. So one question that come up like when I say we use physics prior like what physics prior to use okay often people confuse this physics prior with a physics engine or a simulator but here in my talk I specified very clearly like we use PD prize because I was getting this question again and again but where is the simulation here so it's not a physics simulation it's a PTE that
9:08
we use as a prior so when we discuss PTE one PD that is you may have heard is uh HJBpd Hamiltonian Jacobe Bellman equation and it governs the motion of some dynamical systems but that PD is very hard to solve. It has like singularities multimodel problems and then when you apply both kinematic and dynamical constraint it become very difficult to solve. Then we analyze that equation and we said if you assume trivial dynamics that equation boils down to this econal PD. So this econal
9:44
PD has two functions. One is this T and other is that S. So T is an unknown function and it is a value function. We want to solve this PTE to find this unknown function T. S is a known function and it is basically it defines your constraints. So it could be distance to obstacles. So given the configuration of the robot, let's say QS, this S will tell me how far my robot is from the nearest obstacle. Is that clear? Or I can even change that constraint to something else like
10:19
instead of having a distance to obstacle, it could be distance to nearest manifold. Okay? It could be manipulation manifold or something. So this s function is known. I want to solve this PD to find this T which is a value function. And once you solve this equation, it gives you these wavefronts using which robot can navigate from one point to another. So given this source point, you solve this uh PTE, it gives you this travel time function which is this t. This travel time function is
10:53
also a value function. So just following the gradient of this T you can navigate to any point in the environment. So numerically there is a method called FFM fast marching method. It solves this PTE but it's a numerical approach. It cannot scale beyond three dimension. So when uh we studied this PD our main interest was can we scale it beyond three dimension to actual complex robot systems. So neuronet network gave us like uh a possible way like we can instead of
11:24
using numerical solver we can use neural network to solve this PD in high dimension. So the first work that we did you have this neural network that tries to solve this econal PD. So the input to this neuronet network. So this neuronet network is a black box but we put structure into this neuronet network that respect the properties of uh this PD. I will briefly discuss that but if you are interested check out the paper in order to understand the structure of
11:54
this neural network but for simplicity I will assume it's a simple blackbox neural network that take robot start QS robot goal QG and environment perception as an input. The output is this travel time function. So this is an unknown function. So I want to train this neural network to solve this PTE so that once trained I have this accurate or approximately accurate travel time function. Now the way we train this model is you give this input and it uh predict
12:27
this travel time then econal PD says if you take the gradient of this travel time with respect to input Qs in this case this is equal to inverse of your constraint function. So this PD is giving us some structure. So the training become very simple. So you you sample bunch of robot start and goal configurations. You have environment perception for each of these pair. They go through neuronet network. You predict their travel time. Then you back prop to compute this approximated constraint
13:00
function according to your econal PD. Then your law. Yes. >> Question., So, how, how, do, you, encode, the problem in your efficient? Uh is it within uh SQ or uh >> that's, a, good, question., I, I, will, come, to that. >> Okay. >> In, a, moment., So, QS, is, in, this, case, is, so the motion planning problem is defined by robot start goal and environment perception right standard motion planning problem. You give it to classical method or optimizer. it take
13:31
the start and goal and your environment like distance to obstacle or whatever observation you have available right so that's define the problem and in this case we are giving only this problem set so QS and QG and environment perception goes through neuronet network and you predict this T then the way we train this loss fun this neuronet network is through gradient matching loss function so we back prop compute this gradient with respect to QS and then econal PD
14:00
says inverse of this gradient norm is basically your constraint function. Okay. So we approximate this constraint function and compare it with ground truth constraint function like your ground truth distance to obstacle. Is that clear? So the loss function is on the gradients of the neural network. It's not on the forward pass and then you compute some output and then you define the loss function is on the gradient of the neural network. Yes. >> Question.
14:33
>> Could, you, clarify, about, the, output, of the dual network case and how do we use the output to gather the desired poses of the robots to read? >> Yeah,, I, will, come, to, that, shortly, but that's a very good question. So the output in this case is a travel time t in this case it's a value of function. So in order to get the gradient, you just have to follow the gradient of that value function and it will give you a trajectory. Okay. So we use this simple
15:00
gradient matching loss function. So if you think about the only data that I need is randomly sample start and goal and their distance to obstacle. That's it. So this is your standard motion planning tool is like if you use for example sampling based method like RRT or RRT star they require some clean checker and way to randomly sample your configuration and then they have some machinery to find the trajectory right our data is similar we are requiring randomly sample robot configurations and
15:30
their distance to constraint or client in this case and then we use that constraint function as an expert to train our model Because over neural network gradient leads to approximation of constraint function and then we compare this approximated constraint function with actual. Does that answer? So theoretically it it all sounds simple but when we actually applied this we could not scale this method beyond four dimension. So numerical solver was solving it for three dimension. We
16:03
showed that okay we can do four dimension but numerical method could also do four dimension. it was just little bit faster and then we started investigating like what's the issue there were and we found out there are two main limitations one there is like econal PTE has multiple solution and our neural network is unable to capture those multiple solution so so it it is suffering for from that uh multimodel problem second because I am training this model on randomly sample start and
16:34
goal the gradient between my consecutive configuration remains uncontrolled. But if you think about the trajectory, it's like you start from start, you go to next configuration, then you go to next configuration and it gives you a trajectory. So if my gradient between my consecutive configuration remains uncontrolled, it can lead to error and I will show you shortly like how how that looks. So these were the two limitation we found that we thought is basically limiting our model for from scaling
17:05
beyond four dimension. So the first solution we explored like can we use viscosity econal PD. So one of the problem I highlighted econal PD has multiple solution. So if you add this lelassian then this econ has a unique solution and that did lead to like much better performance with this model. We were able to scale to up to like uh six draw robot arm and it could apply motion in very narrow passage but this uh viscosity conity came with a cost. So now I have this double derivative of
17:45
neural network with respect. So I have this lelassian. So that was very computationally expensive to compute and training cost went very high. So that's again was against the objective that I wanted some model that is very fast to train. But this viscosity conal PD did lead to better performance. However, we still did not do anything to regulate the gradient with between my consecutive configuration. We still trained this model on randomly sampled start and
18:15
goal. There was an other technique that we proposed along the way that you can approximate this lelassian with rishlay energy minimization. I I won't go into the detail but if you are interested uh check out this paper it's a recent IROS uh publication. So it discusses like how can you use dishlay energy minimization to approximate lelassian as you train your model. That's a cheaper approximation of uh this lelassian. Still it takes lot of iteration to train
18:47
because you are over time approximating this lelassian. So at the beginning your model will perform very bad. So coming back to those properties that it has multiple solution this in and it has like and the gradient between convective uh configuration remains uncontrolled. We found out that solution to these problem rely on within the properties of econal PD. So econal PD is a is a judacic distance. The solution of econal PD is a judacic function. So it should respect the properties of a
19:25
judacic distance like triangular inequality and there are different properties of those judacic distances that it should satisfy and judacic distance it also has like multiple solution. There are it covers captures multiple judacic like for example if you have start and goal I can either go this way or I can also go this way. So there are multiple solution available and all of these solution may also be shortest uh travel time solution. So there could be some symmetry in in your
19:56
configuration space or something there could be multiple solution all of them can be equally relevant. Second properties is that the solution of econal PD is a value function. So it should satisfy Bellman principle of optimality that I will discuss shortly. So let's start with first property. The solution of econal PDE is a judacic distance. So so we should use a metric learning that enforce the prediction to a valid metric space so that we respect all the properties of a judic. So we should
20:33
ensure consistency with judesic distances like it has uh symmetry property. Uh it has a triangular inequality like your your path straight line path between start and goal and your path through some middle point the length of that path should be higher than this straight line. Okay. So that's the triangular inequality. So this udesic distance it should neural network should satisfy that property. So we proposed this uh structure for uh this neural network. What this structure is
21:06
doing before we were doing this. So consider this problem. I have two paths from A to B. If I just use L2 norm in my loss function, it will squash them into this straight line. But if you use this loss fun this structure that we have it is taking your QS pushing them to some latent state and then it is applying some max pooling on them. So what it is doing it is peacewise approximating this solution. So you see you get this diamond shape approximation of your path. Instead of
21:39
these circles you are peacewise approximating all those multiple solutions. So, so this metric learning here, it takes your QS and QG pass them through some neural network and then it does something with the latent features of those QS and QG like it apply this max operation and then it applies this uh infinite norm uh on that latent encoding. So if you think about it, we move this start and go to some latent space and then learn the judicent space. Is that clear?
22:15
Okay. So that was the first change we did. Second, the solution of this econal PD should is a value function. So it should respect Bellman principle of optimality. And then we found out this is actually solving this can solve the problem of over gradient being uncontrolled between my consecutive point. So think of this simple econal PTE. So it is absolute function. So I have this absolute function and the way I am training my neural network is to randomly sample those green points.
22:47
So if I train my neural network on those point it will only minimize loss function on those points. It won't do anything on those segments. So for that neural network both functions are accurate because my error on those green point is zero. But if I have little bit jump in between them, my neuronet network is unaware of that because I did not do anything to regulate the gradient between those conductive point. But if you think of robot trajectory, it start from one point and then you you keep
23:23
going. If you have these little errors in between, your trajectory can go away from your desired goal. Does that make sense? So we had to regulate this gradient so that these bumps that you are seeing it should capture this absolute function. So the solution to this problem is uh by treating a corner PD solution as optimal value of function and it must satisfy Bellman principle of optimality. So when we say it must satisfy Bellman principle of optimality, we have to do
23:58
temporal difference learning. How many of you are aware of standard Q-learning? Deep Q-learning. Some like most of you are aware. So in Q-learning you have your reward function. Q function at your next state next action minus Q function at your current state current action. Right? So that's your Q learning and it it came from your temporal difference like value function. You have reward value at next state minus value at your current state. Right? So this equation is similar. We
24:36
show complete derivation on how we arrive at this equation. But the main point is this is a value function. So you have your QS. We perturb that QS towards some direction. So we have value at next state plus some reward and value at your next uh previous or or your current state. So t is a value function. We perturb our qs in some direction to get the next state. So this is value at your next state. That's value at your current state. And this is the cost function which is
25:13
inverse of your uh constraint. Whatever you give like it's a distance. So if I want to minimize or or maximize distance to obstacle, so that's your reward function or the cost function. So when you combine this temporal difference learning, so in our paper we show how you can analytically compute this U. So how can you perturb this QS in a direction that will give you the next conceptive state. So basically this u is uh short answer just the gradient of your travel time. So you move a small
25:50
step to in the direction of your gradient of your travel time. So that's your next state. Okay. And we can an analytically compute that gradient. So then we combined our gradient matching loss with this 3D loss and then this model was able to perform really well. We were able to scale it to very high dimension problems that I will show you shortly. So first on a very simple problem. So this is maze-like environment. If you combine your uh our approach like gradient
26:26
matching loss with TD learning, you get these kind of contour and you can see it is matching this expert uh FFM. It is a numerical approach. It can only go till three dimension and if you have very good computer it can do four dimension. But you can see time contours are almost similar. So when you do this uh uh neuronet network approach it can capture similar solution. However these are two method it's our first method it was without any TD learning or metric learning. You can see the
26:59
gradients at the source at the beginning are very good but as you go far from your source point the gradient start becoming very bad. So the error between your consecutive state start propagating and everything goes wrong. And this is viscosity econal PTE. It still has better solution than this one. But still it could it did not regularize uh gradient between my consecutive state. So still there are some artifacts that you see in that plot. So combining gradient matching with TD learning leads
27:34
to like much better result. So now I will discuss one by one inference efficiency, training efficiency and adaptability of this model. So let's start with inference efficiency. So this is on a simple uh seven uh setup. So overs you can see it's like take07 second to find that plan. And this is uh empire is uh from Nvidia. It was trained on huge amount of data. I will show you. It took them several weeks to gather that data and it took them one week to train that model. Compared to our model
28:14
it took us I think I will show you less than uh less than a minute to gather data less than an hour to train like uh this model on 300 environments. So this is one model generalizing to 300 different environments. Still empireet and uh overs you can see success rate are very similar and planning time uh is much faster in our case and for lazy PRM these times only show graph query time. We did not count graph construction time. So we pre-constructed the graph
28:52
and then use that graph to uh query the time.
28:59
Then it it also stayed to like a very complex uh indoor environment. So this is multifloor environment and success rate again you can see are very high and planning times are very low compared to like uh other approaches. So there is a difference like because it's a value function you can incorporate this value function into MPC or MPI. So any available tool that can use your value function you can even incorporate it into trajectory optimization. So we show
29:30
if you combine it with MPPI it perfor it is much faster because then you don't need to compute the gradient you just use the uh prediction directly the cost to go or the value function output but this gradient you can also use the gradient of this travel time to find the trajectory still uh it the computation time does not increase significantly and the success rate is high and then we were able to scale it to 12 dos. This is a very complex environment for these models because it has very
30:04
thin obstacles and these PD solutions struggle when you have these thin obstacles. So we specifically picked this environment to show it can nicely capture these thin obstacle and still uh do planning. This is 15 dos and this environment is particularly challen challenging for this robot. This robot is very huge and this room was very small. So it's a very confined environment for this robot to be able to move from one point to another. So that's about like inference
30:36
efficiency. I I showed you these models like as you scale to high dimension they retain their computational benefit. Now my favorite slide I I want to talk about training efficiency.
30:50
So these are different environments. So if you just pick this seven do environment for that environment it took like several uh like uh for us it took us 50 minutes to gather data. So per environment it was taking few seconds. So multiply by 50 it took us like 50 minutes. Training took 46 minute compared to and we compared it with empire and these numbers are taken from their paper. It took them several weeks on a large amount to gather large amount of data and then one week to train. We trained
31:24
our model on one GPU standard 3090 and but Empire was trained on u eight Nvidia Tesla GPUs. So it took several uh a huge amount of compute power to train that model and uh these Gibson environment like you can see uh it it takes 24 second to gather data 9 minute to train. So what I'm I want to highlight here when you use these PD priors you have like much higher training efficiency like the way I see physicist gave us these physics models that govern some dynamical system. Now we are
32:04
reathering the trajectories ignoring these models just to reproduce these models. Okay. So why not just use them? they are there and if you use them you will have very high training efficiency you will have very high inference efficiency and these models can still adapt to very complex environment and these models are transferable so when I started this research our models were training like two days to train like uh training would go two days because we were not satisfying
32:36
properly the properties of these PDS but with these properties we satisfied it came down to like less than an power and my lab is still pushing further to go beyond that like uh we we we are still in preparation for a paper I think our latest model like uh on on 12 do it takes like less than 5 minutes to train so it's much faster yes >> constraint, that, you'll, have, in, like, the s uh in the s function is that only just the travel time or you have other constraints
33:09
>> no, s, is, the, constraint, function, It's, a distance to ken. >> Yeah. >> So, it's, not, a, travel, time., Travel, time is t unknown function given this constraint function. So we solve that PD to find this travel time. >> And, also, is, the, model, architecture between I guess like between like the MP and your paper how different model architecture. >> So, our, model, architecture, are, very different. So our model architecture is designed to respect some properties of a
33:39
conal PD. For example, I will give you one property like uh it has the symmetry property. So travel time from your start to goal and travel time from your goal to start shoulds be same. So it has the symmetry and our architecture is designed to respect that symmetry. It enforces that this value function respect and then other thing I mentioned our value function is a judic. So architecture is designed to respect those triangular uh inequality and those kind of things. Whereas empire is a is a
34:09
standard standard architecture that we usually use for large models. >> Yes. >> Um, so, I, think, it's, a, very, good, idea, to apply econo PD for a manipulator task. Uh because I see in the econom you don't have the dynamical constraint uh in it. So in every state you probably assume the manipulator can move any direction. Uh but in reality it still has some dynamical constraint and you have singularity in the manipulator. So have you experienced any? >> So, that's, a, very, good, question., So
34:47
currently we like in this work we consider kinematic. We have a paper accepted at new reps this year where we solve kodnamic problem with these econal PD and uh we basically show that this econal PTE can still regular regularize the solution to HJBPD. So it can you can think of precondition of warm start to some HJBPD solver and we showed like some kinodnamic like uh on a humanoid task like how we can do contactrich uh uh locom motion and we also showed manipulation if I remember
35:23
correctly but and s point you pointed out to something else as well like uh manipulation I'm coming to manipulation shortly. Yeah. Yes. Uh just just wondering when do you need to do the data generation and training? Uh see the chain a little bit location are we obs. >> So, in, this, case, yes, so, we, have, a, uh, I will discuss one more work like we can selectively retrain neuronet network like I will show you how you can select parameters that you need to retrain if
35:55
some part of the environment is changed. You you do not need to retrain everything. So and the purple training I >> you, can, give, the, the, arbitrary, start, and the location. >> Yeah., Right., Okay. >> Yeah. >> So, so, far, we, have, discussed, training efficiency adaptability to complexity like I have shown you that it scared to very high dimension but I want to discuss like how it can scale to manipulation or or very large environment. And neuronet network has
36:22
this problem like if you have a very large environment it struggle it suffer from spectral bias issue of uh your neuronet networks. So first to scale it to manifold or manipulation problem all you have to change is your expert speed function instead of distance to obstacle make it distance to constraint manifold. Like if for example you are doing door opening task that's one manifold and if you can compute the distance of robot configuration to that manifold you can
36:52
use that as a training function and then we were able to solve this like uh so this model is like opening door and then it will move that cup without tilting from one point to another in in this confined space. So again you can see uh computation times are very low success rate is very high and we compare it with C by RT which is a sampling based approach. Compnet X is a data like imitation learning based approach. So Compnet X I proposed during my PhD uh to solve these
37:26
uh manipulation problem but you have to gather data using some C by RRT or classical planner to train that model. And then question come like how can we scale to multimodel problem like for example if you have this cup you are moving it keeping it upright then you are tilting these are two different constraint and then this robot also need to open cabinet retrie something and if you are doing steering so that spoon motion is one manifold then steering is another manifold if you are doing
37:56
hammering or all your tool manipulation they are multimodel uh uh constraint problems. So how can we extend these methods to solve those multimodal constraint? So the the solution relies uh in in in classical PD literature like in if you study classical PD literature they used to represent PD solution as a sum of finite basis functions. Okay. So if each of these basis function represents a manifold then I can combine them to get my global trajectory. So this is what we do. So we decompose our
38:35
problem into so this is just for depiction. Think of this 2D environment. I can break it down into multiple pieces and then for each piece I can learn some basis function and then I can combine the basis function to get my global value function. But there there was a challenge there. So I want my value function to be continuous like fully spatially connected because in trajectories if you have these decomposition you can have singularities at all of these partitions.
39:08
So how can we get this global value function so that I can go from any point to any point from this segment to that segment without experiencing any discontinuities or local minima. So that was the challenge and then we proposed some architectures if you're interested you can go into the details there but uh this architecture basically deco compose your problem into subdomains and then learn this global function. So that addresses your question like if some
39:37
part of the environment change you don't need to retrain everything you just need to retrain that basis function and that's very fast. So we were able to scale it to very large environments uh and uh it still retain like very high success rate. Another advantage of this decomposition you need fewer parameters compared to your end to end function. So that's what we found and then this is like on a real our campus like this robot is navigating these multiple narrow passages from one point to
40:12
another and this entire environment was decomposed into uh multiple basis functions and then the same idea applies to your multimodel manipulation. You have multim model multiple manifolds you can represent them as basis function you can combine them to get trajectory. So this is one neural model doing uh door opening and then it will retrieve this cup without tilting and then it will do this pouring task.
40:48
We recently moved to humanoid robot. So all these tasks are being solved by humanoids. So so this decomposition is also on the space. So robot can move anywhere in the environment. So that's uh decomposition on the plane. Then we have de composition on the manifolds. So all these like multimod model problem is being solved and this paper hopefully we plan to submit to a a conference very soon. Uh but we can solve multimodel manipulation problem that you see very
41:19
popular with imitation learning or datadriven approaches nowadays. We can solve all those problems uh with these models. Another area that we extended to like unknown environment. Right? Now we assumed we know the environment and then from that environment we computed the uh distance to obstacle. I need to speed up. So so one thing that we think motion planning problem is computationally expensive is because your mapping feature is not good for motion planning.
41:49
There is a huge gap between your mapping features and your motion planner. Like you have the sign, distance field or occupancy map that necessitate the building of some computationally expensive tools for motion planning like for example to translate those map into CPACE you either have to use sampling based technique or optimization then you find a trajectory in your C space. So can we have a better mapping that does not require this uh very complex uh tools. So answer is yes. If you can learn
42:23
uh maps as a travel time function then you don't need any planner. You just follow the gradient of that travel time and it it will give you a trajectory. So we we investigated that. So as your robot explore the environment it gets the depth perception and you can locally approximate your constraint function. So as your data is coming you can train this model. So you here this video is showing that robot is exploring the environment perception is coming in stream and it is
42:54
building this arrival time field map and then once you have this map you don't need any motion planning tool just follow the gradient of this map it incorporates that geometrical representation that helps robot navigate the environment and so over this paper the mapping time was twice than uh standard mapping like occupancy maps but our latest model we were able to reduce this time by like 40%. So uh each time like in each frame neural network just take two second to
43:31
train in this case but our latest model I think it takes less than 1 second to train as robot explores the environment and it even scales to like uh uh so this is a rare robot mapping this environment.
43:51
So you can see as it is exploring it is training the model on the fly and uh to get those travel time. So if we can get these maps in almost same amount of time as your occupancy map then you don't need any motion planner. You can just deploy your robot anywhere in unknown environment and it will figure out how to move in that environment with these plugandplay models. And another advantage of this like you can even do manipulation with it. You can equip robot with your inhand camera
44:22
and it will explore and construct these arrival time field maps in C space and still yet it can still like navigate. So we show that in our uh TTRO paper if you are interested check this out. Okay. So this is like the robot arm setup we had. It was using this inhand camera to construct this arrival time field map in six dimension C space. Another area we are interested in like how can we solve these PTEs for uh uh multi- aent setting. So basically instead of like JBPD you basically solve
45:01
JR you may know work from SL Benel here and uh the way we differ from his work is like we want to push the scalability of these method to very complex robot manipulator with very complex geometry of obstacles and this is one of our work like uh uh here robots are trying to reach their target while reactively avoiding cleon. We also scaled it to like uh assembly line task where multiple robots are performing their uh uh their task while avoiding clean. So uh if you're interested like check out
45:36
this work. So in conclusion if you should use physics prior because it gives you all these three features inference efficiency training efficiency adaptability in our future direction we are first we want to improve our training efficiency. over current models takes like five minute I want to achieve like real-time training so so that this neuronet network will be a plugandplay model that's my vision like if you have this plugandplay model transferability generalizing to new environment won't be
46:07
an issue because robot can just go there and train on on the fly and perform all your manipulation motion planning problems and other areas that we are interested in is reactive because these methods are very fast so if they can generalize life to very different settings. Even if you disturb the environment, they can still recover from it very quickly. And this is like a multimodel problem like where robots are planning for grasp and manipulation together. So that's another
46:37
area that we are extending. This is a new problem I recently got interested into. It's called uh assistive manipulation where robots are manipulating human body to do different tasks like for example assisted dressing is a manipulation problem. You have to respect biomechanical con constraint in order to move human body. I think it's a very nice motion planning problem to study and we are extending the methods to those domains as well. So this is a bed wiping task uh in a simulation like
47:10
robot is moving human limb from one point to another and other robot is wiping the body. So with this I will conclude. I'm almost on time. So this is my lab. So without them nothing would have been possible. So if you are interested check out our lab website and uh that would be it. Thank you. Any questions?
47:36
Yes, >> it's, thanks, for, the, great, talk, and, I just wanted to ask how how would you compare like your method to say some of the other more recent motion let's say geometric fabric complexity
47:57
>> so, so, graph, of, conert, like, I, think, there are assumptions these methods are uh uh performing like for example having a con convex obstacle. So that's one of the assumption like for example if you see what's the name of that there's a krobo if you see how krobo works they assume robot and environment is represented with multiple spheres and then they solve motion planning problem very fast okay and even these parallelization like this their test work on parallelization
48:31
they heavily rely on this feral spherical representation of the robot and environment And they solve this motion planning problem in basically in workspace. But there's an inherent issue there. Like if you are doing solving this problem in workspace lot of current work like in imitation learning they are solving this problem in workspace. There is no guarantee that you can map this trajectory into cspace trajectory. Your robot may run into local minima. often time like if you are performing very
49:01
complex task you will see that uh you don't have direct mapping from this workspace trajectory to cpspace like uh if you if you really study motion planning and check out the literature from 19 early 1990s they were doing workspace motion planning then they figured out this is the problem and then they moved to csp space planning unfortunately we are going back to workspace planning so these latest method they are very fast but They rely on these assumptions which eventually
49:32
will lead to problem of mapping to sea space trajectories. >> I, don't, think, neither, are, on, the, exposed space. >> So, geometric, fabric, I'm, unaware, of, that but I think the other paper you mentioned they assume fair spherical representation. No maybe I'm I'm thinking of some other paper then. Yeah I am then not aware of that. >> I, guess, like, maybe, for, like, geometric things very sick. >> Sorry., So, basically, you, model, your system as a dynamical system and then
50:02
you try to essentially extract load vector fields. Yeah. Or now like I guess maybe in that sense it would also be an >> yeah, so, basically, how, does, that, converge class say trying to take out functions. So I I think there's a very strong relevance like uh our over our recent work that we submitted to new reps it's basically on kinodnamic like how can you come up with these travel time fields in in dynamic space I'm unaware of works that you are discussing so I I cannot
50:33
comment comment on their relation but in general we we want to go toward vector field but we want to go toward that without expert demonstration without learning from expert data or just by solving ing those PDS. Okay. I'm I I'm unaware of what the work maybe you can share and I will be happy to get back to you. Yeah. Any other question?
51:02
One problem I have is that um if the ball is close to the constraint uh then the solution can be pretty unstable. Um during solving process uh the system bounce back and forth around to the constraint uh does your model has a similar problem? >> Yes., So, the, way, you, use, simple, data, is very important like uh I think it's in my in our experience like creating a curriculum helps like starting with a simpler problem and then slowly coming up with start and goal that are very far
51:39
helps and then another thing that helps is like having more samples that are near obstacles because those are the areas where you have sharp features and you want your neuronet network to learn those better. So yes, so we we do suffer from those problems and having some adaptive sampling helps. >> I, see. >> But, after, you, finish, your, training successfully, uh that doesn't mean that it's equally cheaper to sample goal near obstacle or far away from obstacle.
52:10
>> If, you, are, following, the, gradients, and because it's an approximation, it can go to local minima. But if you use this neural network as a value function and incorporate within some MPPI then it helps overcome those local minima issue and it it does see yes thank you and there was
52:33
>> do, we, know, what, methods, the, uh commercial
52:39
>> I, think, usually, they, are, like, I, don't know what they are using but usually it would be some classical technique or or or some classical motion planner or something. But I I would assume that would be the case, but I
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat