Skip to main content
Loading Explore guides…
Numerical Integration of ODEs · Acedra
Home
Explore
Mathematics
Numerical Integration of ODEs
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
51 segments
Copy
Hide video
0:01
[Music] foreign
0:09
So I'm really really excited about this next sequence of lectures on the numerical integration of ordinary differential equations because this is where we take a lot of what we have learned so far about how to solve systems of ordinary differential equations and we move this into much more real world settings. So we've talked about how to solve linear coupled systems of first order equations "dx/dt = Ax"
0:34
and really these are valid near linearizations near the fixed points or equilibrium of general nonlinear systems but we haven't talked much about how to handle a generic non-linear system of equations "dx/dt = f(x)" where f is a non-linear function We know that linear systems benefit from the fact that linear superposition holds so you can break this into simpler sub-problems
1:01
like you can diagonalize "A" and solve each of those decoupled problems, and then add those up and and get a solution. There aren't generic solutions techniques like that for "dx/dt = f(x)" for a non-linear function f. And in those cases we often resort to numerical integration. So most of the systems we care about like integrating... predicting the future of a weather system or an epidemiological system
1:27
or a fluid flow system or even a very complicated mechanical system... typically we can't ... we can't solve them using these linear techniques and we have to resort to numerical integration. So that's what I'm going to show you in this sequence. I'm going to walk you through kind of in this lecture the overview and then we're going to dive into some topics so I'm going to talk a little bit more about what I mean by this differential
1:52
equation x dot equals f of x here x is typically a vector of the state of my system it's a vector representing the state of my system representing the state
2:09
of my system it's often called a state Vector that's what it's called and f f of x f is a function of X is generically some non-linear function it may be a non-linear function it may be a non-linear function a non-linear function and it might even vary in time so I've written x dot equals f of x where f is kind of a constant it's just a function of X that doesn't change in time but this function could actually be evolving and changing in time so that would be an autonomous or
2:45
a non-autonomous system would be whether or not this does or does not depend on time so we'll talk about both of those actually and one of the examples I like to give of this is let's say that I have the fluid flow like in the Gulf of Mexico something like that so let's say I have you know this is Florida this is the Gulf of Mexico I'm from you know somewhere in Texas over over here
3:10
and essentially what we have is some some fluid velocity field sometimes we call this a vector field so we have some you know Vector field I'm just going to draw kind of arrows and making this up you know maybe there is some area of big circulation right that's uh not always a good thing when that happens
3:33
um okay so I've got some big Vector field for the fluid flow in the Gulf of Mexico and let's say I am lost at sea so I'm like a little you know I'm on a life raft or something and I am at this initial condition uh X at time zero so I not only have this differential equation I also have some initial condition exit time zero equals x naught and what I might be interested in this system is how to so so first off this Vector field would be f of x it literally says at every
4:05
point in Space X what is the vector the fluid velocity field that is changing my position in time so I assume that my particle kind of advects or flows along with this you know fluid velocity field and at every point in space here at every point x this is my function f of x okay is this kind of vector field velocity field here and you'll notice that in this example this two-dimensional fluid flow my function f has two components it has the kind of velocity direction
4:38
in the X component and the velocity component in the y in the y direction so here you know f of a point x would have you know some uh kind of x velocity and some y velocity component okay f of x would have some x velocity and some y velocity component I'm also going to say that this is not just some function of X but it's also a function of time so realistically I actually should
5:12
um you know generically add in this um possibility that my function my right hand side ordinary differential equation can be changing or evolving in time so of course you know this flow velocity is constantly evolving in the in the Gulf of Mexico it's constantly changing in time uh because you know the ocean is churning and wind is blowing the surface and changing things so this is actually changing in time and these pink arrows are actually going to be moving in time themselves
5:40
What I really am interested in is if I have some initial condition 'x_0' how can I predict where that trajectory is going to go in future times? So can I actually that's what I mean by numerical integration of ODEs, of an ordinary differential equation, is can I tell where this trajectory is going to go in the future by stepping It forward in these discrete discrete 'delta t' advances in time. So if I know 'x_0' can I predict what it'll be
6:15
one time step in the future? Can I predict what x at time one will be? can I predict what x at time two and x at time three and so on and so forth... can I predict a trajectory? So that's what we call a trajectory I'm just going to write this down so we say a trajectory
6:34
is a path -- and I'm really going to say it's some path 'x(t)' -- so it's some path 'x(t)' -- so this is x(t) -- but numerically when I do this on a computer when I solve this on a computer what that is actually going to be often is discretized at discrete points in time. So I'm going to approximate this continuous path with these discrete points x_0 x_1, x_2, ... all the way up to x_n
7:13
and sometimes if I drop my under bar here you know these are still vectors of states so please don't be confused if I forget an underbar and so what I mean by a trajectory is typically you know I have some X naught and that's going to be going to some X1 and that's going to be going to some X2 dot dot dot all the way to some X n so that's what I mean by a trajectory is a kind of either continuous or discrete path
7:40
of points in time where x k is equal to X at some kth time instance and oftentimes I'm going to say that these are sampled at a at a constant discrete delta T So TK my kth time instant would actually be K delta T time steps okay you don't have to take constant time steps but usually in our numerical integration schemes we're going to be taking these constant time steps we'll we'll cook up an algorithm that will step the system forward that will approximate this true
8:15
trajectory at these discrete time instances for this differential equation and again I'm I'm going to reiterate the reason we need to resort to numerical integration of these non-linear ordinary differential equations like finding a trajectory evolving in the Gulf of Mexico is because there is a fundamental limit to what we can do with analytical techniques, with linear techniques, with pencil and paper techniques. So in really really big problems, or in really complicated
8:44
problems, these linear techniques are only valid near fixed points, in neighborhoods of fixed points, typically. And it might be really hard to find these fixed points or hard to analyze these for very high-dimensional systems. And so we have to resort to numerical integration for most real world problems of interest. So in this example I might want to predict, if I'm in some life raft in the Gulf of Mexico, someone might want to predict based on my
9:10
last location where am I going to be a day from now, or two days from now, or something like that. You might also predict where was I last seen... you might actually integrate this backward in time and figure out: okay, well this is my last location... you can integrate these trajectories backwards in times, sometimes, and you might do that to figure out where you came from... where a particle or a trajectory came from. Integrating particles in the Gulf of Mexico is
9:41
also useful if there is a big oil spill and you want to know where this blob of oil is going to go five days from now, if you want to do a cleanup. And so that begs the next question and that's something I'm going to talk about in this so so in this series what I'm going to do I'm just going to write some of these things out here so we are going to derive numerical schemes numerical schemes uh to construct these trajectories to approximate these trajectories to approximate
10:15
to approximate uh kind of X at some next time point x k plus one equals some function of x k so if I know my position at time step now I want to be able to predict my position at time Step at the next time step so let's say this is x k and this is X K plus one I want to be able to build some numerical scheme some big F that approximates a one delta T time step forward so then I can build
10:44
this trajectory starting from my initial condition by just running it through my numerical scheme and so some of the numerical schemes we're going to cook up are going to be things like the forward and backward Euler forward Euler and backward Euler I might even derive one of these today in this lecture backward Euler we're going to derive things like the Runge-Kutta schemes these are really really powerful so Runge-Kutta I've heard this pronounced uh Runge-Kutta I call it
11:15
Runge-Kutta and so there's you know second order and fourth order and so I'm going to talk about these Runge-Kutta schemes which are kind of the built-in integrators in Python and Matlab very very good all-purpose numerical integrators quite accurate and then I'm also going to talk about Advanced schemes like variational and symplectic integrators advanced advanced integrators
11:41
these numerical Miracle schemes are called integrators because we are kind of integrating you know this D X DT we're integrating it in time to time Step It Forward so then I'll talk about Advanced integrators for chaotic systems for chaos for let's just say for chaos for chaotic systems because for chaotic systems you need a little bit more specialized Machinery okay so that's one thing we're going to do is we're actually going to derive these different schemes
12:07
kind of starting with the very very simplest then getting to kind of the all-purpose integrators and then getting into some Advanced integrators if you really really need Advanced capabilities for integrating these trajectories for these schemes we're going to talk about specific properties like error properties we're going to look at the error and we're going to look at stability
12:28
stability and specifically I'm mostly going to look at the error and stability properties of these simple Euler and Runge-Kutta schemes but that's going to build our intuition for more generalized integrators that you might come into contact with later in your in your research or in your work so for all of these schemes we need to be concerned with how accurate is this scheme if I assume that I have a good estimate of f how accurate you know what is the error for
12:55
each of these schemes and what is the stability will these actually you know are these going to Faithfully reproduce my my trajectory or is my forward Euler sometimes going to diverge if I use too big of a Time step that definitely happens and I'm going to tell you why that happens okay so these are you know probably going to be four or five or six lectures building on on this kind of overview lecture maybe I'll have a whole series you know that I'll add
13:19
to occasionally because I actually do a lot of work in this numerical integration world
13:24
um good other things that we care about that I want to tell you about is I like this oil spill example because I think it's pretty intuitive to think about... or maybe ... or maybe my lifeboat example. Maybe I get lost at sea and there's some report that I was missing sometime around 3 P.M on Saturday on this beach... some beach here... and so based on that information, there is some uncertainty in my initial condition and my initial time. And so if
13:56
I have a little bit of an uncertainty here, so I don't know exactly what my initial condition is, but I have some little Gaussian of uncertainty or a little uniform uncertainty how does that propagate and grow? How does that uncertainty grow as I go forward in time with these different integration schemes? Or if I integrate my dynamics, even if I integrated this perfectly, how does this uncertainty propagate forward in time? That's
14:21
a super important question in engineering and scientific applications. I want to know how my uncertainty relates to things like my time step to the dynamics themselves, whether they're chaotic or not chaotic. I want to know for what parameter values does this uncertainty grow a lot or a little? Those are very important questions and they're deeply related to this concept of chaos, of sensitive dependence on initial conditions and chaotic systems. You can get
14:50
really different trajectories even if their initial conditions are arbitrarily close together. So we're going to also talk quite a bit about chaotic systems and how to integrate and propagate uncertainty in these systems, which is a really interesting and really rich area. What else do I want to tell you about [Music] um so this function f itself could be the solution of another differential equation or dynamical system so the vector field F itself so you know
15:25
F maybe some non-linear function we call this a vector field it's a we call it sometimes a vector field when we're integrating particles through it because it's literally at every point in in X and every point in state space it establishes a vector telling you how X changes in time at that point and so it's a vector field a field of vectors and this may be this may be the solution
15:51
to a partial differential equation and so actually the sequence of lectures in differential equations and dynamical systems very soon after this I'll tell you about partial differential equations so these are ordinary differential equations because they only have a single independent variable time but soon I'll tell you about partial differential equations for how this function f evolves in time so for example in a fluid flow f the velocity field or the vector
16:20
field of a fluid flow it actually satisfies its own partial differential equation which would be something like you know the partial derivative of f with respect to time equals some other non-linear function of F and maybe it's spatial derivatives so the Navy or Stokes equations for fluid flows can be written kind of like this so the function f itself might be the solution of another partial differential equation and so we're interested in this lecture series on integrating a
16:52
particle through this ordinary differential equation Vector field but that Vector field itself might be the solution of another bigger dynamical system like the fluid flow equations like the Navier-Stokes equations okay so I just think that's kind of interesting there's kind of layers and layers of complexity of where you can where you can zoom into these systems
17:12
okay um so we're going to be talking about these different schemes probably in the next lecture I'm going to derive forward Euler and backward Euler and look at their error and stability properties then I'm going to introduce Runge-Kutta and we're going to look at you know how to code those up what are their error and stability properties how do we actually you know compare those and their properties to these schemes and then I'm going to talk about chaotic systems
17:38
and how to you know propagate uncertainty and integrate and when you need advanced integrators maybe one last thing I'll show you before I kind of go into the next lecture is I'll just give you a flavor for how we're going to be building things like forward Euler and backward Euler
17:52
Point already have a pretty good idea um of how to do this what we're going to do is essentially we're going to take this differential equation x dot equals f of x and for now we're going to drop the function of T and I'm just going to say it's x dot equals f of x and we're going to assume that F doesn't change in time just to make my life easier but all of this applies to time varying Vector fields we're going to do is we're going to take our DX
18:20
DT and again remember that this is approximately equal to this finite difference derivative that I showed you in the last few lectures so um what I can do is I can say that my derivative is you know so this is x dot that's approximately equal let me let me uh yeah so at some time T at some time T the derivative of x of T at some time T is going to be X at time t plus delta T minus X at time T divided by delta T we know that there's some error associated with this we can
19:00
look at the Taylor series of this expression plug it in and figure out how much error there is so this is really only approximately equal to this but we can use this approximation if we have a small enough delta T and we can set that equal to f of x at T okay and so based on this expression I can now solve for x at my next time step in terms of X at my current time step so I get X at t plus delta T is approximately equal to again this is only approximately equal to the right hand
19:32
side because this is an approximation of x dot so X of t plus delta T is approximately equal I multiply both sides by delta T I add X of T and I get X of t plus Delta t f of X of T okay and so this is the simplest possible numerical integrator to take if I have exit time now I can step it forward and get X at 1 delta T in the future so this is my forward Euler scheme this is forward Euler because Euler developed this and looked at these properties for forward Euler
20:15
what I want here which is to build a trajectory if I have exit time 0 I can step that forward and get x one time step in the future at Exit time one if I have exit time one you know at one delta T then I have exit delta T and I have f of x at delta T I can get f of x at 2 delta T and if I have f of x if I have X at 2 delta T I can get X at 3 delta T and 4 delta T and I can step that forward and build this integrator so this is that big f of x k that I was talking about
20:45
over here and so sometimes you're going to see us write this as you know X at time step K plus 1 1 delta T in the future is equal to my current time step exit my current time step plus Delta t f of x at my current time step and the way we would interpret these subscripts is this means I'm at time t k or time K delta T and so if I want to write this out long form I would say this is X at time X at time t K plus delta T that's time K plus 1 is TK plus another delta T
21:28
equals x at time t k plus Delta t f of x at time t k okay so that's how you can kind of go back and forth between these subscript K's and some time uh some absolute actual clock time 't_k' okay the K time that I'm keeping track of and so this is going to be the name of the game for this whole lecture series for forward or backward oil or second order fourth order Runge-Kutta and these Advanced schemes it's super super simple at the
21:58
heart what we're going to do is we're going to approximate DX DT using some some type of finite difference derivative scheme and we're going to to use that we're going to set that equal to f of x and we're going to build a scheme to predict x one time step in the future based on X now okay so that's all coming up I'm going to derive all of these look at their error and stability and we're
22:22
going to start integrating starting with simple systems we're going to integrate like you know a mass on a spring with a damper so we'll go back to our kind of trusty canonical spring Mass damper we'll integrate a non-linear pendulum with a large amplitude so it's not going to we're not just going to use the small angle approximation anymore so a large amplitude pendulum and then we're going to move into chaotic systems like the Lorenz attractor let's see if I can draw
22:48
my my Lorenz attractor we'll get into chaotic systems like the butterfly attractor and like the double pendulum so the single pendulum is deterministic and it gives you a Clockwork of periodic motion but the double pendulum where I have a single pendulum attached to another pendulum these are both chaotic systems so we're going to build these integrators and test them out on all of these interesting systems that's all coming up in the next let's say five six
23:17
lectures I'm looking forward to it I hope you are this is going to be kind of the culmination of what we've talked about in ordinary differential equations we're going to use our intuition for the error analysis we're going to use eigenvalues and eigenvectors to understand stability of these schemes and then we're going to take this into the next level of really applying this to hard real-world fully non-linear systems uh that's all coming up all right thank you [Music]
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat