I want you to see and understand the idea behind Runge-Kutta methods, which will be used in place of the Taylor methods. But the Taylor methods provide the impetus for these methods - they're a critical place to start.
The problem with the Taylor methods is that we have to compute all kinds of partial derivatives to make the methods work. Runge-Kutta methods will approximate those partial derivative terms by a succession of approximations computed using only the function f(t,y(t)).
For the Taylor method of order 2, we start with
in which we estimate 
as
This leads to the scheme
for which
But those partial derivatives sure are a pain: any way we can get there more easily?
Yes, but we have to go through some more partial derivative pain: we consider Taylor's Theorem extended to functions of two variables:
Theorem 5.13
Suppose that f(t,y) has continuous partial derivatives of order less than
or equal to n+1 on 
, and let
. For every 
, there exists 
between 
and t and there exists 
between 
and y with
where
and
Now we're ready for the trick: this is the use of the Taylor series that you
might not have thought of, out there on your desert island, even if you'd sat
there for a few years. We want to replace 
in (1) with a value of
f computed at an intermediate location, rather than on the endpoint, as Euler
did. That is, consider 
, expanded using the Taylor
series:
while
The expansion certainly resembles , and if we make a good choice of
and 
, we can make them equal (up to the term 
, which we're
going to drop as our error):
Given this choice, is 
, just like Taylor 2; we call this method
Runge-Kutta-2, (aka RK-2, or the Midpoint method):
This sort of trick can be repeated for higher order Taylor methods, although it's usually only continued up to RK-4:
You might recognize the sum 
in the formula for
as the composite trapezoidal scheme for approximating the integral
This scheme is 
, just like Taylor-4, only we didn't need to compute a
single partial derivative. It's wonderful! This method, therefore, will extend
easily to systems of initial value problems.
We can revisit our pendulum example, for example, to see if we see any improvement in the solution. Here it is in lisp:
Compare RK-4 to Euler on the Pendulum problem: Here it is in lisp: