Bob Hastings came by the office one day, after we'd discussed the convergence of the secant method, and asked me how we know that, when n is large,
as our book claimed. We sat down, and worked out the details, but he wanted to see it done more rigorously. So I wrote the following. I hope it helps!
Newton's method can be represented as
where m is the slope of the tangent line at the point 
. If we don't
want to calculate derivatives, we can always approximate m as
a discrete approximation to the derivative. This is the secant method,
which requires two approximations to start ( 
and 
), and doesn't
converge quadratically (although it does converge super-linearly, with exponent
of the golden ratio 
). Here's how:
First, we need to show that
We'll use Taylor series expansions, and order arguments.
Starting from (1), we add and subtract a p to get
Having popped out the 
term, it comes down to showing that
when n is large. Taylor says that
since f(p)=0, we can reexpress A as
having cancelled a common term of in numerator and denominator.
Now Taylor chimes in again, with
so that
So we replace m by its value in A to obtain the following:
(notice that I don't need but one power of 
in the denominator
- don't carry more than you need!).
Now we expand both f' and f'' in Taylor series about p, and throw all the ``order stuff'' into one term while keeping all the good stuff in the first term, to get
The first term on the right-hand side above simplifies to
as one can check, whereas the second term goes to zero quadratically in terms
like 
, , or . Hence,
as 
(and 
). Thus
as our book claimed, when n is large we have that
where
Now, assuming that
then
Thus
and, by continuity,
But this is the same convergence criterion, meaning that secant converges of
order 
, and also of order 
: hence
or
with solutions
and since 
,
Furthermore,
so
and
or
QED.
