Day 14 in Mat360

Today:

• Announcements:
  • Your exam problem revisions are due, as is your most recent homework.
  • I'm still refining your project assignment.
    • I'll have it to you by this weekend. And we'll make it due at the end of the week following Spring break (so you'll have three weeks to do it).
    • If there are any groups formed, then I want to know, so that I can assign the rest. Groups?

  • Your next homework assignment is due next Thursday: to derive and demonstrate a method called Muller's method.

• Chapter 3: fixed point iteration
  • Let's wrap up root-finding today. Last time we got started on this agenda:
    • get back to bisection and Newton's method
    • look at two different ways to extend Newton's method
    • examine issues of convergence.
    Today I want to generalize Newton again, in what are called "fixed point methods". Next week we'll turn to interpolation.

  • Bisection:
    • I want to return to exercise #2, p. 80. Last time we had come to a rather unsatisfactory point: we could use bisection to do division with only addition and multiplication, but
      • whereas if $c>1$ then we can use $[0,1]$ as the interval, and away we go;
      • our interval was [0,machine max] in the case where $c<1$. That's rather big (but I suppose it will work...:)!

      The question is this: can't we make that interval a little better? In particular,

      What if we knew the machine representation of the number?

    • The same idea can be used for the case of $c>1$, too, to improve on our starting interval.

  • Newton's method:
    1. relies on differentiability of the function $f$ and a good starting guess $x_0$. The derivative is assumed to exist and be available for calculation -- otherwise one uses the secant method (which we'll discuss later).

    2. We've derived the formula on p. 82. Here's one implementation of Newton's method.

    3. Now that we've got the formula, we should discuss the two examples on p. 82:
       1. Division without division...
       2. Babylonian roots

    4. One needs to think carefully about the form of the function that one uses in Newton's method.
       • For bisection we used $f(x)=cx-1$
       • For Newton's we used $f(x)=\frac{1}{x}-c$
       • What if we tried bisection's $f$ with Newton's method, to do division without division? (Exercise #3, p. 89)

    5. An examination of what can go wrong (figures on p. 85).
       • That Newton's method won't work for

  • Extensions/Alterations of Newton's method
    • "discretization" $\longrightarrow$ secant method
    • Another version of "Newton's Method" can be derived in similar fashion to Newton's method, only using a "tangent quadratic".

  • Order of convergence. I want to motivate that result on p. 86, using Taylor series, from which follows the rather astonishing conclusion of our authors: "Each iteration doubles the number of accurate decimal places...."

• Fixed Point Iteration:
  • Today I just want to explore fixed point iteration, and talk about it graphically. You should be able to reach some conclusions today about when and where fixed point iteration will work.

    We'll start by just checking out some Mathematica examples of fixed-point functions. What do we notice as we look over these examples?

  • Newton's method is an example of fixed point iteration, and a very good example. But there are good contenders among other fixed-point methods. For example, consider roots of $\cos(x)$. We can compare Newton's method to the simple fixed-point function $g(x)=\cos(x)+x$.

  • Eventually we'll talk about order of convergence (p. 86).

    Definition 3.3 (p. 87): If a method converges to root , and if

    

    then the method is convergent of order , with asymptotic error constant .

    We can show that Newton's convergence is quadratic (with Taylor polynomials).

    Question: What would you expect for bisection? You might think that it is linear, but interesting things can happen....

    Let's

    • look at the convergence of a general fixed point method.
    • See again why Newton's method is much better, in general.

    Examples:

    • Newton's method showing off its better fixed-point credentials
    • Consider the fixed-point approach to finding a root of . We could consider these function (verify that they're both fixed-point iteration functions that solve this problem):
      • 
      • 
      • 
      • for non-zero .
      Which will converge fastest? We just look at the derivatives of these fixed point functions at the root (which, as one can see from the quadratic, is x=2).

      The rule is that the general fixed-point method has order of convergence 1 (linear), with asymptotic error constant (where is the root).

    You might think that bisection is linear -- we've talked about how you get the root narrowed down to one additional bit with each iteration of bisection. However we can show that bisection is not necessarily linear.

    Nonetheless, one can say that we narrow down the interval in which a root is found by one binary digit (that is, "linearly" -- the ratio of interval widths is 1/2 --

    with each iteration. This is the message of homework problem p. 80, Exercise #1.

• Links:
  • The IEEE standard for floating point arithmetic
  • Mathematica notebook for bisection (from this source)
  • naiveBisection.nb
  • Mathematica demonstration of the next step using Newton's method
  • In lisp: