Day 30 in Mat360

Today:

• Announcements:
  • Final Exam: 10:10, Thursday, May 4.
    • Your final will be in two parts: old stuff and new stuff.
    • You may have one 8.5x11 sheet of paper with anything you'd like on it.
    • Bring a scientific calculator.

  • Your final homework, on DEs, is due tomorrow. Just get it to me some time before 1:00 pm, because I'm leaving for Washington, D.C. for the weekend and will be grading on a bus...:(

  • Any last minute questions on project 2?

    Hopefully you received my email about my own version of the global adaptive quadrature scheme.

    Let's talk about how it's gone, and any discoveries we've made about the two different approaches, Mathematica, tips for tricky functions (without specifics, please), etc.

    Your projects are due by Midnight tonight! And remember to polish them up a bit....

• Some final comments on ODEs and the two contrasting methods: Taylor versus Runge-Kutta
  • Taylor Methods
    • Use the Taylor series expansion
      • Euler's method in action, \[ U(t_{n+1}) = U(t_{n})+ h U'(t_{n}) + \frac{h^2}{2}U''(\xi_{n}) \] so \[ U(t_{n+1}) \approx U(t_{n})+ h U'(t_{n}) = U(t_{n})+ h f(t_n,U(t_n)) \]
      • Taylor-2:

        \[ U(t_{n+1}) = U(t_{n})+ h U'(t_{n}) + \frac{h^2}{2}U''(t_{n})+ \frac{h^3}{3!}U'''(\xi_{n}) \]

      • Etc.! (I could do this forever!)
      and the ODE $Y'(t)=f(t,Y(t))$ (plus the initial value) to generate a solution to a well-posed problem (or even one that's not well posed, in which case it may be a "solution").

  • Runge-Kutta Methods

    eliminate the derivatives, by using clever choices of intermediate observation, which we can think of as being obtained by simpler methods (e.g. trusting Euler only half way...). We can use the multivariate Taylor series expansion to both obtain these methods, and verify that they're good. So last time we saw that Runge's midpoint method is Taylor-2, up to the quadratic term (at which point their error terms are going to differ, but be of similar order in $h$).

    • Now let's expand Runge's trapezoidal in order to show that it's of the same order as Taylor-2.
    • Then let's compare these estimates on Example 8.11, p. 347:

      Euler's
      Taylor-2
      Runge Midpoint
      Runga Trapezoidal	1.583
      RK-4

    • The answers
    • In the end, many folks' go-to ODE solver is RK-4, with step-size control. I've used RK-4 in "Andy's Fun Project", the creation of $Exp$ with only +, -, *, and / (well, I also use the Floor and Ceiling functions -- but we know how those work at the machine level).

      By the way, evidently Exp is implemented in Mathematica via its Taylor series expansion. So I'm using a cannon to kill a fly -- which sounds pretty merciless....:)

• Review of the course: You could think about this course as opening up and explaining features of your calculator.
  • The primary fact of life is that calculators are almost always wrong; but they're so close that we don't care. "All models are wrong; some models are useful."

  • We're going to make errors: there are those we can control, and those we can't. We should do everything to minimize those we can control, and to remain on the alert for those we can't control.

  • What does your calculator do? It
    • performs the basic arithmetic operations,
    • graphs functions,
    • computes derivatives,
    • computes integrals,
    • solves equations (e.g. finds roots), and
    • may even solve ODEs.

    These are among the things we learned to do in this course.

  • An approximation without an estimate of the error made is a poor thing.

  • We started with representation of numbers. Numbers in the computer are approximations, not exact. Very few (relatively) of the reals are in there (and in this picture we can see what kind of error we're making in representing any real number):
    

    (This is my favorite figure of the text, if you haven't guessed!)

    One problem is that computers speak base 2, and we speak base 10; and only rarely do these both end up being machine numbers. So one or both of us is in error.

    Rounding or chopping occurs, and we need to be careful in how we do that. Rounding-to-even is a way of avoiding bias in our calculations.

  • Algorithms for even some of the most common operations you've always known may not be ideal ("ill-conditioned", or inefficient).
    • Quadratic formula
    • Evaluation of a polynomial should be carried out via Horner's method.
    We need to avoid "catastrophic cancellation", as well as needless operations -- generally the fewer operations, the better.

  • Representation of function is similar: sine is not in your calculator; a Taylor series representation of sine is. This is so important because we replace complicated functions with the operations of addition, subtraction, multiplication, and division -- those operations which we can program computers to do.

    We've seen that, even if our division button isn't working, we can use Newton's method or bisection to program that!

    It turned out that Taylor series were useful tools in many places across the semester. One thing that Taylor's theorem does is allow us to bound errors.

    Errors also compete: one of my favorite parts of the course is discovering that the step-size for integration should be made small, but only so small: if you make it too small, the round-off errors from all the additions you do start to add up and overwhelm the advantage achieved by reducing the truncation error.

    There's an optimal value, given the form of and the machine you're using.

    Sometimes errors can even cooperate! This is the lesson of Simpson's rule: wherever you have two estimates, you have a third; and if you're clever, you can add them so that their errors cancel:

    

  • We can use polynomials to capture the behavior of a series of points (splines), a function (Taylor), or even for design (e.g. Bezier curves, which we didn't get to unfortunately...:().

  • The most important polynomial is linear: linear functions are incredibly important. We base a lot of methods off of the tangent line, for example:
    • Newton's method for root-finding
    • Euler's method for solving ODEs.
    Each of these methods can be improved by using higher-order Taylor approximations: e.g. rather than a tangent line, use a "tangent quadratic", which gets the slope and the curvature right at the point.

  • Methods involving the computation of derivatives may be improved (or faked) by replacing analytic derivatives by numerical versions. We perform a rather precise balancing act, similar to what happens in Simpson's rule. And so

    Newton's method is replaced by secant (and then Muller's), or

    Higher-order Taylor methods for ODEs are replaced by RK methods.

  • Each of these is an iterative scheme: we compute

    

    and hope that we have improved.

    Newton's method is an example of one of these "fixed point" schemes, where you know you're done when

    

  • Some schemes are recursive, e.g. adaptive quadrature. Our adaptive quadrature code was also useful for intelligent choice of points for plotting a function (the points being joined by a spline to make a nice smooth curve).

    We have to listen to our functions! (Did you ever think that you'd be listening to your functions?) Sometimes they'll tell you lies, however; and sometimes you'll leap to false conclusions. "Beware the Jabberwock, my son (and daughter)!"

• I hope that as you go forth mathematically, you'll join me in pursuing the goals of my Learning Objectives for this course.

  Best of luck to you all!

• Here's my lisp version of the plotting routine, then turned into an adaptive quadrature scheme:
• Here's a similar lisp version of a local adaptive quadrature routine, which is then turned into an adaptive plotting scheme:

  It's not quite what you're asked to do in this algorithm, however, since you're to keep track of the errors, and keep poking at the largest:

• Here's the global adaptive quadrature routine, which is then turned into an adaptive plotting scheme:
• Comparison of methods on