Day 20 in Mat360

Today:

• Announcements:
  • Thanks for your projects: I'll hope to have them back by next time.

  • Your next homework is due Tuesday (concerning interpolation). Questions?

• Today:
  • We wrap up an important example: the "linear spline" error term (section 5.5): the linear spline interpolator is $C^0$: we have continuity only.

    Last time we showed that we need around 600 points ($ceiling\left(\sqrt{\frac{8 \times 10^{-6}}{e}}\right)=583$) on the interval $[0,1]$ to compute $e^x$ with an error of at most $10^{-6}$.

    Now, we've got something cool: a really good approximation of $e^x$ for all $x$ on [0,1]. Let's do some more:

    1. Having defined all the interpolators $l_i(x)$ for $x \in \{0,...,n\}$, how can I use the floor function to define the spline $s(x)$ on the interval $[0,1]$?
    2. How can we use $s(x)$ to create an approximation $linearExp(x)$ to $e^x$ for all $x \in \Re$?
    3. Once we have $linearExp(x)$, how can we use it (and the secant method) to create an approximation $linearLn(x)$ to $\ln(x)$ for $x \in \Re^+$?

  • So we've used linear functions, +, -, *, and / to fake $e^x$ and $ln(x)$ extraordinarily well. But our spline isn't $C^1$: it doesn't have derivatives everywhere. How can we do better?

  • An example of cubic splines: how do we intepolate two points and their slopes using a cubic? We can derive the so-called "Hermite spline" as a variant of Lagrange interpolation, featuring "get out of the way" functions!

    Here's another picture of how we might build one of these.

    • One advantage of Hermite polynomials is that the derivative of the spline interpolates the first derivative -- the interpolator is $C^1$.

    • One disadvantage of Hermite polynomials is that we need to know the derivative (! we asked for it!) of the function.

      Question: if we don't have a function, e.g. just data, how might we proceed?

    Other methods are, of course, possible. Some popular splines include "natural" and "clamped" cubic splines. These achieve other objectives.

    I can think of several ways of approaching the fitting of this data with the Hermite interpolant. How might you go about it?

    You might not be surprised to learn that about the easiest formulation is Newton's, only we're going to repeat some abscissa:

    $f[x_1]+(x-x_1)(f[x_1,x_1]+ (x-x_1)(f[x_1,x_1,x_2]+(x-x_2)f[x_1,x_1,x_2,x_2]$

    Since we'll need this over and over, we should go ahead and simplify those divided difference coefficients, using the assumption that $f'$ is known (use the recurrence relation at the top of p. 215 to do so).

    This will have the effect of fitting the two endpoints, and the derivatives of $f$ at the two endpoints.

    I'd like to discuss my implementation of this in Mathematica.

    What error are we making when we use Hermite interpolation? Proposition 5.6 gives us the answer (p. 219):

    $(x-x_1)^2(x-x_2)^2f[x_1,x_1,x_2,x_2,x]$

    which is equal to a derivatives times some stuff:

    $\frac{f^{(4)}(\xi)}{{4!}}(x-x_1)^2(x-x_2)^2$

    where $\xi$ is some unknown number between the minimum and maximum of $\{x_1,x_2,x\}$.

    While my Hermite spline does the job (approximating Sine for all real numbers), one might do the same job with Taylor polynomials.

• Hopefully we'll start Chapter 7 (numerical differentiation and integration) today.

  You won't be surprised to learn that the Taylor Series is the key to analyzing some of the most obvious choices.

  Our objective is "adaptive quadrature", section 7.5. If you understand how to do that, I'll be happy! It's very cool.

  The good news is that some of this will already be familiar; the analysis part will probably be new, however.

  • Section 7.1: numerical differentiation
    • We begin with the one-sided forward-difference formula, which comes out of the most important definition in calculus (the limit definition of the derivative) by simply dropping the $h$:
      $f'(a)\approx\frac{f(a+h)-f(a)}{h}$

      The authors define the truncation error as the difference of the approximation and the true value:

      $\frac{f(a+h)-f(a)}{h}-f'(a)$

      and the power of $h$ on this error will be called the order of accuracy.

      The text illustrates this in Figure 7.1, p. 256, as the slope of a secant line approximation to the slope of the tangent line (the derivative at $x=a$).

      We explore the consequences using the Taylor series expansion:

      $f(a+h)=f(a)+hf'(a)+\frac{h^2}{2!}f''(a)+\frac{h^3}{3!}f'''({\xi}(h))$

      What is the order of accuracy of this method?

      We call this a forward definition, because we think of $h$ as a positive thing. It doesn't have to be however; but we define the backwards-difference formula (obviously closely related) as

      $f'(a)\approx\frac{f(a)-f(a-h)}{h}$

      which comes out of

      $f(a-h)=f(a)-hf'(a)+\frac{h^2}{2!}f''(a)-\frac{h^3}{3!}f'''({\xi}(h))$

    • Compare these "lop-sided" schemes with the centered-difference approximation:
      $f'(a)\approx\frac{f(a+h)-f(a-h)}{2h}$

      which we can again attack using Taylor series polynomials. What is the order of accuracy of this method?

      We can also derive it as an average of the two lop-sided schemes! We might then average their error terms to get an error term. However, what do you notice about that error term?

    • This is really a good time to make use of our tools from linear algebra, however. Let's take another crack at deriving these methods, but from the LA perspective, and see how matrices can make our problem so much easier, and allow us to generalize these simple ideas quite simply.

    • Now one thing you might think is that, by making $h$ smaller, you'll get better results. This is not necessarily so, and we should make clear that there are "competing interests" (or errors).

      Let's think about computing with error.

      It will turn out that the truncation error of a scheme like these will look like

      $E = \left| \frac{h}{2}f''(\xi) + \frac{e(x_0+h)-e(x_0)}{h} \right| \le \frac{M|h|}{2} + \frac{2 \epsilon}{h}$

      where $M>0$ is a bound on the derivative of interest on the interval of interest (second in this case), and $\epsilon>0$ is a bound on the size of a truncation or round-off error.

• Notes from Class:
  • Linear and Hermite Splines