Day 21 in Mat360

Today:

• Announcements:
  • Thanks for your projects: they're found in our project directory. I haven't finished grading them yet -- I've just looked over most of them. I'll continue to update a "greatest hits" (solution) file.

    Bit of a teaser/thought for the day: we have to be careful not to confound "numerical analysis" and "programming".

    You have various "implementations" of bisection. The idea behind bisection is well-defined. One can conduct bisection as a "thought experiment", without any code at all.

    How many iterations of bisection are required to know which root bisection will converge to for $f(x)=(x+1)x(x-1)$ on the interval $[-\sqrt{2}, \pi/2]$?

  • Your next homework is due Tuesday (concerning interpolation). Questions?

• Today:
  • I used the Hermite cubic spline we developed last time in place of the linear spline, which provides us with a $C^1$ interpolant -- we get function and derivative correct: Mathematica Code

    Notice that I've also taken care of the problem we saw last time, through a log identity. I observed that my fake $\ln$ seemed well behaved on $[0,1]$. Can I calculate $\ln$ well on the interval $(0,1]$ to get values for $\ln(x)$ when $x>1$?

    You might notice how very few cubics we need to do a really great job approximating $e^x$, compared to the number of linear functions.

    What error are we making when we use Hermite interpolation? Proposition 5.6 gives us the answer (p. 219):

    $(x-x_1)^2(x-x_2)^2f[x_1,x_1,x_2,x_2,x]$

    which is equal to a derivatives times some stuff:

    $\frac{f^{(4)}(\xi)}{{4!}}(x-x_1)^2(x-x_2)^2$

    where $\xi$ is some unknown number between the minimum and maximum of $\{x_1,x_2,x\}$. It will be maximized in the middle, as we have seen, so a bound on the error for the Hermite spline approximation to $e^x$ is

    Exp[1]/Factorial[4]*(h/2)^4

    While my Hermite spline does the job (approximating $e^x$ for all real numbers), one might do the same job with Taylor polynomials, for example.

  • Today we start Chapter 7 (numerical differentiation and integration).

    You won't be surprised to learn that the Taylor Series is the key to analyzing some of the most obvious choices.

    Our objective is "adaptive quadrature", section 7.5. If you understand how to do that, I'll be happy! It's way cool.

    The good news is that some methods will already be familiar; the error analysis part will probably be new, however.

    • Section 7.1: numerical differentiation
      • We begin with the one-sided forward-difference formula, which comes out of the most important definition in calculus (the limit definition of the derivative) by simply dropping the $h$:
        $f'(a)\approx\frac{f(a+h)-f(a)}{h}$

        The authors define the truncation error as the difference of the approximation and the true value:

        $\frac{f(a+h)-f(a)}{h}-f'(a)$

        and the power of $h$ on this error will be called the order of accuracy.

        The text illustrates this in Figure 7.1, p. 256, as the slope of a secant line approximation to the slope of the tangent line (the derivative at $x=a$).

        We explore the consequences using the Taylor series expansion:

        $f(a+h)=f(a)+hf'(a)+\frac{h^2}{2!}f''(a)+\frac{h^3}{3!}f'''({\xi}(h))$

        What is the order of accuracy of this method?

        We call this a forward definition, because we think of $h$ as a positive thing. It doesn't have to be however; but we define the backwards-difference formula (obviously closely related) as

        $f'(a)\approx\frac{f(a)-f(a-h)}{h}$

        which comes out of

        $f(a-h)=f(a)-hf'(a)+\frac{h^2}{2!}f''(a)-\frac{h^3}{3!}f'''({\xi}(h))$

      • Compare these "lop-sided" schemes with the centered-difference approximation:
        $f'(a)\approx\frac{f(a+h)-f(a-h)}{2h}$

        which we can again attack using Taylor series polynomials. What is the order of accuracy of this method?

        We can also derive it as an average of the two lop-sided schemes! We might then average their error terms to get an error term. However, what do you notice about that error term?

      • This is really a good time to make use of our tools from linear algebra, however. Let's take another crack at deriving these methods, but from the LA perspective, and see how matrices can make our problem so much easier, and allow us to generalize these simple ideas quite simply.

      • Now one thing you might think is that, by making $h$ smaller, you'll get better results. This is not necessarily so, and we should make clear that there are "competing interests" (or errors).

        Let's think about computing with error.

        It will turn out that the truncation error of a scheme like these will look like

        $E = \left| \frac{h}{2}f''(\xi) + \frac{e(x_0+h)-e(x_0)}{h} \right| \le \frac{M|h|}{2} + \frac{2 \epsilon}{h}$

        where $M>0$ is a bound on the derivative of interest on the interval of interest (second in this case), and $\epsilon>0$ is a bound on the size of a truncation or round-off error.

• Notes from Class:
  • NumericalDifferentiation.pdf