Day 02 in Mat360

Today:

• Announcements:
  • There's a new assignment on the assignment page. Please keep an eye on that page. If I change a deadline or an assignment for some reason, I'll send an email.

  • Sorry if you got slammed in the appendix wars. A few of you have rather big sections. How is that going?

    If you're done, send me your file, and I'll add them to norsemathology.org wiki material.

  • Any questions about our first assignment, on section 1.2?

  • Today we're going to push on into sections 1.3 and 1.4: algorithms, and the errors they make!

• Section 1.3: Algorithms

  One might define algorithms as procedures for solving problems. The authors state that "The goal of numerical analysis [is] to construct algorithms for solving classes of problems...and to find general principles that are useful for constructing algorithms." (p. 12)

  So the authors suggest two parts to the process:

  • solving problems, and
  • finding general principles.
  I found one remark particularly interesting: "...numerical software sometimes fails without warning. Normally this might be regarded as a bug, but for numerical software it is a feature -- in order to be reasonably efficient, the software is deliberately designed not to perform correctly in all situations."

  Question: Is this really a good idea? How do you react to that?

  The section concludes with an example algorithm (p. 13), an oldie and a goodie (dating back to Babylonia, circa 2000 BCE): computation of a square root.

  • The problem to solve: find the square root of (positive) $c$.
  • General Principle: recursion/iteration

  Let's go through it once by hand, first: Let's attempt to find $\sqrt{{4}}$ using this algorithm. (Then we'll let Mathematica help us see how well it's working).

  Note: The really great thing is that I won't ask you to do this in base 60 (yet...:).

  Our authors aren't quite as visual as I'd like them to be, so I'm going to offer you what I think of as a helpful graphical aide in this situation:

  

  Question: Why do I present this particular graph? How does it help? What function is being graphed?

  The code is actually contained in one-line of Computer problem #1, p. 15: (once the variable x is initialized, and the value of c is given):

  it := ( x = (x + c/x)/2)

  The authors say that the "process can be repeated again and again; if it converges, then we can narrow down the search as close as we like."

  The exercises on p. 14 move us toward the objective of demonstrating conclusively that the process converges. Let's take a look at #1 and #2; they're not too bad. (You have #3 for a homework, so if you get ahead of us, you can get started!:)

• Section 1.4: Error

  Okay: we know that we're going to make errors. Even the Bible got $\pi$ wrong (${\pi}=3$)! Well, actually you might take a gander at what is actually said, and see how you interpret it: (1 Kings 7:23). There are other sorts of errors we're going to make through the the necessity of approximations, and it won't always be so simple as an apparently botched calculation of characteristics of a circle:

  • Approximating general functions with polynomials.
  • All reals are rational, to a computer.

  Archimedes got $\pi$ right (mean 3.141851106639839) to a tolerance of about .001, by iteratively bounding it with regular n-gons.

  So, some important questions:

  • How do we define errors?
  • How do we measure errors?
  • How do we avoid errors?
  • How do we prepare for errors?
  • How do we minimize errors (since we're guaranteed to get some)?

  Our authors want to make sure that we are all on the same page with respect to errors, so we start with some definitions:

  • Section 1.4.1: Definitions
    1. error $\equiv$ approximate value - true value

       Our authors suggest that it is sometimes useful to have a name for the negative of the error:

    2. remainder $\equiv$ true value - approximate value

       Or, to turn things around,

       true value $\equiv$ approximate value + remainder

       However, "What we want is the error to be relatively small." (p. 15):

    3. relative error $\equiv\frac{error}{true\ value}$

       And our authors then note that, combining these relationships, that we can write

       approximate value = (true value)(1 + relative error)

    4. ulp: unit in the last place.

       The "unit in the last place" is what we think of as the error when we look at an approximation, such as this approximation of e (= 2.718281828459045....):

       $e{\approx}2.71828182$

       obtained by truncation (losing .000000008459045....)

       I might have rounded, instead:

       $e{\approx}2.71828183$

       In either case, I believe in my heart that the error I'm making is less than (or equal to) .00000001 in size (which is a "unit in the last place").

       Notice that, by specifying "in size", I'm ignoring sign. We often hope that our errors will cancel somehow, and that we'll make some positive and some negative to balance. We'll see examples of actually using the sign of errors sensibly to achieve a balance.

       As the authors say, "An error no greater than 0.5 ulp is always possible, but, as stated in Section 1.2, it is not always practical." In particular, we can get it down to a tenth of an ulp by moving to the next place value -- but that costs something.

    5. $d$ decimal places of accuracy $\equiv-\log_{10}|error|$

       This from solving for $d$, where $|error|=10^{-d}$.

    6. $\delta$ digits of accuracy $\equiv-\log_{10}|relative\ error|$

       This from solving for $\delta$, where $|relative\ error|=10^{-\delta}$.

       Let's consider an example to discuss these two: Suppose that the true value is 1.2345, and that the approximate value found is 1.2346. Let's compute the decimals and digits of accuracy.

       Using properties of logs, notice that

       or and that Question: tells you what?

  • Section 1.4.2: Sources of error

    One of the things that the authors do in this section is discuss errors that we won't concern ourselves with: modeling errors, or programming errors, or hardware errors.

    The errors we do care about they break into two types:

    • Data errors:
      • measurement errors, or
      • errors passed to us by previous calculations.
    • Computational errors:
      • Roundoff Errors, or
      • Truncation Errors
    I might add "output errors" to their Figure 1.7, p. 17. We don't provide as many decimals as we've obtained, say, in a table.

    The authors make an amusing argument that just because we get error-riddled input doesn't mean that we can be sloppy. "It is best to treat the input values as exact and not make judgments about their accuracy or the morality of computing accurate answers to inaccurate problems." (my emphasis -- I think that these guys are really funny!:)

    Data errors are propagated through further calculations (error propagation).

    Let $\hat{a}$ be an approximation to datum $a$: then

    propagated data error ${\equiv}f(\hat{a})-f(a)$

    Then

    computational error ${\equiv}{\hat{f}}(\hat{a})-f({\hat{a}})$

    caused by inaccurately computing f. Figure 1.8 is a good one to illustrate these various sources of error, and the discussion:

    

    Let's do an example:

    • suppose we truncate the Taylor series for $e^x$ with only four terms.
    • Plot $f(x)=e^x$ and $f^*(x)=\sum_{n=0}^3\frac{x^n}{n!}$ from $(0,2)$.
    • Suppose we want to compute $e^\sqrt{2}$, but end up computing $e^{1.41}$ instead.
    • Find and illustrate
      1. the error (and relative error) in the datum,
      2. the propogated data error, and
      3. the computational error.

  • Section 1.4.3: Specifying uncertain quantities

    The authors consider a case where a quantity does not come to us as a value, but rather as a set of values. How do we proceed?

    Obviously we could do a computation for each value. However then our audience is going to ask us for an answer, and we're going to have a bunch! They may not be okay with that....

    We could use some statistics to give an answer. We could use the mean. But we want to avoid losing track (and failing to warn the user at the output end) that there was some variance in that input.

    We treat the value as an interval (from lowest to highest data values), although we'd still have the problem of what to report.

    This section provides a reality check on the term "significant digits" -- "there is no consensus on what this means" -- and they provide an example which illustrates the danger of going with the notion that the least significant digit should be in error by at most 5 units in that place. See Figure 1.9:

    

    Can you assign a digit to the hundredths place that will not be in error by any more than five hundredths? The figure illustrates two cases which won't work -- 1.44 and 1.45 -- each producing an interval beyond what the data support (the top line).

    If not, and we must use 1.4, then another user would assume that we only know the 4 in the tenths place to within five units, and you can see what happens to the actual uncertainty -- it expands grossly, to the interval (0.9,1.9).