Day 05 in Mat360

Last time:

Next time:

Today:

Announcements:
- Your first homework set is graded:
  - First of all, show your work -- and don't erase, or otherwise "destroy" (unless you're sure that you're wrong). Let me see what you tried.
  - Also leave space between problems. Some of your solutions are all crammed together. It makes it more difficult for the grader to follow, and never forget the first rule of homework: make it easy on the grader. You want your grader in a good mood!
    A serious attempt is worth most of the points. I hate to see "failure to start".
  - Some specifics:
    1. #1: I should see two things:
      - that the first difference is in the hundredth's place, and that
      - that ${|x-\hat{x}|{<}.0001{=}{10}^{-4}}$
      Some of you didn't make the second clear -- it would be nice to show this (so that I don't have to be responsible for verifying it).
      A few of you gave answers that failed because ${|x-\hat{x}|{=}.0001$
    2. #2: bounds are the edges of the cage: if we've bounded something, we've got it inside of some boundary. So the bounds, in this case, come from the inequality, which Joey rewrote as \[ -(tolerance) \le computed - exact \le tolerance \] or \[ exact - tolerance \le computed \le exact + tolerance \] or \[ computed \in [exact - tolerance, exact + tolerance] \] or \[ computed \in exact \pm tolerance \]
    3. #3: Two ways to think about it:
      1. Maximizing a difference (mostly did this): $|ab|-|bc|$.
      2. Maximizing a product and difference (which seems to cause more trouble, but isolates the quantities: $|b|(|a|-|c|)$
    4. #4: Some of you wrote something which was true, but not sufficiently general -- e.g. $x The most general thing that one can say is that $|x-y|<|x-z|$.
    5. #5: cases ($x \ge 0$ and $x<0$) works fine.
      Lauren gave a counterexample on part c, to show that it's false: $x=-2$.
    6. #6: you were to prove and motivate geometrically an implementation of the max function.
      (a) "To prove" is a high bar. I should read your work and be convinced beyond doubt that this implementation will return ${max\{a,b\}}$ for all real numbers a and b.
      (b) "motivate geometrically" is a lower bar: but hopefully by examining your drawing I will see why it must work.
      (c) Question: Can we say "Without Loss of Generality" (WLOG), and assume that ${b\ge{a}}$ ?
      Trey seems to think so (and you can!). Nice diagram, too.
    7. #7:
      It appears that we have a triangle inequality deficit! Some people just didn't get started. Of course googling it is one way to start. It's usually visualized in terms of vectors, e.g.
      
      but it also works for real numbers:
      
      ${|x+y|\le|x|+|y|}$
      and there's a "reverse triangle inequality:
      \[ ||x|-|y|| \le |x+y| \]
      Put them together and you get this important result:
      
      ${||x|-|y||\le|x+y|\le|x|+|y|}$
      We could note that \[ |x|-|y| \le ||x|-|y|| \le |x+y| \] and set $x=b+a$ and $y=b-a$, and that gives the result.
      Alyssa did a nice proof using the straight triangle inequality.
      As an alternative, you could do case analysis (a=b). Joey did a nice job along these lines. But the cases require assumptions about the signs of a and b as well, and it's a little tedious. (I'm sure that Joey would agree!)
      There were some illegal moves. For example, some folks just dropped absolute values. Or made up false properties, (e.g. "the absolute value of a sum is the sum of the absolute values" -- nope!)
      So how can the triangle inequality to demonstrate the stated result? (Since that was the hint, it seems a good place to start!).
      Michael M. gives a slick proof, as did Lauren.
- I've prepared some Numerical Analysis preliminary materials for you. Have a look!
- Questions on anything? Current homework?
- You have a new homework assignment (due next Tuesday).
Before we begin: some "fun and games"
- I want to invite into our math circle my very close friend, the Great Fraudini!
- Followed by some comments on the Egyptians.
Section 2.1:

"The main idea is that the computer works with a finite subset of the reals known as machine numbers." (p. 33)
- Section 2.1: Positional Number Systems
  I might have started this chapter with Figure 2.3, on page 41:
  
  It gives a picture of machine numbers on a "toy binary computer". Section 2.1 makes a point about the need to consider other bases (other than 10). Among these other bases, the base 2 is probably the most important.
  Question: why does base 2 figure so prominently in computer science?
  
  There's a beautiful example here of how base 3 is used for tagging hogs (poor hogs!):
  
  Questions:
  - I didn't like the first images alone, so I grabbed the second image above off the -- what didn't I like about the first image?
  - What is the identifier for the first pig?
  Questions:
  - What other bases do you use? Why?
  - Would you know how to convert from a one base to another?
  - Suppose, for example, that you want to convert from 10 to 2; how do you do that?
  - Then, how do you convert from 2 to 10?
  Let's try a few. What's
  - ${87_{10}}$ in base 2?
  - ${10010100111_2}$ in base 10?
  - One more, with a twist: ${87_{10}}$ in base 8?
  Our author makes the point that numbers may terminate in one base, yet repeat in another. $5.2_{10}$ in base 2, for example.
  Questions:
  - What are the consequences, therefore, if we store the number $5.2_{10}$ in a machine built on binary (base 2) architecture?
  - Can you think of an example of a number that repeats in base 10, but doesn't in base 3?
  Examples:
  - p. 36, #4
  - p. 36, #7
  - p. 36, #5
- Section 2.2: Floating-Point Numbers
  In this section the authors describe the manner in which numbers are stored in the computer. They focus on "floating-point numbers", which are represented by three parts:
  1. The sign of the number
  2. The position of the "radix" point (aka decimal point in base 10) -- we might say indicates the "order of magnitude" of the number in that base.
  3. The mantissa (the known "digits")
  So a number is given in what looks like scientific notation, e.g. +2.99792458E08 (m/s) Question: Does that number look familiar?
  Definition 2.1: A real number is said to be an n-digit number if it can be expressed as
  ${{\pm}10^e{\times}d_1.d_2\ldots{d_n}}$
  Question: They then ask "What's an n-bit number?" (p. 39) What do you tell them?
  Let's imagine that our machine has base-10 architecture, with ${-9\le{e}\le{9}}$ , and ${n=4}$ . Then we know exactly which numbers may be represented: numbers from
  
  Largest magnitude numbers -9.999x10⁹ +9.999x10⁹
  
  Smallest magnitude numbers -1.000x10^-9 +1.000x10^-9
  
  You might think that we could consider smaller numbers (e.g. +0.001x10^-9), but our numbers are frequently normalized so that the leading digit () is non-zero (except for 0 itself). This gives us exactly the same number of digits at each order of magnitude of 10.
  Failure to include the denormalized numbers (that don't have a leading 1) leads to a gap around zero in Figure 2.3, on page 41:
  
  On the downside, if we allowed non-zero leading digits, then there would be redundant representations for many numbers (e.g. +1.000x10^-9=+0.100x10^-8)
  Questions:
  - See if you can make sense of the markings on the graphic above: ${-2\le{e}\le{1}}$ , and ${n=3}$
  - What happens if we restrict a machine so that it has architecture ${-1\le{e}\le{1}}$ ?
  - What happens if we expand the machine so that it has architecture ${-3\le{e}\le{2}}$ ?
  - What if we increase the number of digits (n-digit numbers)?
  Now, in reality, computations are usually done in base 2, and the IEEE standard for single precision and double precision are
  
  Single Double
  
  Base 2 2
  
  n 24 53
  
  e [-126:127] [-1022:1023]
  
  Question: in each case, how many exponents are there in the exponent range? Of what significance is that number?
  Question: let's see if we can make sense of this system with this particular example:
  
  Our authors describe the difference between precision and accuracy at this point: I think that it's best done graphically:
  
  Examples:
  - p. 42, #2
- Section 2.3: Rounding
  "The purpose of rounding in computation is to turn any real number into a machine number, preferably the nearest one." (p. 43)
  But there are different ways to do it. You're no doubt familiar with rounding (but how do you handle ties -- that is, how do we round 19.5 to an integer?)). The authors suggest several strategies (p. 43):
  1. Rule 1: "round-to-even": if the digits following the nth digit are
    - less than 500000....
      then discard these digits
    - greater than 500000....
      then discard these digits and add 1 to the nth digit
    - exactly equal to 500000....
      then discard these digits and add 1 to the nth digit if it is odd.
    "Round-to-even" because if nth digit is even, do nothing; add 1 if odd, making it even. All nth digits become even.
  2. Rule 2: "round-to-nearest, with round away from zero in case of a tie"
    Same as Rule 1, except when exactly equal to 500000....
    then round UP (away from zero).
    Inferior to Rule 1, as ever-so-slightly biased away from zero.
  3. Rule 3: chopping -- "round-to-zero" -- truncation.
    Whatever comes after nth, just drop it.
    Inferior to Rule 1, as slightly biased toward zero.
  The biases are illustrated nicely in Figure 2.5, p. 45:
  
  The chopping approximations are all under-estimates; the rounding methods give more balanced overs and unders; but the rounding alone has a bias for giving overestimates, which rounding-to-even balances out.
  There is some vocabulary here with which we should be familiar: sometimes rounding results in
  - overflow ( ${\pm{\infty}}$ )
  - underflow (below the smallest possible machine number)
  - NaN (not a number)
  Definitions:
  
  fl (float)
  converts real numbers into machine numbers
  
  A real number is said to be representable if
  it rounds to a machine number (neither overflow nor underflow, nor NaN).
  
  relative representation error:
  ${\frac{fl(x)-x}{x}\equiv{\delta}}$
  
  The authors make the case that
  ${{\delta}\le\frac{1}{2}\beta^{1-n}{\equiv}unit\ rounding\ error}$ (tiny!)
  where
  - $\beta$ is the base, and
  - ${n}$ is the precision (number of decimals in the mantissa)
  Examples:
  - #3, p. 48
  - #7, p. 49
- Section 2.4: Basic Operations
  By "basic operations" the authors mean using standard arithmetic operations on machine numbers to produce machine numbers. There will be errors.
  Let a and b be machine numbers, and let $\circ$ represent any of the standard arithmetic operations. Then
  
  ${a{\hat{\circ}}b{\equiv}{fl({a}\circ{b})}}$
  I.e., to compute one of these binary operations with machine numbers, you do the operation exactly, and then convert it to a machine number with float (fl). We already know what this will cost: we'll have a roundoff error of $\delta$ .
  As we compute more complicated function, however, with one unary or binary operation after another, the errors continue to accumulate (as seen for example, in section 2.4.5, p. 54).
  Examples:
  - #1, p. 54: Assuming four digit floating-point arithmetic with round-to-even rounding, perform the following computations:
    
    (a) 0.6668+0.3334 (b) 1000.-0.05001
    
    (c) 2.000*0.6667 (d) 25.00/16.00
  - #8, p. 55
  - #9, p. 56
- Section 2.5: Numerical Instability
  This section features several interesting examples of functions, some of them tremendously important, which are also extraordinarily sensitive to errors.
  One of the main points of the section is that a "solution" to a problem may be technically correct, analytically correct, and yet poorly designed to produce good results in general.
  An excellent example is the quadratic formula. Many of you have memorized it as
  
  ${\frac{-b\pm\sqrt{b^2-4ac}}{2a}}$
  We can imagine situations, however, for which this calculation may be dangerous. What do you notice?
  Question: an old trick from your past may be used to improve things: can you think of how to change this formula, to make it less sensitive?

Website maintained by Andy Long. Comments appreciated.

N[Table[2^(-n), {n, 1, 53}], 53]
N[Sum[2^(-n), {n, 1, 53}], 53]

BaseForm[3.75, 2]
BaseForm[3.75, 16]

An interval approximation for e: ${e{\in}[2.71828182{,2.71828183}]}$

One that showed 0, 1, or 2 for positions; we don't see a "2" here.

The figure illustrates two cases which won't work -- 1.44 and 1.45 -- each producing an interval beyond what the data support (the top line).

If not, and we must use 1.4, then another user would assume that we only know the 4 in the tenths place to within five units, and you can see what happens to the actual uncertainty -- it expands grossly, to the interval (0.9,1.9).

The speed of light in a vacuum.

When

${|a+b|<<|a|}$ ,

and, in particular, when ${a+b{\approx}{0}}$ , i.e. when

${a{\approx}{-b}}$ .

Symmetrically, subtraction will suffer the problem when a and b are approximately equal.

Clear[x]
BaseForm[5.2, 2]
it := (Print[Floor[Log[2, x]]]; x = x - 2^Floor[Log[2, x]])
x = 5.2
n = 30
While[n != 0,
 it;
 n = n - 1
 ]

Largest magnitude numbers	-9.999x10⁹	+9.999x10⁹
Smallest magnitude numbers	-1.000x10^-9	+1.000x10^-9

(a) 0.6668+0.3334	(b) 1000.-0.05001
(c) 2.000*0.6667	(d) 25.00/16.00

	Single	Double
Base	2	2
n	24	53
e	[-126:127]	[-1022:1023]

Day 05 in Mat360

Section 2.1: