Day 10 in math modeling

• Your current homework assignment is to fit a linear model (including oscillation) to the more detailed Keeling data (due Friday). Details on the assignment page. This is a warm-up for doing the same for a town in Togo.

  How's that going? Notice that for now we're only looking for annual periods, so we'll use linear combinations of $sin(2\pi t)$ and $cos(2\pi t)$ (don't try to model anything inside the sine and cosine arguments).

  By the way, the $t$ is years -- if you measure time in some other units (e.g. days, then the functions would look like $sin(2\pi t/365)$ and $cos(2\pi t/365)$.

• I'll be assigning groups and towns Friday, after we discuss the fit to the Keeling data in class.

• A couple of comments from the reports, so far:
  1. Understand: remember that the first step in the modeling process is Understand. It was nice to see that two teams began with maps of Togo, to give one an idea of where their town was located, and perhaps some idea of the country's geography, etc.

  2. #1: Plan: you'll need to be ruthless about outliers. Remember that 0 value for Kara? These folks haven't cleaned their data like they should, and we'll have to do it for them -- and for ourselves, if we're to write a paper.

     We'll talk about how we can determine outliers objectively (more matrix stuff...).

  3. Carry out: Some of you over-achievers have already started in on the non-linear modeling. Good! But don't forget that the simplest reasonable model is best.

     Some models you used appeared non-linear, but are linearizable!

  4. Carry out: I'm delighted that we have three different software packages (at least) represented: R, Mathematica, and Minitab.

  5. Evaluate: $R^2$ values will always increase when you move from a linear to a quadratic model, say: adding an additional parameter to an existing model will only improve $R^2$, so an increase in $R^2$ alone cannot serve as a reason for going with a more complicated model.

  6. Evaluate: Residual plots were prominent. That's good. Remember that we're looking for "IID" -- Independent and Identically Distributed. If we see strange patterns in residuals, it probably means that we may need to go back to the modeling step.

     (Tabligbo)

  7. Evaluate: Whatever model you choose, it's important that you be able to interpret the parameters. What do they mean? The Atakpame team was the only one I've noticed (yet) that took a crack at this....

We started looking at some examples, including some of the functions from the "Bestiary", which can be linearized -- meaning that we can use linear regression to estimate their parameters even though the models are non-linear in the parameters.

• Keeling data -- Patrick's exponential model

  Patrick proposed an exponential model for the Keeling data, but it is non-linear in the parameters: \[ y(t)=ae^{bt} \]

  Yet we can still use linear regression to fit it.

  • Construct an exponential model to the Keeling data. How do we compare the exponential and linear model?
  • If we want to compare $R^2$ values, we can do so for the exponential model by using the formula we obtained in class: \[ R^2=1-\frac{SSE}{SS_{Tot}}=1-\frac{\sum_{i=1}^n(y_i-\hat{y}_i)^2}{\sum_{i=1}^n(y_i-\overline{y})^2} \]
  • How do we compare the exponential and quadratic model? It's trickier. But actually, since the exponential model couldn't even beat the linear model, it won't beat the quadratic!
  • Sally observed that the $R^2$ of the linearized model beat the $R^2$ of the linear model -- but it did so in the linearized space -- so we were comparing apples to oranges. The point of doing the $R^2$ calculation using our formula is to do the comparison of the exponential model back in the original data space (at which point we observed that it lost to the linear model).

• Advertising expenditures. This example is from Mooney and Swift's A Course in Mathematical Modeling (p. 169). They suggest two models for the data:
  • Power model: $y(t)=at^b$
  • Exponential model: $y(t)=ae^{bt}$
  • How do they compare?
  • What problem does the power model pose, relative to starting value of time $t$? Does it change if we represent years as 1970 rather than 70?

    How do we interpret the power? (Remember my remarks above about interpreting the parameters of your model!)

    Does the exponential model suffer the same problem? What is the impact of a shift in time scale?

• Sea Anemone reaction time (from Lamm's UMAP model, and in this book)
  • Duration of Response is a function of size (in "conventional units"!). How should it vary, do you think? What do you think from the scatterplot?

  • This model is a move away from an empirical model, and towards a structural model!

• Using Mathematica's LinearModelFit to find linear models that fit Stewart's Keeling data

• An extract from Curve Fitting via the Criterion of Least Squares, by John Alexander. A bit old-fashioned, but it's got all the major ideas, and some nice examples.

  Alexander also illustrates that one cannot simply invert the regression equation $y=a+bx$ to get the regression equation $x=\frac{y-a}{b}$. So it really matters in linear regression which variable is considered "independent" and which is considered "dependent".

• My linear algebra derivation.