By DAVID L. WHEELER
Manufacturers use mathematical models to design airplanes, weathermen use them to predict rainfall, and policy analysts use them to foresee the future of Social Security. Now mathematicians, worried about the reliability of the models, are calling for a new "science of uncertainty" to quantify the models' worth. "I don't know if I believe the predictions based on a lot of computer modeling," says John Guckenheimer, professor of mathematics at Cornell University and past president of the Society for Industrial and Applied Mathematics. "I don't know how much uncertainty is in the predictions, and the results are not presented in a way that makes it easy to evaluate that." Mathematicians say new mathematical theories and tools are needed. "There really is no theory at this stage of the game about how to measure uncertainty," says Donald J. Lewis, director of the Division of Mathematical Sciences at the National Science Foundation. Mr. Lewis and other mathematicians say that those who work with models need to pay more attention to sources of errors, ways of reducing those errors, and ways of estimating the errors that cannot be eliminated. The mindset of many scientists already appears to be shifting. Next month, the National Academy of Sciences, the Institute of Medicine, and the National Academy of Engineering are sponsoring a conference titled "How Much Can We Rely on Mathematical Modeling?" After holding a workshop on "managing uncertainty" with the National Science Foundation and the Department of Energy last year, the Society for Industrial and Applied Mathematics is about to issue a report on the topic of uncertainty in models that will include discussion of what kinds of research are needed. "Science moves in a curious way," says James Glimm, professor of applied mathematics and statistics at the State University of New York at Stony Brook, who has participated in recent discussions on the accuracy of mathematical models. "This is a problem that is both known and unknown. People have had a piece of it here and a piece of it there, but a lot of the pieces are just sitting around. They have certainly never been identified, collected, and put together." Mathematical models are attempts at simulating reality by building numerical representations of processes, whether they are mechanical stresses cracking an airplane wing, changes in interest rates rippling through a country's economy, or air currents rising up inside a thundercloud. Models of the physical world are often based on three-dimensional grids, with a computer conducting calculations for each point on the grid. The calculations are performed on changing numerical values that represent changing conditions in the physical world. Computers have made such modeling possible, in part because they can do the massive amounts of arithmetic needed to simulate a complex process. Without access to a time machine, modeling is often the only choice for those who are trying to predict the future effects of current decisions. Computer simulations have largely replaced the testing of nuclear weapons and have been the foundation of international treaties on reducing carbon-dioxide emissions. "I think the use of computers to simulate things is almost irresistible," says Mr. Guckenheimer. In the past says Mr. Lewis, mathematicians have emphasized building better algorithms -- formal rules for solving a problem -- and sought greater computing power, rather than focusing on uncertainty in models. Still, Mr. Lewis views models as an essential third branch of science, after experimentation and theory. "In the complex science we're doing now, we need all three," he says. "The problems we're looking at are so big, we can't even necessarily figure out what we need to observe." A model, he says, may give scientists clues about what kinds of data they need and how to refine existing data. Mathematicians concede that some problems are too tough to crack with models, although their judgments about what falls in the too-difficult category may be as instinctual as anyone else's. Mr. Glimm suspects that the situation President Clinton now faces in Yugoslavia, like other wars, may not be predictable with a mathematical model. Mathematicians consider other processes, such as the weather, to be so affected by small changes in some conditions that they are beyond the realm of accurate, long-range prediction. Small changes in the data fed into the model can produce big differences in the results. One scientist coined the term "the butterfly effect" to suggest that tiny alterations in atmospheric conditions -- including the breeze from a butterfly's wings -- could eventually have major effects in the weather. Scientists are trying to understand how the nature of the data they feed into models affects the uncertainty of the results. Take, for instance, a mathematical model of the customer lines at an airline's ticket counter. A mathematician wanting to know how long it will take to serve individual customers needs to have some information, such as the arrival rate of customers and how long it takes to serve those customers. But the length of time that it takes to serve the customers can vary widely, depending on whether an airline employee needs only to check a suitcase, or to rewrite the tickets for a whole family. Just knowing the average time spent with each customer, however, may not yield an accurate answer in a model intended to help an airline figure out how many employees to have on duty. A mathematician may need to use more-complicated measures of the way in which the amount of time spent with a customer varies. Under statistical theory, Mr. Guckenheimer says, getting accurate answers to some questions depends on how the data that are going into a model vary -- their probability distribution. (The oft-cited "bell curve" is one example of a probability distribution.) For other questions, the data's probability distribution doesn't matter. "That distinction is frequently not made in the ways that we do simulations," he says. While laymen might assume that the errors in modeling always accrue from differences between reality and mathematical models, in truth, many of the errors are created by differences between the computations that are possible theoretically and those that actually occur in computers. "One aspect of uncertainty has to do with how well computers compute the things that we think that they do," says Mr. Guckenheimer. In some models, such as those created by physicists, millions of large numbers must be used, and the computer has to round those numbers. Some computations also call for certain variables to be random, to mimic what goes on in the real world. But the random-number generators used by computers often depend on a "seed" to start creating random numbers. If the same seed is used for two random variables, the same sequence of random numbers will be generated. Having two supposedly random variables in a model, represented by the same string of random numbers, can reduce the accuracy with which a model mimics the real world. Gregory McRae, a professor of chemical engineering at the Massachusetts Institute of Technology, used to be asked about uncertainty in models when he went to the Senate or to regulatory agencies to testify about environmental issues. He became determined to find ways to reduce the uncertainty. Now, he says, he is building ways of coping with it into models when he creates them, instead of making uncertainty an afterthought. That includes not just mathematics, he says, but strategies for how scientists go about building models. Mr. McRae works with interdisciplinary teams that include psychologists who help to consider how scientists make estimates. Most people, for instance, tend to overestimate the chances of being eaten by a shark and underestimate the chances of dying of a heart attack. Similarly, the way scientists estimate unknown quantities may be skewed by hidden biases. The psychologists weed out some of those biases. The psychologists also want to create simple ways of expressing uncertainty so that the results of a model can be understood by members of the public who are not mathematicians. Mr. McRae and his colleagues have developed ways to run multiple simulations in a model and to study how the results vary. Then they determine which variables in the model are contributing most to the uncertainty in the results. In studying the atmospheric chemistry that produces ozone, for example, a model would initially appear to have to include several-hundred chemical reactions. "If you ask someone which one is the most important one," he says "it's the one he has measured." By studying the effects of reactions on the uncertainty in the results, researchers were able to see which ones they needed to measure more carefully. After analyzing the uncertainty in the model, the researchers also found that solar radiation appeared to be important in driving the chemical reactions that created ozone. While not a totally surprising result, the finding led to laboratory experiments and to recommendations that regulatory agencies install monitoring devices to check the ultraviolet light that reaches the air over cities. The ultraviolet light is the portion of solar radiation that most affects ozone production. Mr. McRae says scientists have gone from being victims of uncertainty to gaining some control over it. He has graduate students working on new mathematical tools and new applications of those tools. He sees the science of uncertainty as an expanding field. "Uncertainty," he says, "is everywhere."
Section: Research & Publishing Page: A19
Copyright © 1999 by The Chronicle of Higher Education