Re: Interesting Take on Modeling -- easier to read

[Jim Devine <jdevine03@xxxxxxxxx>]

February 20, 2007
Books on Science
The Problems in Modeling Nature, With Its Unruly Natural Tendencies
By CORNELIA DEAN

When coastal engineers decide whether to dredge sand and pump it onto
an eroded beach, they use mathematical models to predict how much sand
they will need, when and where they must apply it, the rate it will
move and how long the project will survive in the face of coastal
storms and erosion.

Orrin H. Pilkey, a coastal geologist and emeritus professor at Duke,
recommends another approach: just dredge up a lot of sand and dump it
on the beach willy-nilly. This "kamikaze engineering" might not last
very long, he says, but projects built according to models do not
usually last very long either, and at least his approach would not
lull anyone into false mathematical certitude.

Now Dr. Pilkey and his daughter Linda Pilkey-Jarvis, a geologist in
the Washington State Department of Geology, have expanded this view
into an overall attack on the use of computer programs to model
nature. Nature is too complex, they say, and depends on too many
processes that are poorly understood or little monitored — whether the
process is the feedback effects of cloud cover on global warming or
the movement of grains of sand on a beach.

Their book, "Useless Arithmetic: Why Environmental Scientists Can't
Predict the Future," originated in a seminar Dr. Pilkey organized at
Duke to look into the performance of mathematical models used in
coastal geology. Among other things, participants concluded that beach
modelers applied too many fixed values to phenomena that actually
change quite a lot. For example, "assumed average wave height," a
variable crucial for many models, assumes that all waves hit the beach
in the same way, that they are all the same height and that their
patterns will not change over time. But, the authors say, that's not
the way things work.

Also, modelers' formulas may include coefficients (the authors call
them "fudge factors") to ensure that they come out right. And the
modelers may not check to see whether projects performed as predicted.

Eventually, the seminar participants widened the project, concluding
that erroneous assumptions, fudge factors and the reluctance to check
predictions against unruly natural outcomes produce models with, as
the authors put it, "no demonstrable basis in nature." Among other
problems, they cite much-modeled but nevertheless collapsed North
Atlantic fishing stocks, poisonous pools unexpectedly produced by open
pit mining, and invasive plants and animals that routinely outflank
their modelers.

Two issues, the authors say, illustrate other problems with modeling.
One is climate change, in which, they say, experts' justifiable
caution about model uncertainties can encourage them to ignore
accumulating evidence from the real world. The other is the movement
of nuclear waste through an underground storage site at Yucca Mountain
in Nevada, not because it has failed — it has yet to be built — but
because they say it is unreasonable to expect accurate predictions of
what will happen far into the future — in this extreme case, tens or
even hundreds of thousands of years from now.

Along the way, Dr. Pilkey and Ms. Pilkey-Jarvis describe and explain a
host of modeling terms, including quantitative and qualitative models
(models that seek to answer precise questions with more or less
precise numbers, as against models that seek to discern environmental
trends).

They also discuss concepts like model sensitivity — the analysis of
parameters included in a model to see which ones, if changed, are most
likely to change model results.

But, the authors say it is important to remember that model
sensitivity assesses the parameter's importance in the model, not
necessarily in nature. If a model itself is "a poor representation of
reality," they write, "determining the sensitivity of an individual
parameter in the model is a meaningless pursuit."

Given the problems with models, should we abandon them altogether?
Perhaps, the authors say. Their favored alternative seems to be
adaptive management, in which policymakers may start with a model of
how a given ecosystem works, but make constant observations in the
field, altering their policies as conditions change. But that approach
has drawbacks, among them requirements for assiduous monitoring,
flexible planning and a willingness to change courses in midstream.
For practical and political reasons, all are hard to achieve.

Besides, they acknowledge, people seem to have such a powerful desire
to defend policies with formulas (or "fig leaves," as the authors call
them), that managers keep applying them, long after their utility has
been called into question.

So the authors offer some suggestions for using models better. We
could, for example, pay more attention to nature, monitoring our
streams, beaches, forests or fields to accumulate information on how
living things and their environments interact. That kind of data is
crucial for models. Modeling should be transparent. That is, any
interested person should be able to see and understand how the model
works — what factors it weighs heaviest, what coefficients it
includes, what phenomena it leaves out, and so on. Also, modelers
should say explicitly what assumptions they make.

And instead of demanding to know exactly how high seas will rise or
how many fish will be left in them or what the average global
temperature will be in 20 years, they argue, we should seek to discern
simply whether seas are rising, fish stocks are falling and average
temperatures are increasing. And we should couple these models with
observations from the field. Models should be regarded as producing
"ballpark figures," they write, not accurate impact forecasts.

"If we wish to stay within the bounds of reality we must look to a
more qualitative future," the authors write, "a future where there
will be no certain answers to many of the important questions we have
about the future of human interactions with the earth."

--
Jim Devine / "The truth is more important than the facts." -- Frank Lloyd Wright