Airline crashes and statistics

[MooooMoo]

(edit: Maybe this should have been posted in the "puzzles" forum, but I'm putting it here for now. Also, I heard this puzzle before that Air France crash off the coast of Brazil, so try to ignore that event.)

---------------------------

Two friends were walking on the beach when the subject of airplane crashes comes up. One of them says, "There hasn't been a deadly commercial airline accident in a long time. There's the Hudson River plane crash and the September 11th crashes, but no one died in the Hudson crash, and September 11th was deliberately caused."

His friend says "That's bad, it means we're overdue for a major airline disaster".

They pass a fisherman, who overhears their conversation and disagrees. "You're wrong", the fisherman says. "Each day without a crash increases the safety record of the airline industry, so flying becomes safer every day."

Finally, a sailor comes up and says "Both of you are nuts. Airline crashes are like coin flips. They're completely random, and past records are no indicator of future performance".

So, who's right? The person's friend, the fisherman, or the sailor?

[CRGreathouse]

Originally Posted by MooooMoo

So, who's right? The person's friend, the fisherman, or the sailor?

The friend is falling victim to the gambler's fallacy (and is wrong).

The fisherman is wrong in the short term, though if in the long term there are fewer crashes per flight that reflects improving chances.

The sailor is right except insofar as the fisherman is right.

[Uncwilly]

Originally Posted by MooooMoo

Two friends were walking on the beach when the subject of airplane crashes comes up. One of them says, "There hasn't been a deadly commercial airline accident in a long time. There's the Hudson River plane crash and the September 11th crashes, but no one died in the Hudson crash, and September 11th was deliberately caused."

False information.
Note American Airlines Flight 587 on November 12, 2001. And the flight near Buffalo.

[cheesehead]

Here's my reasoning (but I've never taken a proper course in advanced statistics, so ...):

Originally Posted by MooooMoo

They pass a fisherman, who overhears their conversation and disagrees. "You're wrong", the fisherman says. "Each day without a crash increases the safety record of the airline industry, so flying becomes safer every day."

If we had perfect information about the probabilities of the types of failures that lead to crashes, we could calculate the "true" safety record. Since we don't have perfect information, each day (with or without a crash) gives us more information with which to approximate the "true" safety record. On average, each day gets us asymptotically closer to that "true" safety record. In that sense I think the fisherman is approximately (:-) right.

But all that assumed that the "true" safety record of the planes that are flying is constant. Suppose there is a unique "true" safety record for each plane.

When planes with a worse-than-average "true" safety record are retired (or used less), the average "true" safety record over all planes improves, and the asymptote shifts accordingly, and vice versa for retirement (or decreased use) of planes with a better-than-average "true" safety record. When new planes are introduced, the worse-than-average planes worsen the average "true" safety record over all planes, and vice versa for introduction of planes with a better-than-average "true" safety record.

We can discover the "true" safety record only by successive asymptotic approximations each day (or flight), and if planes are retired/introduced the average safety record over all (active) planes is a moving target.

Originally Posted by CRGreathouse

The fisherman is wrong in the short term, though if in the long term there are fewer crashes per flight that reflects improving chances.

I think the fisherman is approximately right in both short and long term.

Originally Posted by MooooMoo

Finally, a sailor comes up and says "Both of you are nuts. Airline crashes are like coin flips. They're completely random, and past records are no indicator of future performance".

The sailor is wrong insofar as claiming that either coin flips or plane crashes are "completely random". Both have biases reflecting the individual characteristics (such as a minute dent caused by dust particle on the coin's stamping die). Also wrong about past records -- they give data about the successively better approximations to "true", so that future predictions can improve in accuracy as more past records are taken into account.

Originally Posted by CRGreathouse

The sailor is right except insofar as the fisherman is right.

... whereas I claim the sailor is just plain wrong (except possibly about nuttiness).

[Ken_g6]

I think they all may be wrong. The friend may be closest to being right, but not for the reasons given.

Originally Posted by cheesehead

Suppose there is a unique "true" safety record for each plane.

But plane safety depends on more than just construction. It depends on maintenance, both upkeep and checking for broken/cracked/worn-out parts. When there's a crash, all similar planes are carefully checked, in particular for whatever seems to have caused the crash, but they may also be more carefully checked in general. After awhile with no crash, the maintenance checks may tend to get lax.

I recall when the Space Shuttle Columbia blew up, there was discussion of "...a lax safety regime that had developed since the 1986 explosion of the shuttle Challenger...." If the NTSB is set up better than NASA, this shouldn't happen with airplanes; but I seem to recall news stories about plane safety after crashes that indicate it tends to happen with planes too.

But I've never taken a proper course in advanced statistics or anything remotely related to airplane safety, either.

[CRGreathouse]

Originally Posted by cheesehead

But all that assumed that the "true" safety record of the planes that are flying is constant. Suppose there is a unique "true" safety record for each plane.

This is how I interpreted the sailor's statement, so you and I appear to be disagreeing mainly on how to formalize the three statements. I disagree insofar as the true record per plane type is not constant (for example because of maintenance, c.f. Ken_g6).

[R.D. Silverman]

Originally Posted by cheesehead

Here's my reasoning (but I've never taken a proper course in advanced statistics, so ...):

I Have.

If we had perfect information about the probabilities of the types of failures that lead to crashes, we could calculate the "true" safety record. Since we don't have perfect information, each day (with or without a crash) gives us more information with which to approximate the "true" safety record. On average, each day gets us asymptotically closer to that "true" safety record. In that sense I think the fisherman is approximately (:-) right.

Perceptive comments. And very nearly correct. The one piece missing
is that technology does improve with time. One can infer that prolonged
periods without crashes also may mean better maintenance and training
over time. This latter inference does need to be backed with actual
data -- which should be available.

But all that assumed that the "true" safety record of the planes that are flying is constant. Suppose there is a unique "true" safety record for each plane.

I doubt that it is constant.

This assumes that the time series representing the per-period probability
of having an accident is stationary/ergodic. This seems (to me) to be
unlikely. Planes do age. Metal does fatigue (remember the Hawaii flight
where the roof ripped off?). Inspections are less than perfect.

When planes with a worse-than-average "true" safety record are retired (or used less), the average "true" safety record over all planes improves, and the asymptote shifts accordingly, ...<snip>

We can discover the "true" safety record only by successive asymptotic approximations each day (or flight), and if planes are retired/introduced the average safety record over all (active) planes is a moving target.

Bingo!

I think the fisherman is approximately right in both short and long term.

The sailor is wrong insofar as claiming that either coin flips or plane crashes are "completely random". nuttiness).

Yes. Except for the assumption that risk is constant over time. (per plane)

[R.D. Silverman]

Originally Posted by R.D. Silverman

Yes. Except for the assumption that risk is constant over time. (per plane)

A follow-on.

Does anyone remember the DC-10 crash in Chicago back in 79/80?
The engine literally fell off right after takeoff.

I was taking an economics class at the time and there was a discussion
about the economic impact of the crash. The professor made a statement
to the effect that people would be reluctant to fly on DC-10's because
the crash "increased the risk of crashes on other flights".

A heated debate followed. I replied that the crash did NOTHING to affect
the risk of crashes on other flights. Indeed. The risk of a similar crash
was solely determined by the hardware integrity of OTHER planes. That
integrity DID NOT CHANGE as a result of the Chicago crash.

What it did do was to alter the public PERCEPTION of risk. Which is a
different thing together from the risk itself. What it did was make the
public aware that flying on a DC-10 was riskier THAN THEY PREVIOUSLY
BELIEVED.

It might also be argued that the risk of other crashes was dramatically
LOWER following the flight because the FAA insisted on immediate systemic
and rigorous inspections of all other DC-10's. (including x-ray inspections
for metal fatigue)

The professor chewed me out afterward for arguing with him.
I told him that if he had a problem we could take it to the dean, but
if he was going to make misleading and erroneous statements to a
class with a minimal knowledge of statistics that I was obligated to point
out the error. [this class was in the business school]

[Fusion_power]

I would love to have been a fly on the wall listening to that chewing out repartee.


The ocean crash a few weeks ago was likely triggered by weather conditions as was the crash in Buffalo. These events could therefore be considered a random variable that affects the overall safety index. This compares to the California crash of the flight from Alaska a few years ago where maintenance of the rudder controls (or lack thereof) clearly caused the crash.

DarJones

[wblipp]

Originally Posted by cheesehead

When planes with a worse-than-average "true" safety record are retired (or used less),

Or crash.

Not usually used for airplanes, but commonly used to get high reliability electronic parts.

[cheesehead]

Originally Posted by wblipp

Or crash.

... "unscheduled catastrophic retirement"