Statistics are Meaningless – if
YOU are the
one getting Zapped!
Probably the most misleading information promulgated by the news media
today is statistics. First – statistics are used to analyze GROUPS of
events, actions, happenings etc. The smaller the group the less meaningful any
statistic is. They NEVER apply to the individual or single unit. You have to
really be careful looking at what someone is saying; look at what is really
being examined. If, for example, say the behavior of large crowds is being
modeled with a particular statistic, then the statistic CANNOT be applied to
any INDIVIDUAL crowd – even if it is a crowd of a million people.
Now – how does this relate to risks. Risks are often expressed in terms
of statistical analyses concerning some behavior. For instance, as pilots we
are frequently told that engine failure is very UNLIKELY. This
is a hint. Unlikely is a statistical term. It is a perfectly valid approach to
modeling behavior of aircraft engines of a group – say all piston engines or
all radial engines or whatever. It has NO bearing on whether or not YOUR
engine is going to fail TODAY. YOU can affect this – do you maintain it
well, do you operate it in a kinder and gentler fashion? We know these things
have an effect on engine longevity by statistical analysis BUT there is not
even a theoretical way of extrapolating those statistics to say whether or not
your engine is about to roll over and put its pistons in the air. This is not
a que sera sera situation, its just that statistics aren’t applicable to
ensembles of one.
True story – several years ago a good friend was flying a Bonanza in IMC
when the engine STOPPED. There was a bang and the prop froze (hint – if this
happens attempting a restart is largely a waste of time). Turned out later to
be a crankshaft failure due to bad maintenance several owners back. So there
he was – 4000 AGL in the soup with a DOA engine. He happened to be over
rural Louisiana - which is pretty much one big emergency field – he kept his
head, ATC more or less kept theirs, he popped out at about a grand and made an
uneventful landing on a dike between two fields, first real dead stick landing
in his life. So – how does this stack up STATISTICALLY? It doesn’t – it
is an individual event. He individually affected his outcome by 1)
keeping the airplane in good shape so the instruments and radios were working
fine, 2) keeping his head and realizing that the airplane was still an
airplane and flew just fine (after landing he calmly walked to town and called
everyone. Me? I’d be looking for a Walmart for fresh underwear!!)
So – statistics can help us look at what is important; they can tell us
relative probabilities of outcomes and give us tools to manage risk. BUT –
the actual things WE do – e.g. maintaining engines well, are what will
affect our rewards. Now – suppose we could really reduce risks – perhaps
eliminate them. Is this always a good thing?
Risk Management
Before we consider the implications of removing all risks lets talk about a
fairly recent engineering and management concept – risk management. Risk
management evolved over the last 20 years or so driven by the need to control
risks and expenses on fixed price contracts. Prior to this time most
government contracts were some sort of cost plus affair – cost plus
percentage, cost plus incentive, cost plus fixed fee. Basically these said to
the company – build what we want and we will pay you whatever it costs plus
some amount. The government was taking on all the risk. Sweet!! Sure Mr.
Congress-critter – we can build that widget no-one has ever heard of before
– yes sir – you just pay us our costs (wink wink) and a little profit on
the side and you’ll have it on time. Well – this system required two
things – an honest vendor and an customer (government) which understood that
every change they wanted was going to cost extra.
Well – remarkably this worked pretty well. We got some amazing things
this way. You may remember Apollo – or perhaps heard of it? Lots of other
stuff you likely haven’t heard of but which you use the benefits of today.
Who do you think pioneered U/EHF spread spectrum comm gear? That’s what your
2.4 GHz cordless phone at home is.
But there were problems – like people charging off flying their dogs from
NY to LAX to get groomed as a travel expense. So the winds of change blew and
the fixed price contract became the standard.
This was an interesting concept. The government said – We want you to
build this widget, tell us how much you will charge and you better be right
because that’s exactly how much we will pay. Not too tough if the widget in
question happens to be say another two million rounds of 0.50 cal for a Ma
Deuce. However, if you strike out 0.50 cal machine gun ammunition and write in
stealth attack plane you get a whole different problem.
The problem is – you can’t estimate what you don’t know you will need
to do; so the concept of risk analysis and management was born. The idea is to
use statistical methods, lesson learned from similar activities, best
engineering practices, ouija boards and midnight séances to estimate (guess)
what would go wrong during a development and how much it would cost.
Surprisingly – this has actually worked pretty well. Generally speaking
there are three categories of risk – schedule, cost, and technical. What is
meant here really is the impact the risk may have i.e. it may impact schedule,
and/or cost, and/or technical completion. Here’s a hint – most risks have
impacts on all three.
So far so good – you do the same thing when you go out to fly.
Risky Management
But what when we try to eliminate risk. As I have said -there are various
flavors of risk. Some of these are just fine to reduce or eliminate; others
may be more problematic. What are generally considered pure schedule or cost
risk - i.e. a supplier who ships a part late due to say a flood or strike,
that sort of thing - are fine to try to eliminate or mitigate to extinction.
You won’t succeed but its a harmless hobby. My personal experience is it is
much more effective to acknowledge that this sort of thing is going to happen,
plan to aggressively attempt to minimize it and have real, effective plans to
mitigate it when it does. "Run in circles, scream and shout" does
NOT constitute such a plan.
When you start to examine technical risks the situation becomes much more
challenging. Often it is a very fine thing to reduce or eliminate as many as
possible. Pilots do this all time - in fact I think we invented it. That’s
what the preflight and the flight plan are - risk reduction plans. Everyone
knows the airplane will physically fly without them - well probably,
hopefully, maybe - but it is STUPID and increases you risk UNNECESSARILY
without BENEFIT. Think you save time skimping here - I guarantee
that one incident or accident will cost more in all sorts of ways than you
will ever save by short-cutting these processes. Scrimping here is an example
of "penny wise and pound foolish".
It is a different story though with such things as research and development
efforts. How about projects which look for a new process to do an old task,
solve a problem which was previously unsolvable? Most people understand that
pure exploration efforts by definition have risk. That’s why they are called
R&D efforts and generally paid for with at risk money. In these efforts we
are looking for more risk AND reward. Smart companies spread their R&D
budgets among efforts with differently levels of risk and reward. This is the
same thing as having a diversified investment portfolio. It’s efforts in the
middle which are where I am concerned. These are activities which have some
new ground to cover –but are not pure R&D. These are the projects which
historically have lead to tremendous projects with the "Well Damn"
factor. This is when you are doing something you think you completely
understand then something happens which makes you say "Well Damn".
As we analyze, mitigate and reduce all challenges and risks away – we are
eliminating a lot of "Well Damns". And a lot of progress too.
Well, that’s just random neuron firings for the present. Next time – some
ideas really managing the Risk/Reward duality.
