RE: Software fault forces Ford recall

[Tony@xxxxxxxxxxxx]

Automotive software

>From the description of the fault on the Channel 4 program (which featured a
fellow mature student of my wife, who was one of the poor unfortunates with
a Ford Explorer that did this) it looks like there are actually a couple of
problems

1) It is possible to get into a part of the state diagram that no-one has
designed - the glitch seems to send it into a part thought impossible
because the cruise control is not enabled - one driver found that turning
the ignition off and on, although hairy at 60mph (to make sure that you
didn't turn too far and lock the steering) was his way of resetting the
system.

2) (and here I am guessing) the system is poorly event triggered. When I
used to write automotive software in the 1980s before I joined Inmos, inputs
were always carefully considered. For example, a single transition on a
switch was never assumed, because this could always be a spike. Instead, you
looked at the timing - even a momentary switch would end up depressed for
something like a minimum of 100msec, so you would poll it say four times at
10msec intervals for a signal before assuming that it was depressed (figures
vary according to the source and the mechanics). For example, you can work
out how fast someone can floor the accelerator - say 200msec. Assume that
someone can always do it faster than this e.g. 100msec to give a sound
margin of error. Now, you can look at the rate of change in the accelerator
position, and if it exceeds 100% in 100msec, then you almost certainly have
a fault condition. Manufacturers will not release fault statistics, but from
the early 1980s, the proportion of faults that are electrical has dropped
from ~70% to ~50%. Most are connector problems (manufacturers will not spend
a few cents more on a decent connector) and made worse with electronics
because most sensors have only a few mA through the connector.

At Motorola, we used to write demo software - engine controllers, ABS, trip
computers, dashboards etc. blow it into EPROM and put it in one of our lab
cars and then drive it round on ordinary roads. I can tell you from personal
experience that having your engine controller fail during an overtaking
manoeuvre with an oncoming car is entertaining (no - it wasn't my software -
the engine vibration caused a sensor to be sheered off). Another one was an
engine controller done by a colleague, who did not tell me that it was
limited to 4200 rpm, but this was dependent on the battery voltage, and if
the rear screen heater was one, it dropped to 3900, and that he had
implemented "foldback" rev limiting (for the non-engineers, this means that
when you hit the limit of x, it automatically reduces to some factor less
than 1x e.g. 0.8x instead of holding at x. I heard from some engineers at a
large car manufacturer how a car on which they were demonstrating active
shock absorbers had a problem with the engine controller going to full revs
while they were driving round a car park at the factory full of very
expensive brand new cars, and how they had totally wrecked at least 4 of
them before they managed to stop.

When I came to Inmos and discovered occam (at that time, all the car
electronics were written in assembler) it was sheer enlightenment - here was
a language that allowed everything to be expressed in an intelligent way and
checked at compile time. Yet the majority of customers wanted to bypass all
this inherent safety, which is why we ended up with things like C compilers.

In a recent review of some automotive software, I found huge efforts going
in to writing tools to try to identify the possible problems in software.
Funnily enough, they have concluded that they need a subset of C, taking out
all the unsafe constructs that people complained that we failed to implement
in occam.

So at the end of the day, safety critical software is being written in a
subset of C without the nice features of channels, but an OS that does
transputer type timeslicing (more slowly) and critical sections for safe
parameter passing. Some of these embedded systems have a minimum of four 32
bit processors communicating with each other - does this sound familiar?

One day soon, there will be a terrible disaster claiming many lives, that
will eventually be put down to unsafe software. Suddenly there will be a
clamour for a safe programming language. This is where I think that the
market opportunity for a successor to occam will exist.

(I have just had another bizarre thought that proves I should be committed -
I do a fair bit of web work with databases and workflow - an occam like
scripting language would be wonderful if it allowed me to easily spread an
application over several machines using channel like constructs - I often
end up passing parameters on a URL so that there is a more CSP like
behaviour to the structure, and it would solve the problem of framesets -
passing parameters simultaneously to several frames (parallel processes) is
a non-trivial problem).

Tony Gore

email  tony@xxxxxxxxxxxx (alternative if problems tonygore@xxxxxxxxxxxxxx)
tel +44-1278-761001  FAX +44-1278-760006  GSM +44-7768-598570
URL: www.aspen.uk.com
Aspen Enterprises Limited
Registered in England and Wales no. 3055963 Reg.Office Aspen House, Burton
Row, Brent Knoll, Somerset TA9 4BW.  UK



-----Original Message-----
From: Denis A Nicole [mailto:dan@xxxxxxxxxxxxxxx]
Sent: Friday, December 15, 2000 5:59 PM
To: Richard Beton
Cc: occam-com mailing list
Subject: Re: Software fault forces Ford recall


On Thu, 14 Dec 2000, Richard Beton wrote:

> ---------------------------------
> Software fault forces Ford recall
> By: Andrew Thomas
> Posted: 13/12/2000 at 15:10 GMT
> 
> Car giant Ford has instigated a recall of 110,000 Explorer and
> Mountaineer sport utility vehicles because of a programming glitch in
> the car's cruise control equipment. 
> 
Would there be a market for a Tesla-coil based EMP generator that could
shut down the cruise control in emergency?  A sort of electronic "life
hammer".

Denis A Nicole                 WWW:   http://www.hpcc.ecs.soton.ac.uk/~dan
High Performance Computing     Email: dan@xxxxxxxxxxxxxxx
Department of Electronics      Phone: +44 23 8059 2703
       & Computer Science      Fax:   +44 23 8059 3903
University of Southampton
SO17  1BJ
United Kingdom