Paradigms

[Paul Walker <paul@xxxxxxxxxxxxxxxxxx>]

Impressions from reading "The Structure of Scientific Revolutions" by
Thomas S Kuhn, Univ. of Chicago Press, 1962. 

This book is the seminal work which described and popularised the word
"paradigm" and the "paradigm shift". 

As the transputer, occam, and 1355 (link) communities have all been
involved in promoting new paradigms, and we could perhaps say with
incomplete success, I thought it might be useful to see where our
experience matches what Kuhn describes. It matches remarkably well.

In amongst what is inevitably a very subjective summary of the book,
I'll try to show some of these matches.

Paul Walker


A paradigm is a common consensus about the rules and bounds of a
subject, shared by the bulk of the practitioners of that subject. It
defines the sort of problems that can be investigated and the bounds of
the results that are expected. It need not be codified in writing, but
conventional science textbooks tend to do so. A student's education in
science, using these textbooks, is not a broadening of his knowledge but
a distinct narrowing of the context in which he may think or work. 

The rules and bounds make the science very precise, so that it becomes
possible to observe anomalies which are not adequately explained by the
current paradigm. For a long time these are ignored, treated as failures
of the experiment or of the scientist, and plenty more work is done to
provide more confidence in the current paradigm rather than to try to
find an explanation for the anomalies.

Eventually the number or scale of the anomalies becomes so great that
the science gets into a crisis, where the paradigm is clearly inadequate
to explain things, and yet there is no new paradigm to tie it all
together.

Someone, usually either young or from outside the current paradigm,
tries to make sense of the anomalies and fit them into a different
pattern of relationships. Often there are several such people at the
same time, each having different flashes of inspiration which see
different aspects of the relationships. Even after the inspiration, it
is extremely difficult to articulate what the new paradigm should be,
and as a result it can be impossible to say at what point in time a
discovery was actually made. But there becomes a community which
iterates its explanations, initially probably within a clique of like
minded souls, where at least this group begins to have the consensus
needed for a new paradigm.

In adopting the new paradigm, this group has inevitably destroyed some
major content of the current paradigm, and a competition ensues between
the two paradigms. Advocates of both feel that they have solved the
right problems --- and within the context of the paradigm which defines
the set of permissible problems, they are both right. With this context
and scope of permissible problems and solutions, there is inevitably
communication difficulty between the two groups. There are no rules for
resolving the disputes, no legislative authority to appeal to, just the
consensus of the community involved in the work.

The process of conversion is best summarised by Kuhn: (last para. ch.
XII, p158)
  "At the start a new candidate for paradigm may have few supporters, and
  on occasions the supporters' motives may be suspect. Nevertheless, if
  they are competent, they will improve it, explore its possibilities, and
  show what it would be like to belong to a community guided by it. And as
  that goes on, if the paradigm is one destined to win its fight, the
  number and strength of the persuasive arguments in its favor will
  increase. More scientists will then be converted, and the exploration of
  the new paradigm will go on. Gradually, the number of experiments,
  instruments, articles, and books based upon the paradigm will multiply.
  Still more men, convinced of the new view's fruitfulness, will adopt the
  new mode of practising normal science, until at last only a few elderly
  hold-outs remain. And even they, we cannot say, are wrong."

Kuhn quotes Max Planck as not expecting the opponents to see the light,
but to die "and a new generation grows up which is familiar with it".


I'm sure you will recognise aspects of the transputer/occam/link history
in this story. In programming, the sequential constraint, and in
interfaces the bus constraint, narrow the minds of the practitioners
(and their students), but in the process make the subjects manageable. 

Both subjects are clearly in crisis, with huge anomalies that are swept
under the carpet. The explosion of new bus standards and RAM interfaces
is particularly demonstrative of a crisis for the bus. All the new
versions are finding they have to discard one or more of the constraints
set by the bus paradigm, but all are equally nervous of leaving it
behind. They are all aimed at increasing the bus bandwidth, but ignore
the anomaly that the usable bandwidth is dictated by latency, not by raw
bandwidth. The latest attempt to resolve the crisis, 1394, is even
experiencing a crisis itself, with competing "improvements" for the
bandwidth, one of which is compatible but does nothing for latency, the
other of which improves latency but is incompatible and still does not
scale.

In our replacement paradigm, of low-cost packet switched networks, we
find multicast/broadcast difficult, so we don't admit it. In occam, we
find recursion difficult, so we don't admit it. So our solutions, while
they may solve far more problems than the paradigms we are trying to
replace, do not solve all the problems and actually create difficulties
for adherents of the old paradigms.

In any circumstances, we certainly find difficulty in articulating the
new paradigm in terms which our opponents can understand. So we have
formed cliques such as WoTUG and the 1355 Association, so that we have
like-minded people to talk to. At least we've acknowledged the ease with
which we become a clique, and are making valiant attempts to make the
ideas available to the world outside. Even so, our biggest problem
actually remains that the world has never heard of us.

For ourselves, as well as trying to tell the world, we must continue to
improve our paradigm. For 1355, I'm still concerned that we should be
able to talk tens of metres rather than tens of centimetres, and won't
be satisfied with our solution until we can. Where we have eliminated
something that others see as having value, such as broadcast, we must at
least provide a simple and transparent way for them to do it so the
objection is one of optimisation rather than of basic functionality. We
must see the hesitant moves towards our technology, in FibreChannel,
Switched Ethernet, and SCI, as helping our case --- even if they are
actually manifestations of the crisis in the existing paradigm. And
where as with MyriNet there is so much similarity of concept, we must
see them as allies to the paradigm rather than competitors to our
technology.

Perhaps I should here re-quote the entire passage quoted from Kuhn's
book. But we are improving our paradigm, we are improving the
persuasiveness of the arguments in its favour, we are increasing the
number of experiments, products, articles and books, and we are
convincing a growing number of people.

Perhaps the biggest lesson to learn from Kuhn is that our experience is
nothing like unique. Many have been here before. Recognising this, their
successes, and how difficult sometimes was their success, should make it
easier for us. 

The book is sometimes heavy going, but it's only 170 pages, and you'll
get far more from the whole book than from this subjective summary.
Happy reading!

-- 
Paul Walker                      4Links                      phone/fax
paul@xxxxxxxxxxxxxxxxxx          P O Box 816, Two Mile Ash    +44 1908
http://www.walker.demon.co.uk    Milton Keynes MK8 8NS, UK      566253