I did some work on this some time ago and discovered what Rick refers to in that there is no mechanism for concurrent goroutines to join so that you can check/wait for termination of a parallel.
I communicated with the go developers having raised a bug and in fact had an interaction with the main scheduler developer. He really was not interested in what I was saying indicating that this was the way the scheduler was designed and that was the way it was going to be. What happens is that gorotines that do input create lots of instances in preparation for input and do this until you run out of memory. I had a working system until I put a time delay in to slow it down ie
sleep for some short period
It worked for a bit and then went into garbage collect mode and then ran out of memory and the comment was "that is what it is designed to do" I got frustrated and stopped working with go.
I did find a paper from CMU on go scheduling problems which was very informative and included a discussion as to why the go people should follow the example of the Ritson and Barnes design for occam-pi!
I attach the somewhat unhelpful discussion . The reference in the last email to an on-line document can be found a bit up from the bottom and when I have to go to gctrace and dumps then I tend to decide that life is too short!
From: occam-com-request@xxxxxxxxxx [occam-com-request@xxxxxxxxxx] on behalf of Rick Beton [rick.beton@xxxxxxxxx]
Sent: 29 March 2015 17:35
To: Occam Family
Subject: Re: Go - deadlocked processes causing memory leaks
see notes inserted below...
On 29 March 2015 at 17:01, Roger Shepherd <rog@xxxxxxxx> wrote:
That is a reasonable interpretation. Go has quite straightforward memory model, using the stack as much as possible and the garbage-collected heap otherwise. Anything that goes out of scope is reclaimed either by being on the stack (and the stack pointer is moved), or by having no more references to it. This applies to both the channel and the goroutine in the specific example.
The buffered channel fix works because the sender can write without waiting; it then immediately terminates and its memory can be recovered.
This looks to me like a good representation of the relevant bits of the Go. The important difference is overall termination: in the Occam case, the join semantics means that the whole block will deadlock in the timeout case. Go is different because the join semantics are different - the Read function can terminate even with the secondary goroutine waiting in its deadlocked state. It's easy for the developer to overlook this, because there may not appear to be any termination problem.
Go is like Occam in this: only when every process is deadlocked do you get notified by the runtime that there is a major problem. There is no earlier notification that things are not going well.
You mentioned that Go and Occam are both a bit low level. Perhaps the issue is that the original author of the faulty code was attempting to write an I/O function for fetching something off a network, with a timeout option. Here I suspect that there may exist alternative solutions in which this network request would be blocking and the timeout would be handled elsewhere.
However, it's a sunny afternoon and I shall leave that thought dangling for now, without exploring it further.