Re: Go - deadlocked processes causing memory leaks

[Rick Beton <rick.beton@xxxxxxxxx>]

Hi Roger,

see notes inserted below...

On 29 March 2015 at 17:01, Roger Shepherd <rog@xxxxxxxx> wrote:

I think I understand the explanation in Stack Overflow. Essentially this says that if a one-buffered channel is used, the sending process wonât block even if the receiver is dead, and the process and channel will be garbage collected. Presumably in the implementation, a processâs memory is reclaimedÂonlyÂwhen it terminates, whereas dangling channels are garbage collected.

That is a reasonable interpretation. Go has quite straightforward memory model, using the stack as much as possible and the garbage-collected heap otherwise. Anything that goes out of scope is reclaimed either by being on the stack (and the stack pointer is moved), or by having no more references to it. This applies to both the channel and the goroutine in the specific example.

Â

I deduce it is not an error to have a program which outputs N items into a channel but inputs only N-1. [I know how you get there, how you end up with buffered channels, and then end up with the nasty consequences. I think the solution is change the programming model so the cases that cause problems can be better expressed - occam and Go are too low level).

I think the buffered channel âfixâ only works if the channel eventually becomes ready - if it doesnât, the process cannot complete.

The buffered channel fix works because the sender can write without waiting; it then immediately terminates and its memory can be recovered.

Â

I canât be bothered to go through the whole of the background to this, and my knowledge of go is only superficial. However, I did design and implement the occam implementation of InputOrFail etc. so I think I have some insight into this. However, I donât understand the motivation of this example, i.e. what the programmer is really trying to do. (I suspect that some problems are caused by having these mono-pole processes which fork but do not necessarily join).

In occam we write an input with timeout as:-

ALT

  InputChannel ? message

HandleMessage(message)

  TIME ? AFTER timeout

HandleLackOfMessage()

Of course, this doesnât really match the go example which is a bit like

CHAN c OF Thing :

PAR

  SEQ

ÂTIME ? t

    ÂTIME ? AFTER t + delay

Âc ! someThing

  ALT

    c ? message

 ÂÂHandleMessage(message)

    TIME ? AFTER timeout

 ÂÂHandleLackOfMessage()

which makes clear the deadlock in the case the ALT chooses the timeout path.

This looks to me like a good representation of the relevant bits of the Go. The important difference is overall termination: in the Occam case, the join semantics means that the whole block will deadlock in the timeout case. Go is different because the join semantics are different - the Read function can terminate even with the secondary goroutine waiting in its deadlocked state. It's easy for the developer to overlook this, because there may not appear to be any termination problem.

Â

So, we might take the Go example as an example of problems with process termination rather than a problem with channel.

Quite.

Â

But, as I said I donât really understand the semantics of Go. Is the system supposed to detect and suppress deadlocked processes?

Go is like Occam in this: only when every process is deadlocked do you get notified by the runtime that there is a major problem. There is no earlier notification that things are not going well.

You mentioned that Go and Occam are both a bit low level. Perhaps the issue is that the original author of the faulty code was attempting to write an I/O function for fetching something off a network, with a timeout option. Here I suspect that there may exist alternative solutions in which this network request would be blocking and the timeout would be handled elsewhere.

However, it's a sunny afternoon and I shall leave that thought dangling for now, without exploring it further.

Rick :-)

Â