Towards Oberon+ Concurrency
oberon-lang.github.io
oberon-lang.github.io
"Show HN: Towards Oberon+ concurrency; request for comments"
in the hope to receive comments specifically on the proposed language extensions.
As for the proposal itself, it seems to lack the recognition that channels + ability to spawn fibers/threads has not shown themselves to be sufficient in practice. Most of the time people don't want unbounded and unknown lifetimes on the executors, but instead want to be able to see that directly in the code. In go, that resulted in wait groups, and in other languages (java, swift, kotlin etc) a thing called structured concurrency is talked about, see https://vorpus.org/blog/notes-on-structured-concurrency-or-g... for the article that started that.
> Most of the time people don't want unbounded and unknown lifetimes on the executors, but instead want to be able to see that directly in the code.
You mean something like join? This can easily be done by adding a channel on which each thread of the interesting group sends when finished. Thanks for the link, I will have a look at it.
EDIT: Ok, after looking through the referenced article I can confirm that it is about join and that there is a solution with channels as described. Unfortunately I don't have an example in Oberon yet, but here is one from my C library: https://github.com/rochus-keller/CspChan/blob/6867c1a28e1443.... What is called a "Nursery" in the article is just a procedure here (testSieve2) which initiates threads in a similar way like Go, passes an eof channel to them an then waits until it receives on this eof channel.
What I meant was, channel types is something you get for free if they are implemented in terms of generics, without having to resort to custom syntax. Example, in Crystal: `channel = Channel(Int32).new`, which would then only accept and produce Int32.
> You mean something like join?
I meant an obligatory join, that no fibers or threads can be created without knowing where in the code it will have ended.
> This can easily be done
The problem is that this is both repetitive and very easy to get wrong. Hence the need for wait groups and/or nurseries.
But this requires extensive casting features which we don't have in Oberon+. Here is a discussion about this: https://github.com/oberon-lang/oberon-lang.github.io/blob/ma...
> I meant an obligatory join
This is how it was solved in Occam, Joyce or SuperPascal. It's just another language philosophy, and also less flexible than the detached go routines. Using Go (or Oberon+ in future) requires another way of thinking and architecture. Requiring wait groups or other features from the sync package seem like an indication of a mismatch between the architecture and the language philosophy. I'm actually very interested in what things really don't match the Go (or Oberon+ so far) language philosophy and cannot be solved satisfactorily in Go.
I've seen claims though that it is best practice to always use wait groups but I can't judge the quality of those claims as I don't actually program go.
> I've seen claims though that it is best practice to always use wait groups
Is this from the original authors? Can you provide a reference, please?
Or perhaps a lot of things were added because it makes the language easier to write and read and less prone to failures, when solving real problems. Pragmatism wins, every time.
Frankly, Go doesn't seem like a language that would add stuff to ease migration or to simplify for people coming from other backgrounds.
> Is this from the original authors?
No, random forum posts by seemingly experienced developers.
More importantly, I haven't (yet) read closely enough to tell if you only have blocking constructs in at the moment, but if so you probably want to be able to block on an (unordered) collection of channels. I don't think it matters much if this operation unblocks and performs a channel action or just unblocks when a channel action becomes possible, but it does matter that one should be able to block on a mix of both reads and writes in the same group.
VAR_x = (left?y -> VAR_y | right!x -> VAR_x)
as Hoare notes on p116: (not wrt mixed select, but the ultimate logic is the same)> To wait for the first of many possible events is the only way of achieving this: purchase of faster computers is useless.
(note that Hoare78 discusses the desirability of output guards in §7.8 but there it isn't clear that they ought to occur mixed with input guards)
FWIW Hoare thinks it makes for a simpler system if it is always possible to rewrite any SEQ as a PAR and vice versa.
But in general any time you might want to block until the first ready channel, it doesn't work if eg a channel becomes SEND'able but your code is currently blocked on the group RECEIVE. (this situation can be kludged with nonblocking calls, but a blocking call is friendlier to whatever else may be running in the system)
> in general any time you might want to block until the first ready channel, it doesn't work if eg a channel becomes SEND'able but your code is currently blocked on the group RECEIVE
Sure, but I think this can easily be avoided by using different go routines, i.e. not routing everything through only one. So I would be interested in examples where this is not possible or not feasible for some reason.
One simple use case is for a dynamic graph of channels, where one wishes to be able to 'cancel' a blocking send, that being done by having a 'cancel' read channel which one can select upon. Then one uses an Alt over both the send and the receive, with the expectation that usually it is the send which completes.
I've used that pattern in recent Go code.
Of course I can add yet another built-in procedure, e.g. SELECT() which accepts both in and out channels; but since Oberon tries to be as simple as possible, I shouldn't do so as long as there are feasible work-arounds.
No, one can not just close the channel, because it is the sending end (which can close the channel) which is blocked, and the desire is to tear down the graph upon which its send is blocked.
Certainly in Go, if one closes the channel upon which the sender is blocked, then it panics due to sending on a closed channel. It doesn't matter if one closes the channel before the send, or closes from another goroutine during the blocked send, the sender will still panic.
So the only way to safely unblock the sender, is for its blocking send to be part of a select containing a read channel. Then when woken up, the send channel can be closed.
Is there an added value for a non-blocking variang (i.e. like default in the Go select statement)? I could do that e.g. with a boolean parameter.
EDIT: so far my SEND would not panic if the channel closes or is already closed, but just return 0; I thought that there could be yet another process instead of a cancel channel and this process could close channels even if another process is blocked on it, but maybe the SELECT solution would be more elegant (not sure though).
EDIT2: or how about a SELECT procedure with just the in channels first (instead of the number), then a boolean (true..non blocking), and then the out channels?
SEND(VAR c: CHANNEL OF T; v: T) and RECEIVE(VAR v: T; VAR c: CHANNEL OF T), i.e. just a single send/receive, not built-in select, and data flows from right to left.
SELECT( { VAR vN: Tn; VAR cN: CHANNEL OF Tn [;] } { VAR cM: CHANNEL OF Tm; vM: Tm [;] } ), i.e. distinguish receive or send by the order of value and channel, instead of an explicit number or a boolean separator. Still a boolean param could be added to allow non-blocking selects.
The underlying problem is that Oberon lacks an unordered select (WHILE ELSIF comes close but has the unfortunate —in this case, but not for many actor scenarios— repetition). Maybe something could be done with a CASE Poll(...) OF ... END?
If life were a Ponyhof, I would ask for something along the lines of:
PROCEDURE sendit(text: STRING): BOOLEAN;
VAR fail: BOOLEAN;
FailChan: CHANNEL OF BOOLEAN;
BEGIN
fail := FALSE;
FailChan := After(TimeOut);
SELECT
SENDABLE(TxChan) THEN SEND(TxChan, text) |
RECEIVABLE(FailChan) THEN RECEIVE(fail, FailChan)
END
RETURN fail END sendit;
but that'd obviously be too large a change.Concerning your proposal, I would rather not introduce a completely new control structure. But one could - like Wirth with the type CASE statement in Oberon-07 - reuse the WITH statement like:
WITH
SEND(TxChan, text) DO
| RECEIVE(FailChan, fail) DO
END
where I would just have to change the Guard a bit like Guard = qualident ':' qualident | SEND ActualParameters | RECEIVE ActualParameters
and even ELSE is available. What do you think?Assuming I'm correctly inferring from the above, the WITH looks better. Presumably one could have multiple SEND and/or RECEIVE function clauses in the WITH statement, so acting as an equivalent to the Go Select?
What would ELSE do? It it acting the same as the default clause in a Go select, i.e. allowing for a non-blocking call?
The one other question with the above is how would one detect a closed channel when the RECEIVE clause hits?
Correct. Here is the specification: https://github.com/oberon-lang/specification/blob/master/The.... You can have an unlimited number of guards.
> What would ELSE do? It it acting the same as the default clause in a Go select, i.e. allowing for a non-blocking call?
Correct, that's the idea.
> how would one detect a closed channel when the RECEIVE clause hits?
EDIT: I added yet another build-in procedure which returns the closed state of a channel. A blocking WITH would continue if no channel was ready to communicate, but at least one was closed.
A potentially useful SELECT function (if runtime-variable communication graphs are of interest, which they may not be?) would be one which took an array of channels and directions (à la poll), or similar data structure, as a parameter.
Edit: I also suspect that the semantics of channel closure are unlikely to be as simple as you are hoping. (but maybe do something like tcp sockets, and force a consumer to drain the channel before orderly closure?)
(between array riders and lack of display access to variables in enclosing scopes, OBERON SA seems to have evolved in a direction converging upon C?)
That would actually be even a better idea than my present SELECT function, but I don't have a good solution yet for the (receive) variables. The best I can think of is just an array for the values, but that needs a lot of extra copies. I would rather prefer a built-in function than a separate module as in Go.
> OBERON SA seems to have evolved in a direction converging upon C
Not sure what you mean.
EDIT: concerning close synchronization see https://news.ycombinator.com/item?id=38777973
So for the rx (mux) side, they all just write in to one shared chan which it then further processed. For the tx (demux) side, it is a bit simpler, but again with just an extra goroutine per tx side.
This was in a scenario where multiple clients were performing run-time registration with a mux/demux API based upon the comms needs of those specific clients.
There is the Go reflect.Select, however it is awkward, and seems to be discouraged, hence I simply used the pattern above.
Rather something like:
RXORCLOSE(c_j, v_j, VAR closed: BOOLEAN) DO Sj
as an additional guard seems necessary, otherwise these two send side sequences can not be distinguished: SEND(ch, 'a'); SEND(ch, 'a'); CLOSE(ch)
SEND(ch, 'a'); CLOSE(ch)
Or am I missing something?(and FWIW, discarding the SELECT() in favour of the WITH scheme would strike me as preferable)
EDIT: or maybe just allow CLOSED guards in WITH, similar to SEND or RECEIVE, like
WITH
SEND(TxChan, text) DO
| RECEIVE(FailChan, fail) DO
| CLOSED(FailChan) DO
END
But I think CLOSE makes everything more complicated without a clear use; I mostly added it because Go has it.EDIT2: yet another idea:
WITH
SEND(TxChan, text) DO
| ok := RECEIVE(FailChan, fail) DO
END
where ok is a BOOLEAN variable and the assignment is optional; looks a bit strange but allows the same closed handling as in Go.I first came across it in the Go version of them. The libtask/libthread and Alef versions do not include it. Its absence is not really an issue, as the same information can simply be explicitly included in how one uses the channel and the data it carries.
For Go, it works well only because of the multiple return values being present in the select statement.
Rochus, have you considered asking Dr Wirth for feedback? I haven't corresponded with him in about a decade but he was very willing to chat about tradeoffs in language features at the time.
I still think a close feature which - in contrast to Go - would just signal all waiting threads and abandon communication could be useful. I implemented this in the C library I use for experiments: https://github.com/rochus-keller/CspChan
> the proposal for it to take a timeout, which could then be detected via execution of ELSE?
It could still be added in future, e.g. as an additional parameter to SEND or RECEIVE, but in a similar way to CLOSE it makes things more complicated, and there is an alternative solution with a separate thread sending after a delay a over a channel which is received in a WITH statement where also the candidate channel waits.
> have you considered asking Dr Wirth for feedback?
I have qualms about bothering him with this in his well-deserved retirement, especially as he has demonstrated with Oberon-07 in which direction he would develop the language (see also https://oberon-lang.github.io/2021/07/16/comparing-oberon+-w...).
func sendit(text string) bool {
select {
case TxChan <- text:
case <-time.After(10 * time.Millisecond):
return false
}
return true
}You end the section on Go with "Go adopted the concurrency concept from Newsqueak and added new features, such as buffered channels.". However Alef itself added those buffered channels. Which were also available in libthread and libtask, those being what I used before Go came along.
One can find copies to the Alef User[1] and Reference[3] guides from the wikipedia page, and Russ Cox's site also has links to PDFs of them [2,4]. The user guide gives the Alef syntax for such buffered channels as:
chan(int)[100] kbd;
Search for the section titled 'Asynchronous Channels' in the guide.As to buffered channels in general, Hoare's 1985 book mentions (7.3.4 Unbuffered communication) that one could equally use buffered comms as the primitive. One can check the online version at http://www.usingcsp.com, so Alef having buffered channels in 1995 doesn't seem like too much of a stretch, especially given the existence of the actor model.
You may also wish to reference this [5] brief paper by Russ Cox, it happens to mention a couple more languages (Pan and Promela). Also see [6].
The Alef (and libthread) approaches are interesting that they contain a mixture of threads and coroutines; in the former as a language facility in the latter as a library for C.
Now in terms of language syntax forms, having used both libtask in C, and Go, I do appreciate the syntax elements. Having typed (and type enforced) channels in Go makes it easier to reason about the code, likewise the ability (in Go) to split a channel in to a send-only and a receive-only version helps. With typed channels, one can also retrieve a degree of flexibility if the type in question is an interface, then one type switches on receipt of a message.
The real gain though is in select syntax, where having to manually manipulate the array of Alt elements in the libtask/libthread implementation was a distraction. Oh - and obviously Go having a GC makes for an easier time, though it is still possible to "leak" channels and/or goroutines if one freely creates/spawns and they're not carefully closed down. Hence my earlier reference to dynamic graphs.
I suspect a language with sufficient compile time generics could convert a form of libtask chanalt() call in to something which looks a lot like a Go select statement, and hence avoid the distractions. I was pondering having a go at doing that for a Zig or D library.
As to your other question, yes I'd suggest non blocking forms are necessary, I've occasionally had to use them. See the libtask/libthread 'nb' variants. They're actually implemented with a special value in the Alt arrays. I'd imagine a generic version of 'select' taking some form of lambda's could be wrap those without being too ugly.
[1] https://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ug
[2] https://swtch.com/~rsc/thread/ug.pdf
[3] https://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ref
[4] https://swtch.com/~rsc/thread/alef.pdf
[5] https://swtch.com/~rsc/thread/
[6] https://swtch.com/~rsc/talks/group02-thread.pdfConcerning select, there is now a proposal in the updated paper which follows the discussion here (see https://news.ycombinator.com/item?id=38772561). For the time being I added both, a SELECT function and an extended WITH statement to the proposal, both with non-blocking versions. It's still to be discussed whether both are needed, or only one, or whatever other good idea is around.
In any case, when writing APIs, particularly for back-end code, I prefer to remind the caller to think about "what's the longest I'm willing to wait here?" and force them to be explicit if they're really willing to wait years for a result. Even/especially for async APIs, I prefer to make all of the timeouts explicit. More than one developer has asked me for help with running out of memory that happened to be from queued async thunks due to absurdly slow response from some other back-end service.
Depending upon the OS-exposed primitives, perhaps the waiting forever and waiting zero ns might be optimized to use different syscalls as an implementation detail.
For libthread, I'd suggest trying Plan 9 from User Space[2,3]. It has been a while since I played with that, but it is included as part of it.
[1] https://swtch.com/libtask/
[2] https://9fans.github.io/plan9port/
[3] https://github.com/9fans/plan9portAlso, every HN submission is implicitly a request for comments, so I took that bit out of the title too. (Submitted title was "Show HN: Towards Oberon+ concurrency; request for comments")