Hacker News Reader

Comment by redact207 | Hacker News Reader

I've been working on a similar workflow engine for node at https://www.npmjs.com/package/@node-ts/bus-workflow

The main objective of workflows is to manage long running processes. By processes I mean business processes like coordinating the activities of fulfilling a customer order (settling charges, picking inventory, packing, dispatching, email receipts etc). It's a way to keep all those individual commands decoupled but coordinate them at a higher level.

This isn't a new concept by any means, and is often paired with Domain Driven Design and message based systems. Doing so gives you a library of events everytime something happens in your system that can be reacted to in a workflow.

If you've ever dealt with microservices, or even a monolith where two internal services are incorrectly coupled together then this approach may be worth looking into.

View on HN

Thank you. I feel kind of silly about this but I feel like I've had a hard time understanding when an org should, or could use something like this. I have seen them mentioned but every time it's explained it's explained with more abstract language on top of it that confuses me. I keep hearing "it manages business processes" but then it fails to mention if this means like, a human being's process within an org, or something coupled with an application of some sort that has business processes in the application? Does this type of thing replace sort of what Jira does, make a ticket and then pass it off to the next team or whatever? Do you ship it with the app for on-premise deployments of a software product? I have a hard time seeing the big picture with things like this sometimes. Then I hear workflow orchestrator and I think, oh okay so like ansible, but for, work...flows? But what is a workflow really exactly?

My personal opinion is that the workflow is any business logic that goes beyond a single request reply. Examples of workflows:

* Service deployment.

* Uber trip

* Media processing (download file, transcode, upload result)

* Order processing

* Customer support ticket processing

* Customer incentive program management

* Data pipeline processing

* ML Training

* Distributed CRON

* Customer signup flow

and many others.

I can relate, it didn't make sense to me either in the beginning but eventually the penny dropped after working with it in a past company.

Rather than give a half cooked explanation here and risk more confusion, I'll update the readme with some examples and ping you if you're interested.

redact207, could you please tell me if I understood it correct?

Say, I have two services deployed individually working in their own domains.

UserService and EmailService

When the task is a simple user signup & welcome email

1. Workflow requests UserService.signupUser

2. UserService.signupUser creates User, then dispatches UserCreated event.

From my understanding, this is where it's different between having Workflow and not having it.

3. Workflow receives UserCreated event, then requests EmailService.sendEmail

IF I did not have Workflow in my design, then EmailService will be listening to UserCreated event from UserService directly.

It sounds almost same as having an orchestration service called UserSignup Service, and do the same thing what Workflow does.

Can you say my understanding correct? Thanks!

Yep! The one thing I would change though is that it's common for workflows to be started by an event. So in your example, the first step would be UserService.signupUser, that emits a sign-up event, that starts a workflow that sends the email.

Without the workflow/orchestration, we're effectively coupling the EmailService to the UserService, and it's that type of coupling that reduces reusability and isolation.

Thank you very much for your reply! I wonder if the flow can start from Workflow not from UserService.

E.g., 1. Browser requests with UserSignup event (or API gateway receives it, do not call UserService but emits event)

2. Workflow receives the event, then calls UserService.signupUser activity

3. UserService creates a user then return the call back to Workflow

4. Workflow resolves the call(singupUser) then calls EmailService.sendEmail

5. EmailService.sendEmail sends an email then calls back to Workflow

6. Workflow resolves the call(sendEmail) then the flow is completed

The difference is that Every workflows will be defined inside Workflow and Services won't serve requests directly, which I believe this gives a complete view to the flow.

However there must be something that I'm missing here since what I'm describing seems like an anti pattern.

It's certainly acceptable to start workflows explicitly! However it wouldn't be a good fit for the user signup process.

In the above example, the user just wants to signup. They don't care about receiving a welcome email or being subscribed to a mailing list or anything else, they just want to register an account. That's a pretty good use case for just `POST /signup`, that hits the user service and spits out an event that the user has signed up.

Starting a new workflow when that event is published makes sense in this case.

An example when a workflow is started explicitly could be something like doing a fire system test. You could:

1. shoot off a command that starts the FireSystemTestWorkflow

2. the workflow sends a bunch of commands to test sensors and sirens

3. those things publish events that they're functioning correctly

4. the workflow waits for all of these events to come back

5. the workflow publishes a FireSystemTestedSuccessfully event

The nice side of this is that the workflow can respond if a sensor or siren fails and does what's called a "compensating action", ie: compensates for the deviation of the successful path by performing a corrective action like sending a command to start the device or notify a technician.

Wow. Thanks for the extra explanation about when the different approach can benefit.

I love you. Will definitely give bus-workflow a thorough shot.

And a very last question if you could spare a bit more time..

Again the user signup & event flow,

When Workflow calls EmailService.sendEmail (given that the communication is via RPC and EmailService.sendEmail is an async operation that will resolve if the email was sent successfully), should Workflow wait for the sendEmail operation to resolve and complete the flow? Or should EmailService dispatches an event EmailSent so that Workflow can complete the flow?

This is a bit off the topic but I've been sticking with RPC style call rather than Events but still don't know yet what the best practice is.

Many Thanks

Look at the Cadence samples. You write code as a synchronous event handler, but it is interpreted as a workflow.

https://github.com/uber/cadence-java-samples

This could also be used to kill off systems like SharePoint in many businesses and that would be great.

Seriously, its workflow engine has race conditions, randomly fails and has no transaction management. But there are few alternatives. I don't know why there hasn't been any real contender. You would need a full suite to challange it though.

Speaking as someone who has just implemented a complex business multi-step business process workflow in Sharepoint 2016 - I concur.

Reply on news.ycombinator.com