mfateev

The main difference is that Temporal is using general-purpose programming languages to implement workflow code. This gives developers unlimited flexibility and doesn't require learning a new specialized language just to add resiliency to their program. Some other benefits of using programming language instead of XML/JSON based language:

* Strongly typed (for languages like Java or Go)

* Uses standard error handling of the language of choice. For example, Java SDK throws exceptions, and Go one returns errors.

* Use standard tools for development like IDEs, Debuggers, Linters, Unit Testing frameworks, etc.

* Allows using the same language for both activities and workflows.

* Allows programs of practically unlimited complexity through standard programming language techniques like OO, functions, etc.

* Easily supports handling of asynchronous events

* Supports updating definitions of already running workflows

* Allows reuse of standard libraries. For example, if workflow needs to keep a state in a priority queue, it can use an existing one.

My personal opinion that BPMN tries to cater to two distinct groups of people. Non-technical domain experts and software developers. And it is essentially a compromise that cannot serve both of them well. Non-technical domain experts still cannot implement production-ready workflows, and developers have to use limited complicated UI/XML-based technology that is inferior to the programming languages and environments they are used to. The Temporal approach gives developers the best tool for their job and lets them decide how to interact with non-technical users. In some domains, it is possible to create DSL for non-technical users to use. Temporal is an excellent technology for implementing such domain-specific languages. BPMN's problem is that it is not domain specific and tries to serve as a general-purpose language without being one.

The short answer is that Temporal is a fork of Cadence by the original founders of the project. The Temporal fork also has a VC backed company behind it. The Temporal is still an open-source project under MIT License. We believe the success of the open source project is essential for the success of the company, so we are putting significant resources into its development.

We spent almost year working on various improvements before declaring the first production release. The most important technical difference is that Temporal uses gRPC/Protobuf when Cadence is TChannel/Thrift. One implication is that TChannel did not support any security and Temporal supports mTLS. We are currently working on a comprehensive security story.

Here is a stackoverflow post that has more details about this: https://stackoverflow.com/questions/61157400/temporal-workfl....

Why not assuming that you can roll it back to any previous state any time later? Temporal supports such rollbacks.

I wish it were that simple :).

A workflow is not just externally stored data with a well-defined schema. It is the computation's whole state, including threads blocked on API calls and local variables on the stack. So it is not possible to define translations between different versions the way Edit Lenses do.

I see your point. But so far we didn't hear many complaints about this feature from the users. The main problem is in forgetting to version code that changed. But the system has protections against such cases.

It doesn't end up in spaghetti code as old branches are proactively removed after they are not needed anymore. Temporal provides APIs that allow to count number of workflows using each version.

I don't think it is a leaky abstraction as it is indeed a part of the business logic of deciding how old version of the state should become a new one. I don't think there is a generic solution that allows implicitly migrate old state to a new state on any long running computation.

Your question is actually is pretty important as upgrading long running applications is indeed non trivial problem.

The standard approach of versioning the whole workflow is OK for introducing new features but doesn't really support bug fixing. For example, a workflow is expected to run for three months. A bug is found at the end of this workflow definition. The bug is fixed and deployed as a new version, but all workflows that started up to the fix and used previous versions will keep failing for the next three months. So there is a need to patch workflow without changing its version.

The approach Temporal takes is that every part of the code is versioned independently. This allows deploying changes at any time, even for libraries shared by multiple workflows, and doesn't require running multiple worker versions.

Look at how Temporal represents workflows as code. It doesn't use any intermediate representation like DAG or state machine. It executes your code directly as a synchronous program with blocking operations taking as long as necessary to execute. For example the following code would be absolutely valid as a Temporal workflow:

    waitForApproval(purchase); 
    sleep(Duration.ofDays(30));
    sendEmail(email);

Thank you for all the support! Without the incredible community, the project wouldn't be where it is today.

IMHO the best of both worlds is Azure Durable Functions.

I'm original founder of Cadence project and now CEO of Temporal. AMA.

The visual interface for workflow definition is more hassle than benefit. The majority of such applications' complexity is not in the sequencing of operations but in state management, expressions, data manipulation, parameter passing, and retry policies. None of this is visible in the diagram. So engineers have to use UI/DSL to design the flow, but then implement most of the complex logic in non strongly typed code as parameter passing is usually using a map.

Temporal represents all the business logic in one place in the programming language of your choice. All parameter passing is strongly typed.

I see people advocating visual representation for workflows. But if it is so great for programming in general, why they don't advocate the same for systems programming, for example? Linux kernel in JSON/XML anyone?

Temporal is the fork of Cadence by the original creators.It is still open source under MIT license.

I'm former creator and tech lead of Cadence and currently tech lead of Temporal.

Which bucket temporal.io and Azure Durable Functions that support `workflow as code` do fall into?

temporal.io is home of the Temporal Workflow project. It is not about temporal queries, but about writing resilient distributed application. The state is maintained in an event log and supports rollbacks, thus the name.

It is similar to the Amazon Simple Workflow as both founders of the Cadence and Temporal projects worked on the Simple Workflow. I actually was a tech lead of it.

I cannot comment on why AWS made certain decisions as I left Amazon soon after SWF launched.

I guess that SWF was hard to use for novices. When developing Cadence and later Temporal, we fixed the majority of rough edges that SWF had. And it turned out that the core "workflow as code" model was something developers love. And with improved developer experience, the adoption is going very strong.

Here is Hacker News thread that compares the two: https://news.ycombinator.com/item?id=19733880

In my opinion, there are two classes of workflow definition languages: domain-specific (DSL) and general-purpose.

Configuration based languages are awesome for domain-specific use cases. For example, AWS Cloud Formation or HashiCorp Terraform configuration language are good examples of domain-specific workflow definition languages. They solve just one specific problem that allows them to be mostly declarative and omit most procedural complexity. Even in this case, I'm pretty sure that Pulumi folks would not be 100% in agreement.

The general-purpose workflow definition languages are procedural. And I believe that procedural code in YAML/XML/JSON is a bad idea. It looks ugly, doesn't add much value, and never matches any of the general-purpose languages in expressiveness and tooling. Such configuration languages work in limited situations, but developers quickly hit their boundaries in most real use cases and have to look for real solutions.

BTW Temporal and its predecessor Cadence are perfect platforms for supporting custom DSL workflow definitions. Many production use cases run custom DSLs on top of them.

I'm not sure how useful all these counts are. If the static analysis that you described was that important we would write most of the software in YAML instead of C/C++/Java/Go/Python, etc. Linux kernel in YAML would be really cool :).

In my experience, no developer ever asked for this information, especially if the price is writing code in turing complete YAML/XML/JSON based language.

Temporal is the Cadence fork by the founders of the project. You can think about it as a next generation of the project. The biggest difference is gRPC protocol and TLS support.

Here is the stackoverflow answer with more info about the new features: https://stackoverflow.com/questions/61157400/temporal-workfl...

It is a workflow engine. So the actual work happens inside the activities. So from this point of view it is the same. The difference is that the orchestration code has the full power of a programming language including threads and OO techniques. And all the tools like IDEs, debuggers, unit testing frameworks work out of the box and don't need to be reinvented for YAML based reimplementation of Java.

There are lot of advantages of using a general purpose programming language for implementing workflows. An incomplete list in no particular order:

   * Strongly typed for languages that support it
   * No need to learn a new programming language
   * Practically unlimited complexity of the code 
   * IDE support which includes code analysis and refactoring
   * Debuggers
   * Reuse of existing libraries and data structures. For example can YAML based definition support ordered maps or priority lists without any modification?
   * Standard error handling. In Java, for example, exception are used.
   * Easy to implement handling of asynchronous events
   * Standard toolchains just work. For example Gradle for Java and modules for Go.
   * Standard logging and context propagation can be supported

And so on. Any new language has to have a ton of tools, libraries and frameworks to be useful. And using an existing language allows to benefit from the existing ecosystem out of the box.

Look at my project temporal.io. It does the same thing using general purpose programming languages. Currently Java and Go are supported with Python, Ruby and C# under development. This is absolutely valid workflow code:

     public void execute(String customerId) {
       activities.onboardToFreeTrial(customerId);
       try {
         Workflow.sleep(Duration.ofDays(60));
         activities.upgradeFromTrialToPaid(customerId);
         while (true) {
           Workflow.sleep(Duration.ofDays(30));
           activities.chargeMonthlyFee(customerId);
         }
       } catch (CancellationException e) {
         activities.processSubscriptionCancellation(customerId);
       }
    }

Ruby, Python and .NET are being contributed by the community.

Look at temporal.io. Many of the companies use it to replace Airflow.

Disclaimer: I'm tech lead of the project.

Look at temporal.io. It already implements the same idea, but without requiring a new programming language. You can write workflows in Python, Go and Java and soon many other languages.

It doesn't even require await/async style programming. You can make a synchronous call which can take a few days to complete without any problems. And in case of any failures the state of all threads and local variables is automatically restored.

Temporal/Cadence provide unit testing framework that automatically skips time when workflow is blocked. So you can unit test using standard language frameworks (like mockito) to inject all sort of failures. And the tests execute in milliseconds even for very long running workflows.

Temporal (Cadence) comes with Unit testing framework. The most interesting feature of it is ability to skip time automatically when workflow is blocked waiting for something. It allows testing long running workflows in milliseconds without any need to change timeouts.

I agree that Flow being POC had all the above issues. Temporal listened to the users and solved all of them:

1. No code generation. So, you define one interface for activities and use it for both calling activities synchronously from the workflow and to implement them. Here is how would you do it in Temporal:

  @ActivityInterface
  public interface GreetingActivities {
    String composeGreeting(String greeting, String name);
  }

Workflow code:

  GreetingActivities activities = Workflow.newActivityStub(GreetingActivities.class)
  // This is a blocking call that returns only after the activity has completed.
  String greeting = activities.composeGreeting("Hello", name);

The full source at https://github.com/temporalio/java-samples/blob/master/src/m...

2. SWF or Temporal does not drive the deployment strategy. You can run all of the workflows and activities as a monolith in a single process or break them in multiple services. But it is purely your choice.

3. SWF wasn't possible to run locally. Temporal fully supports unit testing of long-running workflows with automatic time skipping and local integration testing using the service running in docker-compose.

BPMN is not declarative. It is imperative. It doesn't say what should be achieved. It gives exact instruction what should be executed and in which order. And a big chunk of those instructions are not really visible in the diagram. For example which parameters each activity takes or shared state updates are hidden.

So it has all the issues that I pointed out above.

The reason is that workflows are inherently imperative.

Configuration formats work well for domains where declarative programming makes sense. For example, Terraform defines "what should be done" instead of how it should be done. Some workflows can be described using declarative syntax. But these belong to some specific domain. In this context, Terraform HCL can be seen as a workflow definition language for infrastructure deployment.

Temporal/Cadence are targeting nondomain specific workflows, so they have to be imperative. And I strongly believe that JSON/YAML/XML/etc. are awful languages for writing imperative programs. They add no value but introduce immense complexity. And they almost always embed some expression language for condition and other evaluations.

For imperative programming, any general-purpose programming language beats heads down any configuration based language in clarity and other features like IDE support, debugging, unit-testing, mocking. We have software systems built form millions of lines of code, and people still make sense out of them. Just imagine Linux kernel written in JSON instead of C.

Note that some workflow engines define workflows as code as well. Airflow is a notable example of this approach. Note that Airflow uses code to define DAG. But the actual execution engine executes the DAG and not code.

Temporal executes the workflow code in a general-purpose programming language directly without any intermediate conversion to AST/DAG or similar.

Then the question is, what makes Temporal workflow code a workflow? Is any code a workflow? Temporal makes workflow state, including local variables and thread stacks fully fault-tolerant. It is like having a computer with a fully durable RAM. Actually, Temporal provides even stronger reliability as a workflow code is not linked to a single computer is automatically migrated between computers or even different clusters when infra failures (or just new code deployments) happen.

I would recommend checking out the documentation (https://docs.temporal.io/docs/overview) and samples (https://github.com/temporalio/java-samples, https://github.com/temporalio/go-samples) to see it for yourself.

In the future WebAssembly will do it out of the box.