Show HN: Obelisk – a WASM-based deterministic workflow engine (opens in new tab)

(obeli.sk)

103 pointstomasol1y ago27 comments

A lightweight engine for durable execution / deterministic workflows I built with Rust, wasmtime and the WASM Component Model. Its main use is running reliable, long-running workflows that can automatically resume after failures. Looking for feedback on the approach and potential use cases!

27 comments

SvenL1y ago

One issue I had many time with workflow engines was updates. I have a workflow and it has already running instances. 2 scenarios:

Can I update the workflow while it has running instances without interfering the running instances?

Can I update a running instance with a new version of the workflow to patch some flaw? If no, can I replay an updated version of a workflow with the log of an old workflow version?

tomasolOP1y ago

Great questions. If you are fixing a bug in a workflow, which has running executions, there are two scenarios:

Either the fix does not break the determinism, meaning the the execution did not hit the fix yet. In this case the execution can be replayed and continue on the patched WASM component.

Otherwise, the execution replay causes "Non determinism detected" error. In this case you need to handle the situation manually. Since the execution log is in a sqlite file, you can select all execution affected by the bug and perform a custom cleanup. Also you can create a "forked" execution just by copying the execution log + child responses into a new execution, however there is no API for it yet.

> Can I update the workflow while it has running instances without interfering the running instances?

If you mean keep the in-progress executions on the old version of the code, you can do that by introducing a new version in the WIT file and/or change the new function name.

halamadrid1y ago

We are using a workflow engine called Unmeshed - which has what you are asking about. Workflow definitions can be updated without running interfering with running instances and if you choose to you can patch updates on to running workflows. And you can also rerun workflows with the same input from an older execution.

emgeee1y ago

This is a pretty cool idea but I'm trying to think of the advantage of WASM vs other execution engines.

It seems to me one of the main use-cases for WASM is to execute lambdas, which are often short-lived (like 500ms timeout limits). Maybe this could have a place in embedded systems?

tomasolOP1y ago

The biggest motivator for me is that WASM sandbox provides true deterministic execution. Contrary to engines like temporal, using hashmaps is 100% deterministic here. Attempting to spawn a thread is a compile error. It also performs well - the bottleneck is in the write throughput of sqlite. Last but not least - all the interfaces between workflows and activities are type safe, described in a WIT schema.

AlotOfReading1y ago

WASM isn't quite deterministic. An easy example is NaN propagation, which can be nondeterministic in certain circumstances. Obelisk itself seems to allow nondeterminism via the sleep() function. Just create a race condition among a join set. I imagine that might even get easier once the TODO to implement sleep jitter is completed.

It's certainly close enough that calling it deterministic isn't misleading (though I'd stop short of "true determinism"), but there's still sharp edges here with things like hashmaps (e.g. by recompiling: https://dev.to/gnunicorn/hunting-down-a-non-determinism-bug-...).

3 more replies

jcmfernandes1y ago

Somewhat similar to Golem - https://github.com/golemcloud/golem - correct?

So, I like this idea, I really do. At the same time, in the short-term, WASM is relatively messy and, in my opinion, immature (as an ecosystem) for prime time. But with that out of the way (it will eventually come), you'll have to tell people that they can't use any code that relies on threads, so they better know if any of the libraries they use does it. How do you foresee navigating this? Runtime errors suck, especially in this context, as fixing them requires either live patching code or migrating execution logs to new code versions.

1 more reply

Philpax1y ago

I believe https://flawless.dev/ is another implementation with a very similar technology stack. I'd love to know how you compare and what the key differences are!

tomasolOP1y ago

Indeed. I cannot compare the implementations as flawless is not open source. However on a high level they both share the same philosophy.

I believe the biggest difference is that Obelisk relies on the WASM Component Model:

Obelisk aims to avoid vendor lock-in. It is possible to write activities, workflows and webhooks with no obelisk SDK. Activities and webhooks are WASI 0.2 components that can be run on any compatible runtime like wasmtime e.g. for testing. This should also help with the adoption as any runtime will need a ton of integrations.

jusonchan811y ago

I’m not sure there is much risk in vendor lock in. Look at Temporal. If you use the SDK you are probably locked in for life.

euroderf1y ago

Would it be possible to see an example using Go ? Admittedly, docu for Go in the Component Model is pretty uneven.

1 more reply

disintegrator1y ago

Really nice project. What’s the reasoning behind the AGPL licensing. My understanding is that it will hurt adoption unless you’re planning to offer paid licensing options? Either way it’s a really nice project and I’m keen to try it out. I’ve found it tricky to get a WASM/WASI setup where I can at least my http requests (probably my own skill issue).

tomasolOP1y ago

Thanks for the kind words. In an ideal world I would like to offer a cloud version that would be monetized. There are a few examples on how to do HTTP requests, I have a demo repository [1] with GraphQL and regular JSON-over-HTTP activities. I do agree that the ecosystem is not mature yet, but I was able to generate HTTP activities using LLM on a single shot.

1: https://github.com/obeli-sk/demo-stargazers

chaosprint1y ago

I have a similar idea but for audio effect and music production:

https://github.com/wasm-audio/wasm-audio-examples

For workflow, how do you think of async in wit at this moment?

tomasolOP1y ago

The structured concurrency paradigm in workflows is stricter to what I'm reading here [1]. The whole execution model is different from the Task [2], as workflows are transparently unloaded and replayed.

Obelisk has a concept called join sets [3], where child executions are submitted and awaited. In the future I plan on adding cancellation and allow custom cleanup functions.

[1] https://github.com/WebAssembly/component-model/blob/main/des...

[2] https://github.com/WebAssembly/component-model/blob/main/des...

[3] https://obeli.sk/docs/latest/concepts/workflows/join-sets/

j / k navigate · click thread line to collapse

27 comments

SvenL1y ago

One issue I had many time with workflow engines was updates. I have a workflow and it has already running instances. 2 scenarios:

Can I update the workflow while it has running instances without interfering the running instances?

Can I update a running instance with a new version of the workflow to patch some flaw? If no, can I replay an updated version of a workflow with the log of an old workflow version?

tomasolOP1y ago

Great questions. If you are fixing a bug in a workflow, which has running executions, there are two scenarios:

Either the fix does not break the determinism, meaning the the execution did not hit the fix yet. In this case the execution can be replayed and continue on the patched WASM component.

> Can I update the workflow while it has running instances without interfering the running instances?

If you mean keep the in-progress executions on the old version of the code, you can do that by introducing a new version in the WIT file and/or change the new function name.

halamadrid1y ago

emgeee1y ago

This is a pretty cool idea but I'm trying to think of the advantage of WASM vs other execution engines.

It seems to me one of the main use-cases for WASM is to execute lambdas, which are often short-lived (like 500ms timeout limits). Maybe this could have a place in embedded systems?

tomasolOP1y ago

AlotOfReading1y ago

3 more replies

jcmfernandes1y ago

Somewhat similar to Golem - https://github.com/golemcloud/golem - correct?

1 more reply

Philpax1y ago

I believe https://flawless.dev/ is another implementation with a very similar technology stack. I'd love to know how you compare and what the key differences are!

tomasolOP1y ago

Indeed. I cannot compare the implementations as flawless is not open source. However on a high level they both share the same philosophy.

I believe the biggest difference is that Obelisk relies on the WASM Component Model:

jusonchan811y ago

I’m not sure there is much risk in vendor lock in. Look at Temporal. If you use the SDK you are probably locked in for life.

euroderf1y ago

Would it be possible to see an example using Go ? Admittedly, docu for Go in the Component Model is pretty uneven.

1 more reply

disintegrator1y ago

tomasolOP1y ago

1: https://github.com/obeli-sk/demo-stargazers

chaosprint1y ago

I have a similar idea but for audio effect and music production:

https://github.com/wasm-audio/wasm-audio-examples

For workflow, how do you think of async in wit at this moment?

tomasolOP1y ago

Obelisk has a concept called join sets [3], where child executions are submitted and awaited. In the future I plan on adding cancellation and allow custom cleanup functions.

[1] https://github.com/WebAssembly/component-model/blob/main/des...

[2] https://github.com/WebAssembly/component-model/blob/main/des...

[3] https://obeli.sk/docs/latest/concepts/workflows/join-sets/

j / k navigate · click thread line to collapse