The libraries are also not functionally equivalent. Some use OT, some use CRDTs, some persist state, some are basically websocket wrappers, fairly different perf guarantees in both memory & latency etc. The very different capabilities make it complicated to evaluate all the tools at once.
Obviously I'm partial the Fluid Framework, but not many realtime coauthoring libraries have made it as easy to get started as Replicache. Kudos to them!
A few solutions with notes...
- Fluid Framework - My old work... service announced at Microsoft Build '21 and will be available on Azure
- yJS - CRDTs. Great integration with many open source projects (no service)
- Automerge - CRDTs. Started by Martin Kleppman, used by many at Ink & Switch (no service)
- Replicache - Seen here, founder has done a great job with previous dev tools (service integration)
- Codox.io - Written by Chengzheng Sun, who is super impressive and wrote one of my fav CRDT/OT papers
- Chronofold - CRDTs. Oriented towards versioned text. I'm mostly unfamiliar
- Convergence.io - Looks good, but I haven't dug in
- Liveblocks.io - Seems to focus on live interactions without storing state
- derbyjs - Somewhat defunct. Cool, early effort.
- ShareJS/ShareDB - Somewhat defunct, but the code and thinking is very readable/understandable and there are good OSS integrations
- Firebase - Not the typical model people think of for RTC, but frequently used nonetheless
Built a Google Docs like rich text collaborator for a client on Express/Psql and React. Worked like a charm. The hardest part was dealing with ports on AWS to be honest.
PouchDB+CouchDB work well out of the box with minimal fuss for open pieces you can just plug into this role. PouchDB handles the client's state persist and replication on the client, couchdb is the reliable cloud service you can replicate to.
Meteor, at least their pre-apollo stack had realtime collab type features with their mini-mongo client and oplog tailing.
I know of one big company using it extensively, hundreds of millions of messages a day! :)
Like a Google Docs kind of thing or w?
Maybe it's just me, but it has a SignalR + Orleans sort of vibe to it when I think about the types of problems it solves. I will definitely be digging into this a bit more.
We gave up on the websocket part and implemented basic polling, because they were not supported by App Engine at the time (things might have moved on since then, which is a couple of years ago). Yet, for a note/todo/habit tracking app, it simply doesn't need to be real-time from our experience.
Have a play at https://www.mindpad.io/app/. You can see how it works if you open up the web app in two incognito tabs, or on an iPhone and the web.
It never really took off in the mainstream - I think because it was before many developers really trusted JS on the server, and a "full stack" framework is quite a big commitment for a team to shift to. Also most CRUD apps don't need real time collab.
I remember being amazed when changes were instantly propagated between my phone and laptop browsers with almost zero lag. This was the demo that sold it for me https://www.youtube.com/watch?v=MGbmW9bwJh4
There are a couple of open-source CRDT libraries that provide both clients and servers (yjs [0] and automerge [1] are two big ones for JavaScript I'm aware of).
My basic assumption is that as long as you put all your relevant data into one of these data structures and have the CRDT library hook into a server for storing the data, you're basically done.
This may be a simplistic view of the problem though. For example I've heard people mention that CRDTs can be space inefficient so you may want/have to do periodic compaction.
At the very least, the automerge maintainers seem to be very actively tackling performance problems.
RDBMS A.C.I.D. and transactions are also capable of much of the same.
That being said you can totally implement collab without CRDTs and if you don't particularly need offline it should be easier.
- WebSocket (and Phoenix Channel) is just a communication method. To maintain consistency and resolve conflict, you need something like Replicache.
- CRDTs are more suitable for p2p scenario while Replicache works better for client-server apps.
- Phoenix's Presence is built with CRDT but it's just a single feature, not a general CRDT toolkit.
The thread: https://twitter.com/aboodman/status/1410441402366922762
> People who love "native apps" can complain about Electron all they want—but there's simply no replacement for the real-time collaboration offered by web-based apps like Figma!
As someone who's not exactly thrilled with Electron and its memory usage—is there a reason the two go together? Is there a reason we can't build collaborative apps in Cocoa and GTK? I think these systems are awesome, I just think they'd be even better if they weren't also running full web browsers!
It's just much much easier and cost effective to build a single code base and hit many many targets platforms with it.
Computing history has also shown that publishing efficient lean software doesn't help in the market. At least not over time to market, getting the key features right, and your ongoing costs.
This is reason enough. Already you now have to build the UI twice because there is no GUI framework that actually looks good on all OSs. You see this all the time where apps made on linux but technically work on macos just work terrible or look super ugly on macos.
You also have to remember windows, ios and android. When you build something targeting web browsers you only have to worry about screen sizes rather than OSs.
To answer your question, collaborative apps ideally need to target the widest possible audience. Barring a massive budget, the best way to accomplish this is to also have a singular compile/build target. In most cases, that’s the web platform.
If Sketch had had Figma's collaboration features, we wouldn't have switched. But during the pandemic it was necessary.
"This means that transactions are guaranteed to be applied atomically, in the same order, across all clients. Further, all clients will see an order of transactions that is compatible with causal history. Basically: all clients will end up seeing the same thing, and you're not going to have any weirdly reordered or dropped messages."
[0] https://doc.replicache.dev/design
Note: There's more in their links, but the linked sites are down..
See also the commentary here: https://doc.replicache.dev/guide/local-mutations
This sounds a lot like Operational Transform but without the transform part - it assumes that locally applied mutations can be undone and rebased without user interaction. But I feel like the Google Wave team would have a lot of objections to the idea that this can just be ignored. If your state is just a group of key value stores where last write wins and everyone can agree on who's last, that's fine, but text/token streams require a notion of transformation that I'm worried Replicache simply glosses over.
This route seems like a lot of extra work for pretty similar functionality.
Am I wildly off base here? Is it just that middle tier jump to over 10k that is a no go?
Again, I don't have a horse in this race or even my own startup, just trying to understand if my own judgement is way off.
[Error] Could not connect to the server.
[Error] Fetch API cannot load https://damp-fire-554.fly.dev/replicache-pull?list_id=kx1I-gXPWwOxU9teRUJ_c due to access control checks.
[Error] Failed to load resource: Could not connect to the server. (replicache-pull, line 0)
https://stackoverflow.com/questions/63141448/safari-fetch-ap...
I work on Fly.io, but there's very little vendor lock in here. We can't afford to lock people in, we're too small. We need to make their existing stuff work with zero friction.
Also, integrations use polling: https://github.com/FastComments/fastcomments-integrations/tr...
The integrations work kind of like DB slave replication. They do an initial sync and then maintain state via the event stream.
Excellent write. It would be great to know why CockroachDB failed your needs.
Definitely interested in understanding end user benefit of the distributed database given one of purposes of library is to hide write latency and there needs to be coordination for every write.
web apps are sorely lacking a core storage technology
whoever gets their first may not make a lot of money but they'll be more influential than react (because the schema design will penetrate native dev as well)