by calcsam on 9/12/23, 8:46 PM with 88 comments
by thecodrr on 9/13/23, 8:42 AM
1. What to do with stale user data? What happens if a user doesn't open the app for a year? How do you handle migrations?
2. What about data corruption? What happens if the user has a network interruption during a sync? How do you handle partial states?
3. What happens when you have merge conflicts during a sync? CRDT structures are not even close to enough for this.
4. What happens when the user has millions of items? How do you handle sync and storage for that? How do you handle backups? How do you handle exports?
One would imagine that having all your data locally would make things fast and easy, but oh boy! Not everyone has a high end machine. Mobiles are really bad with memory. iOS and Android have insane level of restrictions on how much memory an app can consume, and for good reason because most consumer mobile phones have 4-6 gbs of RAM.
All these problems do not exist in a non-local-first situation (but other problems do). Things are actually simpler in a server-first environment because all the heavy lifting is done by you instead of the user.
by simonw on 9/13/23, 1:56 AM
My notes here: https://til.simonwillison.net/sqlite/cr-sqlite-macos
by nixpulvis on 9/13/23, 4:00 AM
I’m a big proponent of being able to run the code locally, but this seems to be more along the lines of, first update some local DB, then sync it.
I see nothing fundamentally different here than using a well designed API for ‘sync’, and none of it does anything to revive the web browsing experience. Gone are the days where “page load” meant that you could safely begin reading without interruption. This whole “web application” thing was a foobar’d mess. Yes, we probably want some level of interactivity, but I fail to see how casting the web application problem as a database sync issue addresses the fundamental issues.
That being said, I can see the appeal. It’s a clean interface to work with and it allows for new frameworks to displace existing problematic frameworks.
I just worry that in 5 years we’ll all be complaining about poorly designed schema and overloaded table updates again. The solution? Microservices with their own smaller tables which batch updates to the larger system of course!
by pmontra on 9/13/23, 5:08 AM
In complex turn based games there is a lot to do locally before sending the move to the server. Think about moving tens of units on a map. The problem here is of there are unknown parts of the map to explore. You need a connection to the server. However anything else will run faster because it doesn't have to hit the network and a round trip to the database on the server.
The post writes about Figma being developed local first. You don't have to hit the network while drawing, if you don't have to collaborate with someone on a shared document. Then you send the state to the server and your private state becomes known to all the other coworkers.
by matharmin on 9/13/23, 9:08 AM
The biggest benefit for developers that I've seen is that state management gets _much_ simpler when it is all handled by a local database. Just run queries directly on every app/page, with live queries if the framework supports it. No need for maintaining in-memory app state and propagating changes to the different parts, no need for caching and associated cache invalidation issues.
The caveat is that keeping the local database in sync with the server is not easy, and it's best to use other tooling for it. I'm working on PowerSync [1] myself, and ElectricSQL [2] also looks quite promising (both also mentioned in OP).
[1]: https://powersync.co
by svilen_dobrev on 9/13/23, 8:15 AM
Worked wonders - each user reading/posting requests/responses in own "channel", the channel being replicated to server and any of user's devices, central server responding to requests and eventually cross-polinating the channels if need be.
Did a web client (pouchDB), android (touchDB), ios (forgot the name).. essentially each platform has its own implementation of the relatively simple protocol of Couchdb. So all one does is listen/read/write to a single local db.. and then db is doing all the communication, whenever possible. Simplified the clients sooooo much - reduced work and complexity like 50%.
Eh, the thing didn't go anywhere as product but nevermind :/
have fun
by ak39 on 9/13/23, 10:39 AM
They worked and they were necessary.
Things changed when internet connectivity eventually became cheap and ubiquitous and the complexity of developing such applications then didn't merit the effort.
Now, the swing back to "local-first" is mainly for user-experience reasons. The same theoretical complexities of "eventually synchronised" that existed for the "briefcase app" are still present.
Is the complexity worth it though?
by madspindel on 9/13/23, 6:30 AM
by sarchertech on 9/13/23, 2:52 AM
However, the vast majority of apps don’t need this. Please don’t do anything like this for your boring enterprise CRUD app (60 fps CRUD? What even?).
I know many of you will, so I guess I’ll just console myself with the likelihood I’ll have a ton of consulting work in the next 5-10 years.
by bfung on 9/13/23, 7:43 AM
The thin/thick client cycle starts again...
by zknill on 9/13/23, 2:36 PM
I think the collaboration experiences that you see in Figma, Miro, Notion, etc should _just be possible_ for all apps. And for a lot of product development teams they probably look at what would be involved to try and build a realtime collaboration experience and think it's just gonna be either too hard, or too much work. But the 'art of the possible' as definitely moved on (as evidenced by the list of companies in the blog post that are addressing these challenges).
As a bunch of the other commenters point out, the hard-part isn't just building the code for the UI that will show collaboration experiences, the hard-part is often handling the stateful connections (websockets) and making sure the messages are delivered to the right clients. You either build a stateless/polling mechanism over HTTP, or you have to deal with trying to scale and load-balance websocket connections. Neither approach is particularly easy.
by edweis on 9/13/23, 1:53 AM
I am interested in your take on local-first.
by wink on 9/13/23, 12:39 PM
I'm talking hobby projects here where my main problem is getting it to a usable state so that it works (for me) without quirks and actually solves my problem instead of being a proof of concept.
Assuming the data model is kinda boring I wonder if there's a way to leverage some "framework" to offload the syncing part, for a simple CRUD app. There should be no resolution strategies, it has like a users table, a posts table and a comments table (classic blog example), this should be easily abstractable, or not? But then most people are talking about Websites as frontends, I guess, which changes my current "I'm gonna write a Qt native client" idea.
by xg15 on 9/13/23, 7:08 AM
by somishere on 9/13/23, 12:57 PM
I think what I liked the least was the complexity introduced to the client (a fun statement considering some of the apps I've worked on recently). Instead of being able to abstract away business logic behind a simple API, it was there in all it's "haute logique" glory - hardwired into my otherwise dead simple todo app.
That said, still love the concept. Keen to try some of the new names. Tho the idea of it living in a worker, where it can be queried / controlled / deployed (or replaced) as if remote, has a nice ring to it?
by steve1977 on 9/13/23, 7:19 AM
by matlin on 9/13/23, 5:31 PM
One common misconception with local-first, is that it's essentially local-only. But I believe the best experience is where the server and client are equally capable and only a partial set of data is stored on the client. The server still maintains authority but the client is always snappy when updated data or changing queries/views. We basically call this a full-stack database and we're building that at [Triplit](www.triplit.dev).
by emmanueloga_ on 9/13/23, 1:57 AM
by pacifika on 9/13/23, 6:09 AM
Why not skip the whole App Store generation that’s to come and build a browser extension today?
by chadk on 9/13/23, 7:05 AM
by mooktakim on 9/13/23, 8:10 AM
by mike_hearn on 9/13/23, 12:55 PM
Despite the name it's not really a web exclusive community.
Actually the web isn't a great way to do this, there's alternate stacks that maybe can work better. The Android stack has been rewritten in recent years by Google and parts are now usable on desktop and iOS. Someone posted this to the Kotlin slack earlier today:
https://medium.com/@mike_21858/auto-generating-an-http-serve...
I’ve been working on apps for limited connectivity environments such as Afghanistan and rural Rwanda for many years. I think we have to make offline-first a lot easier than it is now. Our own app Ustad Mobile has more than 60 tables in the database. Writing manual logic to run all of those offline would 1) exceed the resources we have and 2) be prone to human error.
This article introduces Door: our open-source way to automate the generation of an offline-first data layer using Kotlin Multiplatform and Kotlin Symbol Processing (KSP). Door is still a work-in-progress and not ready for production use in other apps yet, but we think it’s a game-changer for offline-first development. Feedback on the concept and API is welcome. Door can automatically generate HTTP server endpoints and an offline first client based on a Room database.
The idea is that you can use Kotlin for both server, Android and also a desktop client, and with KMP you're also able to use it on iOS for the backend (UI is still in Swift). So you can share your domain model across all devices and the underlying sync protocols and logic can be auto generated using a compiler plugin. The underlying DB is SQLite wrapped using a library from Google called Room which makes SQLite a bit easier to use.
Deploying such an app on Android is obvious, for iOS you need to learn about Kotlin/Multiplatform but obviously SQLite can run on iOS just fine and Kotlin can interop with Objective-C/Swift. For desktop, deployment is easy if you use Conveyor, which the Ustad guys are planning to do, as it supports Kotlin desktop apps out of the box. You do need to buy signing certificates (or distribute via the app stores) but that's a one time cost.
The main issue here is if you're wedded to the web. Probably stuff can be done with WASM but it's not going to be as natural as just using the tech on the JVM.