from Hacker News

Show HN: Fireproof – local-first database with Git-like encrypted sync

by jchanimal on 11/19/24, 3:19 PM with 48 comments

Hi, HN! As a cofounder of Couchbase, I pioneered mobile sync, and I’ve always wanted to bring the speed and reliability of local-first data to the web, incubating PouchDB among other efforts. I learned the constraints of real world financial applications at McKinsey & Company FinLab, and Merkle integrity research at Protocol Labs taught me smart contract data structures. As part of the JavaScript community (and early hosting provider for NPM) I’ve been waiting, and now with the availability of APIs like Passkeys and Origin Private Filesystem, I’m happy to say the browser is ready to support embedded databases.

Front-ends are a lot easier to write when your database handles live sync for you, but the existing solutions rely on heavyweight cloud APIs instead of putting the smarts at the edge, where it belongs. I started from a different set of constraints, and arrived at a lightweight embedded database that uses a git-like data model to offer cryptographic causal consistency across browsers, edge functions, and anywhere TypeScript runs.

It’s designed to make building full-featured apps as simple as calling `db.put({ hello: "world" })` and syncing them as easy as calling `connect(db, remote)`. People are using Fireproof for AI character chat[1], personal finance[2], and hedge funds[3], and we aim to be simple enough for novice coders to build enterprise-critical apps. Fireproof makes product owners dangerous, because just a little bit of code can define an application’s workflow and data model. See the code sample below.

The reactive APIs[4] are designed for live collaboration so your user interfaces update automatically, making it an easy way to add query collaboration to legacy dashboards, or write new interactive tools for your team. Merkle CRDTs[5] provide multi-writer safety while maintaining tamperproof data provenance, conflict tracking, and deterministic merges. The storage engine writes content-addressed encrypted files that can be synced via commodity backends like S3 or Cloudflare[], without sacrificing data integrity.

Our contributors include legends like Damien Katz, Meno Abels, Mikeal Rogers, and Alan Shaw. Fireproof is open source (Apache/MIT) and we know there are rough edges, so we hope this post stirs up collaborators![6] Please `npm install @fireproof/core` and give us feedback[7]. We are on the stable side of beta, so it’s a great time for the adventurous to join. I’m excited to see all the apps people write now that it’s easy!

[1] https://github.com/fireproof-storage/catbot/tree/main

[2] https://fireproof.storage/posts/quickcheck:-print-checks-at-...

[3] https://fireproof.storage/posts/contributor-spotlight:-danie...

[4] https://use-fireproof.com/docs/react-tutorial

[5] https://fireproof.storage/posts/remote-access-crdt-wrapped-m...

[6] https://github.com/fireproof-storage/fireproof/issues

[7] https://discord.gg/DbSXGqvxFc

  • by damienkatz on 11/19/24, 3:50 PM

    Creator of Apache CouchDB here. Fireproof is a very clever system for building secure, collaborative apps. Your apps can work offline and when connected automatically sync changes to local storage and display them immediately to the UI. It has much of that same CouchDB magic with almost no complexity or headache on the backend.

    Very coo…errr…hot!

  • by jchrisa on 11/19/24, 3:24 PM

    Thanks for reading — Fireproof creator here, happy to answer any questions.

    We are in-flight on our cloud launch, so consider it a preview of the experience we are building. We’ll soon be shipping more complete authorization with UCAN capability delegation, and we are working on mature key rotation. I can't wait to hear what people want to build with it.

  • by jchrisa on 11/19/24, 4:53 PM

    Here is the code sample I mentioned, example React usage (see our homepage for Vanilla JS)

        import { useFireproof, useDocument } from "use-fireproof";
        import { connect } from "@fireproof/cloud";
    
        export default function App() {
          const { database, useLiveQuery } = useFireproof("my_db");
          connect(database, "my-remote");
          const { docs } = useLiveQuery("_id");
    
          const [newDoc, setNewDoc, saveNewDoc] = useDocument({ input: "" });
    
          const handleSubmit = async (e) => {
            e.preventDefault();
            if (newDoc.input) {
              await saveNewDoc();
              setNewDoc({ input: "" }); // Reset for new entry
            }
          };
    
          return (
            <div>
              <form onSubmit={handleSubmit}>
                <input
                  value={newDoc.input}
                  onChange={(e) => setNewDoc({ input: e.target.value })}
                />
                <button>Add</button>
              </form>
              <ul>
                {docs.map((doc) => (
                  <li key={doc._id}>{JSON.stringify(doc)}</li>
                ))}
              </ul>
            </div>
          );
        }
  • by myklemykle on 11/19/24, 7:48 PM

    I wrote a demo with Firebase to sync a bunch of music data between N web clients. EZ-PZ. All of the integrity stuff was icing on the cake, but basically it was an ultra-simple way to replicate data btwn web app users, without worrying about the server side at all.
  • by bosky101 on 11/20/24, 3:47 AM

    The website and example, and usage looks clean. Kudos! I have some questions around what's happening under the hood that werent evident from an initial read of both your website as well as GitHub.

    1. Does subscribe listen for new changes on a transient server(just a queue). Or from a more persistent store?

    2. Where do the events persist? I didn't see a connector to postgres. I did see one for s3.

    3. What is the default persistence layer you are advocating?

    4. Let's say you run 3 instances of the self hosted server. And a random one of them gets a teacher. And 2 random other students gets load balanced to two other servers. How does the teacher get all messages? What's the thread in a distributed setting

    5. How do you filter only messages. Eg: only since time T.

    6. Pagination / limits to avoid any avalanche?

    7. Auth? Custom auth/jwt?

    8. REST API to produce?

    9. Are consumers restricted to browsers? What about one in node?

    10. BONUS: Have you tested if this works embedded as an iframe or embedded in an native/react native mobile app?

  • by johnson_brad on 11/19/24, 4:42 PM

    Fireproof brings really solid DX with a passionate team supporting it. It's deceptively powerful as an embedded db, but delivers the kind of experience you'd typically expect from a good frontend stack. Since modern web development keeps pushing the limits of browsers, I love local-first projects like this.
  • by mschoch on 11/19/24, 5:08 PM

    Fireproof engineer here, I come from the backend engineer perspective, so I thought I'd share some of my personal hacks on Fireproof:

    Fireplace - tooling to deploy Fireproof apps and sync data across your Tailscale network. Once all the computers you care about are on your tailnet, of course you want all the browsers on the tailnet to easily sync with one another.

    Go Implementation - Fireproof bills itself as a realtime database that runs anywhere, and I want to make sure that includes inside your Go applications. This will allow your Go application to become a full-fledged reader/writer of the Fireproof ledger.

    I'm excited to see what other people want to build and answer any questions.

  • by tvachon on 11/19/24, 5:37 PM

    Love this! I've been following and contributing to Fireproof for a while now and it's super exciting to see how far it's come - excited to use it on a project soon!
  • by nadyanadya2024 on 11/19/24, 3:31 PM

    Fireproof has come a long way and If you’re looking to cut down on backend complexity while having a reliable user experience, Fireproof is worth a serious look.
  • by fastandfearless on 11/20/24, 8:37 AM

    I'm happy working on fireproof. The people are very nice. In terms of technology it's a new kind on block. Yes a bit of database and a bit of Blockchain with git rebase.That's why we moving from naming it database to ledger lately.
  • by dscape on 11/19/24, 3:28 PM

    Congrats on launching Fireproof! You mentioned syncing content-addressed encrypted files via S3 or Cloudflare. For developers already using existing cloud-first setups, how seamless is it to integrate Fireproof as a hybrid solution?
  • by 3jnsn on 11/19/24, 5:38 PM

    One of my favorite use cases for Fireproof is storing and searching across vector embeddings. It can act as a RAG for LLMs, operating on the backend for all clients, and/or the frontend for customized or offline scenarios.
  • by jchrisa on 11/19/24, 5:59 PM

    Here are some apps people have built, for inspiration:

    * Bloopernet Drum Machine: https://news.ycombinator.com/item?id=42177005

    * Slack style team chat https://firehouse-chat.jchris.partykit.dev/

    * PartKit Cloudflare https://blog.partykit.io/posts/fireproof-database-connector

  • by p1pp1n on 11/20/24, 11:39 PM

    It has been quite inspiring watching you grow this from a concept into a fully functioning next gen DB. Cheers!
  • by astrophellita on 11/22/24, 8:41 PM

    Nice stuff! It's extremely fast and a breeze to use with cooperative applications that are shared by multiple users in realtime. Pretty sick work on the part of J Chris Anderson and the Fireproof team.
  • by heapwolf on 11/19/24, 6:15 PM

    This looks really promising, this should run in any web-like environment right? Specifically it might be interesting to build a Socket App with this (https://github.com/socketsupply/socket)
  • by k__ on 11/19/24, 7:26 PM

    Does it support custom backends?
  • by pajop on 11/19/24, 8:22 PM

    Are there tutorials available?
  • by lloydcenter on 11/21/24, 7:00 AM

    Thing is.. Fireproof is brilliant and has a fun culture.
  • by bburns_km on 11/19/24, 6:34 PM

    I had recently been experimenting with pouch/couchdb, and got it working locally, then came across a post on reddit about Fireproof. I really liked how it sounded - like the evolution of pouch/couch.

    So I set it up on a node app - it works great locally -

      import { fireproof } from '@fireproof/core'
    
      const db = fireproof(dbname)
    
      db.put(doc)
    
      async read(id) {
        if (id) return await db.get(id)
        return await db.allDocs()
      }
    
    Then I wanted to sync the data to the cloud - just the simplest thing possible, but I got lost in all the connector options and descriptions. I tried setting up PartyKit, but got bogged down in it all and eventually went on to something else.

    So it would be great if the home page included a simple demo with a connector - Amazon S3 or PartyKit - including setting up the cloud db.

    Thanks, and good luck with everything - it looks amazing...

    (side note: HackerNews doesn't let you format code with ``` ??)

  • by hfnciol on 11/19/24, 9:49 PM

    Looks very cool.

    Have been developing something similar (local first data tooling) for a project of my own.

    The one thing stopping me from swapping my incomplete implementation with this is that I am unable to find how to connect to a db I control.

    In another thread you self described your backend options as "maybe too many backend implementions", but I am only seeing established cloud providers (meh) and ipfs (cool, but sluggish).

    https://use-fireproof.com/docs/connect

    Can I have the persistent data stored in PostgreSQL on a server I am running?

    Are fireproof servers involved in mediating the syncing or is it done through the client?

  • by sethmhardy on 11/20/24, 3:41 AM

    Great news! I suspect we'll be using this when we have to go back to on-prem hardware.
  • by wizzard0 on 11/23/24, 5:53 AM

    fireproof is super cool!

    built a lot over couchdb 10y ago and fireproof checks most of the pain points i had with it!

  • by cdurth on 11/20/24, 3:19 AM

    How does this compare to Dexie js and Dexie cloud?
  • by NatMars on 11/19/24, 7:46 PM

    A real solution for real world applications!!!