from Hacker News

Types of events in event-driven systems

by frankdejonge on 2/19/22, 1:07 PM with 42 comments

• by adamkl on 2/19/22, 2:07 PM

  I like the idea of using “fat” events as a means of data integration between systems.

  Calling them RESTful events is a good term as I’ve pitched in much the same fashion. Every time a system responds to a create/update/execute request, drop an equivalent message into an event stream. Initially these can form the basis of a streaming ETL to whatever systems you use for analytics, but as new internal systems are brought online (e.g a new CRM system to help sales manage all the leads signing up for your successful product) the integration is already there, waiting. Just add a new consumer to the existing stream.

  Overtime, you end up with a “data mesh”. If you look past the marketing of the term, it’s basically just means each application in your company should publish both sync (for real-time usage) and async (for data sharing between systems of record) interfaces.

• by ChrisMarshallNY on 2/20/22, 1:33 AM

  I tend to use whatever types of events the system provides, in realtime. In some cases, it's a real pain, as I can't really choose what I get.

  If I am using an event as a "record" of an event, I tend to store a "before" state, and an "after" state. In many cases, I can actually store entire serialized objects. This allows, for example, the ability to recreate an object state/context, simply by re-instantiating the stored object, and using the reanimated instance to replace the current one.

  Sort of a RESTFul event.

  In my work, whenever possible, I like to use a "RefCon" pattern. This is a classic pattern, that I learned from working with Apple code, but I know that it's a lot older.

  We simply attach a "RefCon" (Reference Context) to an event. This is usually an untyped value, that is supplied to the event source, when it is set up, then, to the event target, when the event fires. These days, it's easier, as closures/lambdas often allow you to access the initiating context. In the old days, it might have simply been a single-byte value, that could be applied into a lookup table, to allow the called context to re-establish.

  I recently ran into the need for one of these, in an SDK that I wrote, where I didn't. I'm still kicking myself. I think that I didn't supply it, because I thought it might have been too complicated for some consumers of the SDK.

  In the six years or so, since I wrote that SDK, I have been the only consumer.

• by doctor_eval on 2/20/22, 2:22 AM

  I found that GraphQL subscriptions give a great balance between trigger/signal and fat/REST events in complex environments.

  The events that a subscriber is interested in are decorated with the exact data the subscriber specifies at subscription time, avoiding an API call while providing exactly the level of richness the client requires.

• by hashimotonomora on 2/19/22, 5:24 PM

  Could anyone recommend more resources on this for studying?

• by hcarvalhoalves on 2/20/22, 2:30 AM

  I think what the author named "Domain events" are equivalent to derivatives, "Fat events" are equivalent to step-wise integration, and "Trigger events" represent steps on a function of data on the time domain.

• by morelisp on 2/19/22, 2:45 PM

  I like this ontology; we've divided our events into roughly similar categories but these are better names than we use.

  I'd further advance that

  - Trigger events and RESTful events are roughly the same thing, just a question of what you choose in latency vs. size vs. schema flexibility space. We even have events in our system whose schema inlines data below a certain size but links above that.

  - There is a fourth type: windowed reductions over domain events, e.g. the "changelog" in Kafka Streams. This bears a similar relation to domain events as the RESTful event does to trigger events.

• by pphysch on 2/19/22, 11:25 PM

  If you are using "trigger events" for security reasons, wouldn't it be more appropriate to send an ephemeral "eventID" rather than an actual "orderID"? Otherwise you may be leaking order IDs, which could potentially carry useful information for an attacker.

• by virtualbluesky on 2/20/22, 12:58 AM

  As "restful events" can't exist without "trigger events" to communicate deletion, the categories aren't really separable?

• by mokaman on 2/20/22, 3:58 AM

  How is this different from three types of messages in enterprise integration patterns.