from Hacker News

Ask HN: Should data warehouses be the back end of data apps?

by tamarlikesdata on 12/4/23, 7:50 AM with 2 comments

Are data warehouses a bad choice for data apps because they lack freshness, latency, and query concurrency needs?

Or if we're able to improve ingestion latency, query response time, and concurrency/scaleability, maybe it can be a good choice?

  • by ssss11 on 12/4/23, 9:00 AM

    Data warehouses are for strategic data use cases, that is, data that can be updated once per day or less. Think things like “what is my year on year profit for the footwear segment?” Or “how often in the past 5 years have customers from segment x contacted us for tender, what % did we win, and what was the net profit?”

    If you need near real time data source for operational / transactional types of reporting then it is apparently not the right solution.

    My company has no suitable in built reporting options in source systems so unfortunately we will be trying to serve near real time data via a data warehouse. We may be able to get the refresh pipeline down to about 1 hr by throwing resources at it.