from Hacker News

Amazon CloudWatch Logs Insights

by jbergknoff on 11/28/18, 12:13 AM with 31 comments

• by pcx on 11/28/18, 11:27 AM

  CloudWatch has the worst UX of any log management tool I've used. We are using Logentries currently, and it is so easy and intuitive.

  I just don't get how AWS developers are able to accept the eyesore that is their UX. It's easy to miss things in any AWS service's UI. I keep discovering functionality even after years of use. Google Cloud UX is light years ahead. Also, Google's Stackdriver seems great, haven't used it yet though. Would be great if any Stackdriver users here share how it is better than CloudWatch or other log management tools.

• by nlh on 11/28/18, 3:35 AM

  Interesting -- they're going after the various log management companies (Scalyr, DataDog, Splunk, Sumo Logic, etc.).

  Figured this was bound to come eventually since it's a very very big market and their basic CloudWatch product was lacking in many ways. It's not like Amazon to let an ecosystem eat their lunch.

  Few things stand out:

  (1) Per-query pricing seems...odd? Likely a good deal for small folks with a low volume of logs (i.e. just need to check actual AWS infrastructure logs vs. application logs), but if you have any actual volume this gets absurdly expensive ($0.005/GB scanned = $5 per query if you need to scan a terabyte. Large enterprises ingest multiple terabytes per day.)

  (2) The quote "I pick the first one, click Run query, the logs are scanned and the results are visible within seconds" doesn't sound terribly promising performance-wise. "Seconds" is an eternity in the log management world.

  Still, super interesting!

• by code4tee on 11/28/18, 4:53 AM

  It’s version 1.0 but this starts to make Splunk and similar utilities look less and less important or differentiating in the future of cloud.

  AWS understands most people using tools like Splunk probably only need a few simple features so AWS just goes and builds that and gives a lot of people excuses to dump expensive licenses for the AWS version of it. It’s a sneaky but highly successful business model.

• by allengeorge on 11/28/18, 12:21 PM

  We’ve had a poor experience with Cloudwatch for logs. The UX is poor and queries over large data sets take forever. So much so that I’m sure we’re ‘using it wrong’. What have others’ experiences been like?

• by j4mie on 11/28/18, 9:01 AM

  This looks great, and the query syntax is fairly easy to pick up (just from the "tips" in the UI - I haven't been able to find the documentation yet). The idea of connecting the parts of the query with pipe characters reminds me of https://stedolan.github.io/jq/

  The feature where you can add the queries to a Cloudwatch dashboard seems to be a bit broken at the moment. First, the version of the query that I copied to a dashboard didn't seem to respect the time limit I'd set so instead of looking at the last hour, it was looking at all time - could get accidentally very expensive! Also, I couldn't see a way to show the visualisation (ie the stats graph) on the dashboard - just the raw table of query results, which is not ideal. Hopefully I've missed something, or those niggles will be sorted out soon.

  Overall, very impressive!

• by fomojola on 11/28/18, 3:20 AM

  Any idea what the query engine/translation layer they are using is? I'm assuming there is something like Presto underneath it, but quite curious.

• by throwaway829 on 11/28/18, 9:38 AM

  Here's the user guide for Amazon CloudWatch Logs Insights: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Ana...

• by nprateem on 11/28/18, 10:29 AM

  Have they got an ML tool yet to allow me to configure alerts for things that aren't "normal"? Logs are all well and good but I generally just care about things that are abnormal for my stack.

• by borlum on 11/28/18, 8:42 AM

  Looks like Humio. I wonder if they can match speed and flexibility of Humio.

• by baseballMan on 11/28/18, 2:30 PM

  Reminds me of log analytics in Azure!