from Hacker News

The problem with OpenTelemetry

by robgering on 6/14/24, 12:36 PM with 174 comments

  • by codereflection on 6/14/24, 6:47 PM

    I understand what the author is saying, but vendor lock-in with closed-source observability platforms is a significant challenge, especially for large organizations. When you instrument hundreds or thousands of applications with a specific tool, like the Datadog Agent, disentangling from that tool becomes nearly impossible without a massive investment of engineering time. In the Platform Engineering professional services space, we see this problem frequently. Enterprises are growing tired of big observability platform lock-in, especially when it comes to Datadog's opaque nature of your spend on their products, for example.

    One of the promises of OTEL is that it allows organizations to replace vendor-specific agents with OTEL collectors, allowing the flexibility of the end observability platform. When used with an observability pipeline (such as EdgeDelta or Cribl), you can re-process collected telemetry data and send it to another platform, like Splunk, if needed. Consequently, switching from one observability platform to another becomes a bit less of a headache. Ironically, even Splunk recognizes this and has put substantial support behind the OTEL standard.

    OTEL is far from perfect, and maybe some of these goals are a bit lofty, but I can say that many large organizations are adopting OTEL for these reasons.

  • by doctorpangloss on 6/14/24, 3:47 PM

    I don’t know what the Sentry guy is really saying - I mean you can write whatever code you want, go for it man.

    But I do have to “pip uninstall sentry-sdk” in my Dockerfile because it clashes with something I didn’t author. And anyway, because it is completely open source, the flaws in OpenTelemetry for my particular use case took an hour to surmount, and vitally, I didn’t have to pay the brain damage cost most developers hate: relationships with yet another vendor.

    That said I appreciate all the innovation in this space, from both Sentry and OpenTelemetry. The metrics will become the standard, and that’s great.

    The problem with Not OpenTelemetry: eventually everyone is going to learn how to use Kubernetes, and the USP of many startup offerings will vanish. OpenTelemetry and its feature scope creep make perfect sense for people who know Kubernetes. Then it makes sense why you have a wire protocol, why abstraction for vendors is redundant or meaningless toil, and why PostHog and others stop supporting Kubernetes: it competes with their paid offering.

  • by ankitnayan on 6/15/24, 6:51 AM

    I think all of us agree that OpenTelemetry's end-goal of making Observability vendor neutral is futuristic and inevitable. We can complain about it being hard to get started, bloated, etc but the value it provides is clear, esp, when you are paying $$$ to a vendor and stuck with it.

    OpenStandards also open up a lot of usecases and startups too. SigNoz, TraceTest, TraceLoop, Signadot, all are very interesting projects which OpenTelemetry enabled.

    The majority of the problem seems like sentry is not able to provide it's sentry like features by adopting otel. Getting involved at the design phase could have helped shaped the project that could have considered your usecases. The maintainers have never been opposed to such contributions AFAIK.

    Regarding, limiting otel just to tracing would not be sufficient today as the teams want a single platform for all observability rather than different tools for different signals.

    I have seen hundreds of companies switch to opentelemetry and save costs by being able to choose the best vendor supporting their usecases.

    lack of docs, learning curve, etc are just temporary things that can happen with any big project and should be fixed. Also, otel maintainers and teams have always been seeking help in improving docs, showcasing usecases, etc. If everyone cares enough for the bigger picture, the community and existing vendors should get more involved in improving things rather than just complaining.

  • by no_circuit on 6/14/24, 4:53 PM

    IMO this boils down how one gets paid to understand or misunderstand something. A telemetry provider/founder is being commoditized by an open specification in which they do not participate in its development -- implied by the post saying the author doesn't know anyone on the spec committee(s). No surprise here.

    Of course implementing a spec from the provider point of view can be difficult. And also take a look at all the names of the OTEL community and notice that Sentry is not there: https://github.com/open-telemetry/community/blob/86941073816.... This really isn't news. I'd guess that a Sentry customer should just be able to use the OTEL API and could just configure a proprietary Sentry exporter, for all their compute nodes, if Sentry has some superior way of collecting and managing telemetry.

    IMO most library authors do not have to worry about annotation naming or anything like that mentioned in the post. Just use the OTEL API for logs, or use a logging API where there is an OTEL exporter, and whomever is integrating your code will take care of annotating spans. Propagating span IDs is the job of "RPC" libraries, not general code authors. Your URL fetch library should know how to propagate the Span ID provided that it also uses the OTEL API.

    It is the same as using something like Docker containers on a serverless platform. You really don't need to know that your code is actually being deployed in Kubernetes. Use the common Docker interface is what matters.

  • by serverlessmom on 6/14/24, 5:59 PM

    An argument that OpenTelemetry is somehow 'too big' is an example of motivated reasoning. I can understand that A Guy Who Makes Money If You Use Sentry dislikes that people are using OTel libraries to solve similar problems.

    Context propagation and distributed tracing are cool OTel features! But they are not the only thing OTel should be doing. OpenTelemetry instrumentation libraries can do a lot on their own, a friend of mine made massive savings in compute efficiency with the NodeJS OTel library: https://www.checklyhq.com/blog/coralogix-and-opentelemetry-o...

  • by wdb on 6/14/24, 4:10 PM

    Personally, I like OpenTelemetry, nice standardised approach. I just wished the vendors would have better support for the semantic conventions defined for a wide variety of traces.

    I quite like the idea of only need to change one small piece of the code to switch otel exporters instead of swapping out a vendor trace sdk.

    My main gripe with OpenTelemetry I don't fully understand what the exact difference is between (trace) events and log records.

  • by AndreasBackx on 6/14/24, 8:15 PM

    I have been trying to find an equivalent for `tracing` first in Python and this week in TypeScript/JavaScript. At my work I created an internal post called "Better Python Logging? Tracing for Python?" that basically asks this question. OpenTelemetry was also what I looked at and since I have looked at other tooling.

    It is hard to explain how convenient `tracing` is in Rust and why I sorely miss it elsewhere. The simple part of adding context to logs can be solved in a myriad of ways, yet all boil down to a similar "span-like" approach. I'm very interested in helping bring what `tracing` offers to other programming communities.

    It very likely is worth having some people from the space involved, possibly from the tracing crate itself.

  • by wvh on 6/14/24, 5:26 PM

    I have surveyed this landscape for a number of years, though I'm not involved enough to have strong opinions. We're running a lot of Prometheus ecosystem and even some OpenTelemetry stacks across customers. OpenTelemetry does seem like one of these projects with an ever expanding scope. It makes it hard to integrate parts you like and keep things both computing-wise and mentally lightweight without having to go all-in.

    It's not anymore about hey, we'll include this little library or protocol instead of rolling our own, so we can hope to be compatible with a bunch of other industry-standard software. It's a large stack with an ever evolving spec. You have to develop your applications and infrastructure around it. It's very seductive to roll your own simpler solution.

    I appreciate it's not easy to build industry-wide consensus across vendors, platforms and programming languages. But be careful with projects that fail to capture developer mindshare.

  • by fractalwrench on 6/14/24, 8:08 PM

    The main interest I've seen in OTel from Android engineers has been driven by concerns around vendor lock-in. Backend/devops in their organisations are typically using OTel tooling already & want to see all telemetry in one place.

    From this perspective it doesn't matter if the OTel SDK comes bundled with a bunch of unnecessary code or version conflicts as is suggested in the article. The whole point is to regain control over telemetry & avoid paying $$$ to an ambivalent vendor.

    FWIW, I don't think the OTel implementation for mobile is perfect - a lot of the code was originally written with backend JVM apps in mind & that can cause friction. However, I'm fairly optimistic those pain points will get fixed as more folks converge on this standard.

    Disclaimer: I work at a Sentry competitor

  • by markl42 on 6/14/24, 4:59 PM

    At the risk of hijacking the comments, I've been trying to use OTel recently to debug performance of a complex webpage with lots of async sibling spans, and finding it very very difficult to identify the critical path / bottlenecks.

    There's no causal relationships between sibling spans. I think in theory "span links" solves this, but afaict this is not a widely used feature in SDKs are UI viewers.

    (I wrote about this here https://github.com/open-telemetry/opentelemetry-specificatio...)

  • by tnolet on 6/14/24, 4:41 PM

    A recent example of OTel confusion.

    I could for the life of me not get the Python integration send traces to a collector. Same URL, same setup same API key as for Nodejs and Go.

    Turns out the Python SDK expect a URL encoded header, e.g. “Bearer%20somekey” whereas all other SDKs just accept a string with a whitespace.

    The whole split between HTTP, protobuf over HTTP and GRPC is also massively confusing.

  • by NeutralForest on 6/14/24, 2:37 PM

    It resonates. As an intern I had to add OTEL to a Python project and I had to spend a lot of time in the docs to understand the concepts and implementation. Also, the Python impl has a lot of global state that makes it hard to use properly imo.
  • by BiteCode_dev on 6/14/24, 2:55 PM

    100% agree.

    Every time I tried to use OT I was reading the doc and whispering "but, why? I only need...".

  • by spullara on 6/14/24, 8:03 PM

    There is a huge whole in using spans as they are specified. Without separating the start of a span from the end of a span you can never see things that never complete, fail hard enough to not close the span, or travel through queues. This is a compromise they made because typical storage systems for tracing aren't really good enough to stitch them all back together quickly. Everyone should be sending events and stitching it all together to create the view. But instead we get a least common denominator solution.
  • by drewbug01 on 6/14/24, 3:51 PM

    As a contributor to (and consumer of) OpenTelemetry, I think critique and feedback is most welcome - and sorely needed.

    But this ain’t it. In the opening paragraphs the author dismisses the hardest parts of the problem (presumably because they are human problems, which engineers tend to ignore), and betrays a complete lack of interest in understanding why things ended up this way. It also seems they’ve completely misunderstood the API/SDK split in its entirety - because they argue for having such a split. It’s there - that’s exactly what exists!

    And it goes on and on. I think it’s fair to critique OpenTelemetry; it can be really confusing. The blog post is evidence of that, certainly. But really it just reads like someone who got frustrated that they didn’t understand how something worked - and so instead of figuring it out, they’ve decided that it’s just hot garbage. I wish I could say this was unusual amongst engineers, but it isn’t.

  • by shaqbert on 6/14/24, 4:01 PM

    Otel is indeed quite complex. And the docs are not meant for quick wins...

    Otelbin [0] has helped me quite a bit in configuring and making sense of it, and getting stuff done.

    [0]: https://www.otelbin.io/

  • by epgui on 6/14/24, 4:59 PM

    Anyone else finding this very difficult to read? I’d really recommend feeding this through a grammar checker, because poor grammar betrays unclear thinking.
  • by grenbys on 6/16/24, 7:24 PM

    I think there are two separate perspectives. For developers Open Telemetry is a clear win - high-quality vendor agnostic instrumentation backed by a reputable orgs. I instrumented with traces many business critical repos at my company (major customer support SaaS) with OTEL in Ruby, Python, JS. Not once was I confused/blocked/distracted by the presence of logs/metrics in the spec. However, can’t say much from the observability vendor perspective trying to be fully compatible with OTEL spec including metrics/logs. Article mentions customers having issues with using tracing instrumentation - it would’ve been great to back this up with corresponding github issues explaining the problems. Based on the presented JS snippet (just my guess) maybe the issue is with async code where the “span.operation” span gets immediately closed w/o waiting for the doTheThing()? Yeah - that’s tricky in JS given its async primitives. We ended up just maintaining a global reference to the currently active span and patching some OTEL packages to respect that. FWIW Sentry JS instrumentation IS really good and practical. Would have been great if Sentry could donate/contribute/influence to OTEL JS SIG with specific improvements - would be a win-win. As much as I hate DataCanine pricing they did effectively donated their Ruby tracing instrumentation to OTEL which I think is one of the best ones out there.
  • by hobofan on 6/14/24, 4:46 PM

    This seems to be more of a branding problem than anything.

    OP (rightfully) complains that there is a mismatch between what they (can) advertise ("We support OTEL") and what they are actually providing to the user. I have the same pain point from the consumer side, where I have to trial multiple tools and service to figure out which of them actually supports the OTEL feature set I care about.

    I feel like this could be solved by introducing better branding that has a clearly defined scope of features inside the project (like e.g. "OTEL Tracing") which can serve as a direct signifier to customers about what feature set can be expected.

  • by antonyt on 6/14/24, 3:06 PM

    OTel is flawed for sure, but I don't understand the stance against metrics and logs. Traces are inherently sampled unless you're lighting all your money on fire, or operating at so small a scale that these decisions have no real impact. There are kinds of metrics and logs which you always want to emit because they're mission-critical in some way. Is this a Sentry-specific thing? Does it just collapse these three kinds of information into a single thing called a "trace"?
  • by dboreham on 6/14/24, 3:30 PM

    I've used Otel quite a bit (in JVM systems) and honestly didn't know it did more than tracing.

    That said, I think this rot comes from the commercial side of the sector -- if you're a successful startup with one product (e.g. graphing counters), then your investors are going to start beating you up about why don't you expand into other adjacent product areas (e.g. tracing). Repeat previous sentence reversed. And so you get Grafana, New Relic, et al). OpenTelemetry is just mirroring that arrangement.

  • by edenfed on 6/15/24, 5:03 AM

    You can absolutely use just the OTel APIs and use something else besides the OTel SDK. Here is a blog post about how we did it with eBPF: https://odigos.io/blog/Integrating-manual-and-auto
  • by prymitive on 6/14/24, 4:37 PM

    I only learned about OT after Prometheus announced some deeper integration with it. Reading OT docs about metrics feels like every little problem has a dedicated solution in the OT world, even if a more generalised one already covers it. Which is quite striking coming from the Prometheus world.
  • by PeterZaitsev on 6/14/24, 7:06 PM

    OpenTelemetry is interesting, On one side it is designed as the "commodity feeder" to number of proprietary backends as DataDog, on other hand we see good development of Open Source solutions as SigNoz and Coroot with good Otel support.
  • by ris on 6/14/24, 9:09 PM

    1. The main reason I want to use otel is so I can have one sidecar for my observability, not three, each with subtly different quirks and expectations. (also the associated collection/aggregation infrastructure)

    2. I honestly think the main reason otel appears so complex is the existing resources that attempt to explain the various concepts around it do a poor job and are very hand-wavey. You know the main thing that made otel "click" for me? Reading the protobuf specs. Literally nothing else explained succinctly the relationships between the different types of structure and what the possibilities with each were.

  • by esafak on 6/14/24, 10:07 PM

    This caught my eye:

    > Logs are just events - which is exactly what a span is, btw - and metrics are just abstractions out of those event properties. That is, you want to know the response time of an API endpoint? You don't rewind 20 years and increment a counter, you instead aggregate the duration of the relevant span segment. Somehow though, Logs and Metrics are still front and center.

    Is anyone replacing logs and metrics with traces?

  • by dtjohnnymonkey on 6/15/24, 3:37 PM

    > That means what we actually want is a way to say “hey OpenTelemetry SDK, give us all the current spans in the buffer”.

    Isn’t this exactly what the SpanExporter API is for? This is in the Go SDK, I suppose it may not be available in other SDKs.

    I have used this API to convert OTel spans into log messages as we currently don’t have a distributed tracing vendor.

  • by dan-allen on 6/14/24, 10:50 PM

    I keep checking in on OpenTelemetry every few months to see if the bits we need are stable yet. There’s been very little progress on the things we’re waiting for.

    I don’t follow closely enough to comment on possible causes.

    What I do know is that the surface area of code and infrastructure that telemetry touches means adopting something unfinished is a big leap of faith.

  • by cogman10 on 6/14/24, 3:34 PM

    Perhaps the real problem with OTel (IMO) is it's trying to be everything for everyone and every language. It's trying to have a common interface so that you can write OTel in Java or Javascript, python or rust, and you basically have the exact same API.

    I suspect OP is seeing this directly when talking about the cludgyness of the Javascript API.

  • by zellyn on 6/14/24, 6:53 PM

    Are they basically just saying that the OpenTelemetry client APIs should be split from the rest of the pieces of the project, and versioned super conservatively?

    The simple API they describe is basically there in OTel. The API is larger, because it also does quite a few other things (personally, I think (W3C) Baggage is important too), but as a library author I should need only the client APIs to write to.

    When implementing, you're free to plug in Providers that use OpenAPI-provided plumbing, but you can equally well plug in Providers from DataDog or Sentry or whatever.

    Unless I'm missing something, any further complaints could be solved by making sure the Client APIs (almost) never have backward-incompatible changes, and are versioned separately.

  • by EdSchouten on 6/14/24, 5:42 PM

    > Its not a hard problem, [...]. At its core its structured events that carry two GUIDs along with them: a trace ID and a parent event ID. It is just building a tree.

    I've always wondered, what's the point of the trace ID? What even is a trace?

    - It could be a single database query that's invoked on a distributed database, giving you information about everything that went on inside the cluster processing that query.

    - Or it could be all database calls made by a single page request on a web server.

    - Or it could be a collection of page requests made by a single user as part of a shopping checkout process. Each page request could make many outgoing database calls.

    Which of these three you should choose merely depends on what you want to visualize at a given point in time. My hope is that at some point we get a standard for tracing that does away with the notion of trace IDs. Just treat everything going on in the universe as a graph of inter-connected events.

  • by noname120 on 6/14/24, 5:26 PM

    tl;dr OpenTelemetry eats Sentry's cake by commoditizing what they do and the reaction of the founder of Sentry is to be very upset about it rather than innovating.
  • by jiveturkey on 6/14/24, 11:31 PM

    > Everyone and their mother is running a shoddy microservice-coupled stack,

    buried the lede!

  • by syngrog66 on 6/14/24, 3:42 PM

    Up my alley. I'm the author of a FOSS Golang span instrumentation library for latency (LatLearn in my GitHub.) And part of the team that back in 2006/2007 made an in-house distributed tracing solution for Orbitz.