Logs 2 Tracing

This is just my perspective after using logs for years and finally moving to traces rather unexpectedly (and suddenly). I should also add I’m by no means an expert on tracing but have used logs in some way for years.

So logs, right? Love them or hate them, we need them.

when something goes wrong in prod with no logs/traces, image source here

We place(d) them lovingly within our code, sometimes with little puns on DEBUG log level to make another dev’s day. Then came full-text search tools that allow you QoL on your log searches instead of suffering. Log analysers followed close behind that told you how your application is doing, cool features all over in each one. Fancy log parsing with complexities rivaling interstellar jump calculations were becoming common. Why? All to understand whatever is happening when certain things hit the fan.

Why Tracing Though?

Just take a look at this:

looking complex and crowded, but look closer

Now that looks like a blob of stuff not so different than normal logs but NO. It lists all activity within the code, how long it took and…

making more sense now

…this is all related with each other in a parent-child relationship even within the same service. I can already see an overhead view of some methods taking too long and some problematic designs are more visible this way. All connected under a trace with multiple spans per endpoint I want instrumented.

well

Traces can go as deep in to the code as you decide.

and they are searchable

If you really want, you can throw events to capture specific info about a span…

events: logs who know where they are

…but in most cases, attributes will be more than enough. You can turn these attributes to facets in Datadog to be able to search traces using them.

simple blob reduction

I almost forgot, you can also see the actual problem when things turn sour. All the way from the deepest level you setup tracing for.

clear sight to errors

I still use logs within my code for local tests and haven’t deleted them. Instead, moved them to a method(func) that also sends events to relevant traces. If I also need a specific info, I use it in the log and also add it as a span attribute.

By the way, setting up tracing in code is surprisingly easy as well. For Go, the steps are here

Thanks for reading and see you next time.

--

--

--

Let’s talk devops, automation and architectures, everyday, all day long. https://www.linkedin.com/in/yigitirez/

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Create Your Own Plasma PIC Simulation (With Python)

Plasma PIC simulation of the Two-Stream Instability

How Sensitive Data Discovery Helps Secure PII (Personally Identifiable Information)

Linux Vs Windows10: 8 Reasons Why Linux Is Better

The better way to use Geohashing

Using a Raspberry Pi as HDMI camera

Announcing a Scala DSL for AWS Data Pipeline

The Complete Guide to Rails Internationalization (i18n)

CloudFront Distribution in AWS using Terraform

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Yiğit İrez

Yiğit İrez

Let’s talk devops, automation and architectures, everyday, all day long. https://www.linkedin.com/in/yigitirez/

More from Medium

SRE Principles Part 1

Service Mesh in 2021: the ecosystem is emerging

Newsletter of Carlos Santana — Issue #30

Tracking the efficiency of resource requests on Kubernetes