First caveat: this isn’t a post just for women. This is a post to anyone and everyone who’s asking, wondering, or thinking.

Second caveat: I’m speaking from the perspective of a woman and a data engineer, because that’s what I know. However, the sentiment could certainly apply to endless other…

Early on, software engineers are taught: develop locally, test in staging, deploy to production. What does this mean for analytics?

Photo by Luke Chesser on Unsplash

Development on many analytics teams

This might sound familiar: “Developing locally” meaning running queries in the one data warehouse in hopes of not stalling jobs powering BI reporting; “testing in staging” meaning performing ad-hoc analysis…

Use Grafana on top of the official Apache Airflow image to monitor queue health and much more.

An unsettling yet likely familiar situation: you deployed Airflow successfully, but find yourself constantly refreshing the webserver UI to make sure everything is running smoothly.

You rely on certain alerting tasks to execute upon upstream failures, but if the queue is full and tasks are stalling, how will you be…

Sarah Krasnik

data engineer by day, data blogger / advisor / generally curious data person by night

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store