Issue 41 - What's your weekend plan?
Escaping strategies for vendors lock-in
What's the smartest people do on the weekend is what everyone else will do during the week in ten years
Removing the "smartest people" part, it resonates with how I operated on my weekends during the last few years.
Back then, I used to work on geeky projects over Sunday. It was all about football analytics. The backend dream was to reach a position in a football club, but you know, it was just a dream. I actually never thought that it would happen four years after the first line of code. I was enjoying my learning and applying them using football analytics as an excuse to do nerd stuff.
In that sense, the weekend version of myself was doing what I would do during the week at Olympique De Marseille.
Today, my nerd stuff over the weekend is about writing, exploring how product distribution works, and coding some CLI in Go1.
What's the back dream of this? I'm not sure. Probably that trendy solopreneur, CEO-inspired stuff.
What I'm sure, is that I love what I'm doing right now. I don't have any plans. I just craft. For the pleasure of it. I can't imagine where I would be in 3 years from now. But my gut feeling urges me to continue.
So let's craft 😉
📡 Expected Contents
The Configuration Complexity Clock
I’m not saying that it’s never appropriate to implement complex configuration, a rules-engine or a DSL, Indeed I would jump at the chance of building a DSL given the right requirements, but I am saying that you should understand the implications and recognise where you are on the clock before you go down that route.
Nice thinking about when we need a DSL, and when just a simple boiler code might be enough...
In some senses, it resonates with my recent writing calling for what's next after SQL.2
I do feel we're not using the best language for analytics. It might be the time to move to the next quarter...
The Rise of the Declarative Data Stack
I should stop writing on this subject 😅 But this one from Simon couldn't be in this newsletter edition.
A declarative data stack is a set of tools and, precisely, its configs can be thought of as a single function such as
run_stack(serve(transform(ingest)))that can recreate the entire data stack.Instead of having one framework for one piece, we want a combination of multiple tools combined into a single declarative data stack. Like the Modern Data Stack, but integrated the way Kubernetes integrates all infrastructure into a single deployment, like YAML.
Simon has again the good words to explain where we aim at (personally speaking, it's my day-to-day at Kestra), and especially why we need to move on.
The Revolving Door of BI
Since data modeling issues are relatively invisible to the business, it’s nearly impossible to convince business teams to prioritize improvements until the system starts breaking down. Many engineers can relate if they’ve ever tried to advocate for refactoring their code. It’s an invisible problem, that no one cares about until they need to care. And if you’re at the point where you need to care, then you’re already in hot water.
In the context of data, problems often manifest at the BI layer, even though the root cause lies deeper in the stack. This is where the business feels the impact.
As data teams come under pressure to address the problems, there’s a natural tendency to focus on the BI platform. Replace the BI tool, solve the problem. It’s a win-win for data teams. Not only can they avoid the assumption of responsibility, but a migration to a new BI tool also presents an opportunity to refactor some of the messy data models as part of the effort to implement a new BI platform — a clean slate. There’s opportunity to deprecate legacy metrics (like “total desks” in the WeWork example), and start fresh with up-to-date requirements and cleaned up data models to match.
The problem is solved for the short term, the data team is the hero, and the cycle begins all over again
Nice post explaining how much BI is trailing when it comes to engineering and especially sustainability. I identify my past self a lot, doing dashboard and asking for more modeling upfront 😅
Clock speed
Great analogy about CPU clock speed and the pace of innovation in our companies.
📰 The Blog Post
Last month I cross-posted a blog with Julien. We explored the escaping strategies for vendor lock-in.
Like everywhere, we should find balance. It's actually not realistic and optimal to escape a vendor lock-in. Sometimes (most of the time?) we just want to get things done. Not using a framework that will create more legacy than outputs.
We actually want to write as little code as possible.
🎨 Beyond The Bracket
Last month I played golf for the first time. I loved it!
The sensation of a good drive while walking in nature is the liminal space I'm constantly looking for.
It also reminds me of how I enjoyed mini-golf when I was a kid. And how much I found joy in my daily work. Even on the bad days.
November hits hard. Christmas time is calling. The night is here.
I hope you're doing great! You're more than 1400 reading these lines - all over the world.
I would love to hear from you. Your recent story. Data or not :)
though most of my weekends are about family, friends, walking in nature, sport, food, reading, etc. I still found time to do nerdy stuff from time to time.
I'm giving a talk on this exact subject at the Forward Data Conference, happening at the end of the month.