Issue 2 - Organized crime works quickly because it does not accept inefficiency
Schedule, orchestrate and then trigger.
The last couple of weeks I was wondering about a tweet showing how vaccine data information was distributed in France.
It’s not new that governments are prone to inefficiency. The question of why is quite puzzling…. Anyway, don’t want to get on politics here (part of the answer?).
What’s fascinated me is how large organizations - with all the power and resources in their hand - can’t handle complex distribution schemes in responsive time.
Inertia is a big component here. Still, there are scaled corporations that efficiently disrupt the current state of knowledge or policies to move quickly. Big tech companies. Even mafias or China government in a certain way1…
It is because they do not accept inefficiency?
Expected Contents
The modern data stack
I've been hearing for several months about DBT. This open-source project helps for the transform part in the ETL/ELT paradigm. Plugged with your data-warehouse technology (currently work with Postgres, Big Query, RedShift, Snowflakes), DBT mix the power of SQL and Jinja templating to build strong data workflows.
I took a deep dive into it recently: this is the best tool I have ever used for data transformation.
Using pure SQL with some touches of templating is terrific. You can references data models within other models so building complex flows is very easy.
Spark is dead? I'm looking forward to a big player REX who migrated from a legacy data transformation tool to DBT (which seems to be used more and more these days).
Founder & CEO of Fishtown Analytics - builder of DBT - write a great piece on how data tools and practices evolved and where they can go in the future, especially on the actual weak points. I highly encourage you to look at the DBT blog. If you prefer a video talk, you can also watch his presentation on Youtube.
For those who would like to go deeper:
Gitlab handbook has a part on DBT (+ this handbook is worth the look !).
“Does my startup data team need a data engineer?” from the DBT blog.
Is Airbyte the next tool in the data landscape?
Scrollytelling at its best
The Pudding is by far my favorite digital publication. Their works are top quality. They master storytelling, scrolling, D3.js, and data visualization best practices.
The FAQ and these 3 articles series "How to make dope shit? (part 1 - part 2 - part 3) is worst the look. Kind of Holy Bible for data-visualization practitioners. And yes, they have a GitHub featuring story code sources.
Documentation lovers
80% coding is about reading code. Your code, your teammate's code, stranger's code.
I recently discovered Material MkDocs.
TL;DR; Markdown-powered project documentation.
Markdown is a well-known markup language, often used for documentation and notes purposes. It's easy to learn, easy to use. Still, it's looking a bit raw and not so user-friendly.
With MkDocs you write in markdown but it renders as a proper website, with modern designs and awesome features such as search, content tabs, tables, footnotes, formatting enhancements, etc...
Furthermore, it integrates nicely with GitHub or GitLab CI/CD features, so you can build a documentation website in only a few minutes, updated automatically after each commit.
When we are stuck, reading documentation is usually far away from StackOverflow, asking help from a teammate, or just move on to another easier task...
Documentations are often big and wild. Beginners can be afraid. Experimented craftsmen are often busy and anyway, they know the trick... In both cases, the fear of losing our time through useless code definitions or too complex explanations slows us down to actually read the doc.
Writing good documentation is hard. Still, it's far more efficient to read a document at your convenience than to have to ask or/and explain. Last week I found this system which is worth the read.
MkDocs seems to be a good try to improve overall documentation that is actually read and updated.
A background color that works with black and white elements
We often think about colors when building charts. Still, most of the texts, annotations, and grids are in black or white.
HEX code: #E5E5E3.
This grey perfectly fits both white and black. Even dark greys.
Can't wait to use this trick.
Earth to exoplanet: Hunting for planets with machine learning
I think there is nothing more to say than Google + Kepler satellite data... An old blog post, still a dreamer.
The Blog Post
You don’t need an orchestrator. In fact, you might need one. But look first at cloud provider features: you might already have everything you need.
Beyond The Bracket
If you're always looking for a window position when booking a plane trip, this book is for you!
When we do our shopping or crossing the street to get a new haircut, we don't think about designs, historic movements, and little engineering things that made up our cities.
In The 99% Invisible Cities, Kurt Kohlstedt and Roman Mars guide us to the broad amount of techniques, designs, and social influences coming with the creation and evolution of our daily places.
Do you know how biohazard markings come from an optimal design supported by behavioral studies? Are you wondering why most of the world is driving the right side (and why some countries don't)? Or how engineers design smart - often hidden - solutions to improve city traffics or roadsides?
The 99% Invisible Cities is full of these stories.
I read the 300 pages straight during the last Christmas holidays: the content is clean, well structured. The illustrations of Patrick Vale fit entirely with the book design. Some will love it as a coffee book too. Simply take a few bits from time to time.
If you want to get a pretty good preview of what's inside the book, I encourage you to visit their website and subscribe to their newsletter. The book is the continuity of a project - 99% Invisible - from KALW public radio and the American Institute of Architects in San Francisco.
This is pretty good content out there!
And that’s it! Got plenty of other resources to share, pretty hard to make the selection…
I received some great feedback on the first issue. Thank you all. If you have any subjects or resources you think might worst sharing, drop me an email! I’m always looking for new hidden Internet gems.
Also, don’t hesitate to reach me if you have suggestions, comments or any other thoughts.
Don’t get me wrong here. Big tech companies are not organized crime or some tyranny. I guess.