Issue 15 - The world is small
Reading
Like everyone, I have many anecdotes about how small the world is.
Randomly meeting a friend in a foreign country.
A photo in my furnished flat that represents the friend of a colleague I don’t know.
A girl who was a vague acquaintance back in college, becoming my girlfriend several years later, etc…
Each of us has immediate neighbors, close friends, and family. We also have longer-distance relationships, where distance is either geographical or organizational.
Those connections can be seen as “blocks” with equal distances from each other.
Like in Steinberg’s illustration: the distance from Ninth to Tenth Avenue is the same as the one from the Hudson River to the Pacific Ocean.
These blocks transform exponential relations into linear ones.
We can ask ourselves how these small worlds are constructed and navigated?
What enables us to move from block to block?
📡 Expected Contents
Positive & Negative Engineering
Prefect has written a great article on the different sides of engineering :
Positive data engineering is what we typically think engineers do: write code to achieve an objective.
Negative data engineering is when engineers write defensive code to make sure the positive code actually runs. For example: what happens if data arrives malformed? What if the database goes down? What if the computer running the code fails? What if the code succeeds but the computer fails before it can report the success? Negative engineering is characterized by needing to anticipate this infinity of possible failures.
Jeremiah Lowin, Prefect Founder & CEO, is a former PMC member of Apache Airflow. When he reached Airflow’s limits, he started designing Prefect.
We often concentrate our effort on negative engineering - dealing with code legacy or external system interfaces.
Still, there are not always complex, or sophisticated, or difficult. On the contrary: they are often minor, annoying, and repetitive. Their aggregate impact is extraordinary but we don’t fix them quickly. We prefer to move on to “major” problems.
Negative engineering can be solved in several ways1. The most cost-effective is probably: decoupling things into small pieces. In doing so we also answer positive engineering.
Not a magic sauce, but there is always something to get from building basic - independent - building blocks.
Building an Analytics API with GraphQL: The Next Level of Data Engineering?
Front-end analytics is most of the time a mess.
A bunch of dashboards, reports, and spreadsheets - all with different data sources and definitions.
Recently we see the advent of the “metric layer”. A single place where we expose metrics that can be consumed safely. With consistency.
This post from Simon, go through the technical implementation of such a layer, thanks to GraphQL.
Essentially, GraphQL is SQL on steroids for APIs.
It helps to query several endpoints at one time and get one response.
Check out the full blog post. Really worth the read!
A detailed guide to colors in data vis style guides
This quite long guide is a huge resource for creating data visualization style guides.
Nothing to add - it’s definitely stuff to save and read regularly 📌.
New York’s Shadow Transit
I discovered recently what’s called “dollar vans”.
Those are New York’s unofficial shuttles, in some neighborhoods, making up a thriving transportation system that operates where the subway and buses don’t.
The New Yorker writes a great piece, with maps (❤️) explaining the ins and outs of this smart, still unofficial system.
Bionic Reading
Bionic Reading is a new method facilitating the reading process by guiding the eyes through text with artificial fixation points. As a result, the reader is only focusing on the highlighted initial letters and lets the brain center complete the word. In a digital world dominated by shallow forms of reading, Bionic Reading aims to encourage a more in-depth reading and understanding of written content.
Definitely, something to look at 👀 (I won’t write the next issues in bionic reading mode, but hopefully we will see options on the web to toggle such a feature).
📰 The Blog Post
No new blog post this week 😢.
I really encourage you to subscribe to Medium. It’s only 50$ a year. Definitely worth it2.
Anyway, if you just want to read some of my previous articles :
🎨 Beyond The Bracket
I have recently been a huge fan of DALL·E 2 image generation.
DALL·E 2 is a new AI system that can create realistic images and art from a description in natural language.
Much more than creating images, it can combine concepts, attributes, and styles. It can make realistic edits to existing images from a natural language caption. It can add and remove elements while taking shadows, reflections, and textures into account.
Just look at those images and corresponding natural language inputs :
This is not a montage made by a human. It’s computer-generated.
Look at those examples
Those images speak for themselves… and this is only 2022 state-of-the-art image generation. Imagine what would be possible in 20 years. In a century…
If you wonder about harmful generations, limitations, and capabilities, Open AI teams already have policies to prevent photorealistic generations of real individual faces, violent, adult, or political content.
Should we trust such policies? They definitely deserve to exist. Still, we can be sure that other people could use this kind of technology in other ways…
That’s the price of new technology isn’t?
Update: Google Research, Brain Team just released some related work with their ImageGen AI system.
May is already over? The pace was definitely high. But it was great!
Closing with a quote from my peripheral brain 🧠 :
Every person you meet knows an amazing lot about something you know virtually nothing about. Your job is to discover what it is, and it won’t be obvious.
See you in the summer ☀️.
Chaos Engineering is also a good second-order solution. By deliberately injecting failure, and disabling systems we learn about the weaknesses of our system, so we can build automatic recovery mechanisms to deal with them.
I’m a subscriber for 3 years in a row now and it’s definitely the best 50 bucks I spend every year.