I’ve never used Airflow. I’ve never used Databricks. I’ve never used Apache Kafka. And yet, I’ve built a solid career as a Data Engineer. I constantly see advice saying you must master all. It's wrong. If you want to grow as a Data Engineer, stop asking: 𝘞𝘩𝘢𝘵 𝘵𝘰𝘰𝘭𝘴 𝘴𝘩𝘰𝘶𝘭𝘥 𝘐 𝘭𝘦𝘢𝘳𝘯? Start asking: 𝘞𝘩𝘢𝘵 𝘬𝘪𝘯𝘥 𝘰𝘧 𝘋𝘢𝘵𝘢 𝘌𝘯𝘨𝘪𝘯𝘦𝘦𝘳 𝘥𝘰 𝘐 𝘸𝘢𝘯𝘵 𝘵𝘰 𝘣𝘦? Data Engineering is a broad field. Some are Real-Time / Streaming Engineers: → Kafka → Flink → Distributed systems Some are BI / Analytics Engineers (my case): → dbt → SQL → Snowflake Some Data Engineers work on data platforms: → Docker → Terraform → Kubernetes Once you know what type of Data Engineer you want to be, the path becomes much clearer. That said, some skills are non-negotiable for everyone: • Data modeling • Writing reliable, readable code • Understanding downstream use cases • Thinking in terms of data quality, cost, and scalability Start with the problems you want to solve, then learn the tools that help you solve them well. --- ♻️ Repost if this was useful Follow 👉🏻 José for more about Data Engineering
José Siles Absolutely you just need to know enough to get in the door. Master the essentials you can always add to your knowledge at a later date.
It’s so true, your specific tech stack is largely dependent on the companies you’ve worked for. The tools don’t define your capabilities! Btw, I haven’t used Databricks either!
You're a unique snowflake ❄️ (pun intended) 😅
If you're a Kafka Engineer or a Databricks Engineer, you have an expiration date. If you're a problem solver who understands modeling, cost, and architecture, you're indispensable.
José, exactly. There are many tools, and the right ones depend on the business and the team at that moment. You don’t need to know it all, but curiosity and adaptability matter. And yes, some roles still ask for everything.
Great reminder that tools are secondary, clarity on the kind of Data Engineer you want to be comes first. José Siles
This is same with Analysts I’ve used Python just twice in my career
Nah - it's easy to master them all. There is just a few of them and we obviously use all of them, every day, at any org... ;)