From the course: Complete Guide to Data Lakes and Lakehouses
Join today to access over 25,200 courses taught by industry experts.
Origins and evolution
From the course: Complete Guide to Data Lakes and Lakehouses
Origins and evolution
- Imagine stepping back in time to the early days of the internet, a world blooming with data, but confined by the limited storage and processing power of traditional systems. How did we transition from there to today's sophisticated data lakes? In the late 1990s and early 2000s, the digital universe began to expand at an unprecedented rate. With the rise of the internet, e-commerce, and social media, traditional data storage and processing systems were overwhelmed. The need for a technology that could handle not only the volume, but also the variety and velocity of data became obvious, but there were significant challenges behind it. Traditional databases and data warehouses were structured for consistent, predictable loads of structured data, things like customer names and contact information, financial transactions, and stock information, et cetera. However, the new types of data, like emails, videos, social media posts, sensor data, et cetera were unstructured and didn't really…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.