Homepage
Sign in
Get started
Towards Data Engineering
Navigating the Path to Data Engineering Excellence
About
Follow
Trending
Fail Fast or Quarantine? Two Data Quality Patterns Every Spark Engineer Should Know
Fail Fast or Quarantine? Two Data Quality Patterns Every Spark Engineer Should Know
Learn when to fail fast or quarantine bad data in Spark pipelines.
Marcel Kennert
May 12
15 Common Spark Errors in the Big Data Industry — Causes, Detection & Detailed Fixes
15 Common Spark Errors in the Big Data Industry — Causes, Detection & Detailed Fixes
Apache Spark is widely used for building distributed data processing pipelines, but it frequently encounters operational and runtime…
Solon Das
May 11
APACHE ICEBERG — DEEP DIVE II: HOW READ & WRITE OPERATIONS WORKS
APACHE ICEBERG — DEEP DIVE II: HOW READ & WRITE OPERATIONS WORKS
As a direct continuation, which I discussed architecture of Iceberg layers i want to close this topic with writing and reading operations.
Douglas Souza
Dec 30, 2024
Latest
Say Goodbye to Dirty Data: Build Trustworthy Pipelines with These Pro Tips
Say Goodbye to Dirty Data: Build Trustworthy Pipelines with These Pro Tips
Non-members can access the full article through this Link.
Ritam Mukherjee
May 21
Accelerate your Analytics Journey from On-Prem to Databricks : Introducing Prism
Accelerate your Analytics Journey from On-Prem to Databricks : Introducing Prism
A new lens on SAP data — clear, intelligent, real-time
THE BRICK LEARNING
May 20
100 Days of Data Engineering with Databricks Day 86: Summarizing Product Reviews Using GenAI and…
100 Days of Data Engineering with Databricks Day 86: Summarizing Product Reviews Using GenAI and…
So far in our SmartRetail 360+ journey, we’ve analyzed structured data from SAP to build ML models for churn, CLV, recommendations, and…
THE BRICK LEARNING
May 20
100 Days of Data Engineering with Databricks Day 85: Introducing Generative AI in Retail Pipelines
100 Days of Data Engineering with Databricks Day 85: Introducing Generative AI in Retail Pipelines
Powered by Databricks ai_query()
THE BRICK LEARNING
May 20
100 Days of Data Engineering Day 83: Calculating Customer Lifetime Value (CLV) Using SAP Customer…
100 Days of Data Engineering Day 83: Calculating Customer Lifetime Value (CLV) Using SAP Customer…
In the previous article (Day 82), we built a churn prediction model to identify which customers were at risk of becoming inactive using…
THE BRICK LEARNING
May 19
SQL: JOINS — BASIC CONCEPTS
SQL: JOINS — BASIC CONCEPTS
Understand the basic concepts of JOINS with SQL.
Douglas Souza
May 19
Databricks Data Engineering Day 82: Predicting Churn from SAP Order Trends
Databricks Data Engineering Day 82: Predicting Churn from SAP Order Trends
We’ve reached one of the most high-impact use cases in the world of retail analytics: Customer Churn Prediction.
THE BRICK LEARNING
May 19
How Data Engineering Powers AI-Based Analytics: Feeding the Intelligence Behind the Magic
Introduction
Naidu Rongali - Senior Big Data and ML Engineer
May 18
WealthVista AI Day 1: Why Wealth Management Needs AI-Driven Intelligence Platforms
WealthVista AI Day 1: Why Wealth Management Needs AI-Driven Intelligence Platforms
The Shifting Landscape of Wealth Management
THE BRICK LEARNING
May 17
Databricks Industry Solutions: Unlocking Industry Innovation with 5 Plug-and-Play Data & AI…
Databricks Industry Solutions: Unlocking Industry Innovation with 5 Plug-and-Play Data & AI…
Databricks is revolutionizing how industries unlock value from their data. At the forefront of this transformation, we are proud to…
THE BRICK LEARNING
May 17
About Towards Data Engineering
Latest Stories
Archive
About Medium
Terms
Privacy
Teams