How to Use LLMs as a Weekly Review Partner
Weekly Reviews as Conversations, Not Chores
Jan 26, 202611 min read
Command Palette
Search for a command to run...
Latest articles
Handling Data Redaction in Delta Live Tables for Reliable Data Pipelines 馃敀
In today's rapidly evolving digital landscape, data security and compliance have become paramount. Delta Live Tables (DLT) is a powerful framework primarily used for data ingestion and transformation within the Databricks ecosystem. This article expl...
Apr 14, 20256 min read
Evolution of data platform - Part 3
Part 3: Beyond Analytics
Apr 8, 20233 min read
Data Insights with MindsDB
Recently I was discussing with a colleague that SQL is a pervasive part of a data stack. Their counterargument was SQL has not yet penetrated the ML world. While I was researching that idea, I encountered MindsDB, an AutoML platform that integrates S...
Dec 17, 20224 min read
Evolution of data platform - Part 2
Part 2: Performance, Catalog and Access Control
Jul 7, 20228 min read
Evolution of data platform - Part 1
Part 1: How it began and what we learned
May 6, 20226 min read
12. Future of Data Systems
This section is all about author's speculations about the future of data systems. Martin Klepmann: The goal of data integration is to make sure that data ends up in the right form in all the right places The ability to create derived views of the s...
Apr 23, 20223 min read
11. Stream processing
In previous post we discussed about batch processing. The key assumption in batch processing is that data has a defined start and end. In reality there is no end as data keeps on flowing. Batch works by defining artificial boundaries - end of day, we...
Apr 11, 20226 min read
10. Batch Processing
Broadly data systems can be grouped into two categories Systems of Record : Act as source of truth. In case of conflict, data in this system is considered as correct. Derived System: Derived from Systems of record and generally modeled based on cons...
Mar 29, 20225 min read