Data Engineering - Narrator Data Blog

Does Incremental Modeling Truly Save Money? Avoiding Pitfalls for Data Engineers

Discover effective cost-saving strategies for data engineers to optimize incremental data processing. Learn how to overcome challenges and maximize the benefits of this powerful approach.

Data Analysis

The Data Modeling Paradox: Are we sacrificing insights for structure?

While companies heavily rely on data to inform their strategies, the very data models they employ often hinder their ability to ask deep and insightful questions.

Data Engineering

Leaving Data Chaos Behind: A Paradigm Shift towards Activity-based Modeling

Explore a powerful approach that can revolutionize the way you understand and utilize data: modeling data as activities. Activities can bring clarity and organization to your data, allowing you to gain valuable insights and make the most of your data.

Data Analysis

How to keep your data clean with proper alerting from your data system

No data is perfect, and data quality practices and processes are absolutely critical to put in place before any analysis can be performed.

Data Engineering

How to calculate data warehouse cost and prevent it from exploding

A realistic way to estimate warehouse cost and techniques to prevent the warehouse cost from getting out of hand.

Data Engineering

The cost of product analytics data in your data warehouse

Syncing product analytics data into your warehouse is more expensive than you might think. This explains why, and what to do about it.

Data Engineering

Handling slowly changing dimensions in an Activity Schema

The simplicity of modeling slowly changing dimensions in an Activity Schema vs a Star Schema

Data Engineering

Identifying bot traffic with SQL

Systematically identify bot users in your web traffic with a simple SQL query.

Announcements

Activity Schema at coalesce 2021

Ahmed presented the activity schema at dbt's coalesce 2021 conference, and explained why it's a better alternative to a star schema.

SQL

PostgreSQL's missing DateDiff function

DATEDIFF is an extremely useful function for analytical queries. Given two timestamps it'll tell you how many days, weeks, months, hours, etc they are apart. Here we show an implementation of this function for Postgres.

Data Engineering

Shareable data analyses using templates

We've been using shared data analyses in production for three years. Here's how you can create reusable templates for common metrics and analyses.

Data Engineering

How To Do Data Democratization With Flexible Data Modeling

Data democratization usually means simple dashboards. Here's how to safely give stakeholders the ability to ask any question

SQL

Using PostgreSQL as a Data Warehouse

With some tweaking Postgres can be a great data warehouse. Here's how to configure it.

Data Engineering

Unleash your data's full potential with a modern data stack

How to know if you're ready for a modern data platform built on a data warehouse.

Data Engineering

Data Dictionary: A Symptom, not a Solution

Having a data dictionary is really a symptom of a complex data modeling layer. With a simpler layer it's possible to avoid maintaining a dictionary altogether.

SQL

How to generate a series of numbers in SQL

How to generate a sequence of integers for most data warehouses, including Postgres, Redshift, BigQuery, Snowflake, and Azure SQL

Data Engineering

Why you should be using your data warehouse to sync your SaaS tools

The most robust and simple way to sync between production systems is to use your warehouse as the single source of truth

SQL

SQL Hack: How to join tables when the key is buried in a blob of text

The Situation Text blobs are a pain, but they are commonly used with data from email platforms, Salesforce, or dynamic UTMs. In these cases, the JOIN key is probably hidden in a blob of text. For example, say you want to join your users table using the user_id from

SQL

SQL Hack: How to get first value based on time without a JOIN or a Window Function?

This is my favorite trick in SQL! If you are like me and have been working with data for a while, you have googled “how to get the first value without a join or window function” multiple times in your life. We often hate having to write complex queries to