Nils Dijk

Nils Dijk


Senior Postgres and Citus software engineer at Microsoft. Systems to provide social insights at Bottlenose. Dispatching stack at Uber focused on distributed computing. Enjoys making cocktails. Clarified Whisky Sour anyone?

@ThaNODnl thanodnl

Nils Dijk

Diary of an Engineer: Delivering 45x faster percentiles using Postgres, Citus, & t-digest

Written by By Nils Dijk | September 19, 2020 Sep 19, 2020

When working on the internals of Citus, an open source extension to Postgres that transforms Postgres into a distributed database, we often get to talk with customers that have interesting challenges you won't find everywhere. Just a few months back, I encountered an analytics workload that was a really good fit for Citus.

But we had one problem: the percentile calculations on their data (over 300 TB of data) could not meet their SLA of 30 seconds.

To make things worse, the query performance was not even close to the target: the percentile calculations were taking about 6 minutes instead of the required 30 second SLA.

Keep reading
Nils Dijk

Using custom types with Citus & Postgres, from popular hack to transparent feature

Written by By Nils Dijk | June 25, 2020 Jun 25, 2020

Custom types—called user-defined types in the PostgreSQL docs—are a powerful Postgres capability that, just like Postgres extensions, were envisioned from Day One in the original design of Postgres. Published in 1985, the Design of Postgres paper stated the 2nd design goal as: “provide user extendibility for data types, operators and access methods.”

It’s kind of cool that the creators of Postgres laid the foundation for the powerful Postgres extensions of today (like PostGIS for geospatial use cases, Citus for scaling out Postgres horizontally, pg_partman for time-based partitioning, and so many more Postgres extensions) way back in 1985 when the design of Postgres paper was first published.

Keep reading

Page 1 of 1