If you want to learn more about Citus on Microsoft Azure, read this post about Hyperscale (Citus) on Azure Database for PostgreSQL.

Skip navigation

Citus Blog

Articles tagged: sharding

Craig Kerstiens

Thinking in MapReduce, but with SQL

Written byBy Craig Kerstiens | February 21, 2019Feb 21, 2019

For those considering Citus, if your use case seems like a good fit, we often are willing to spend some time with you to help you get an understanding of the Citus database and what type of performance it can deliver. We commonly do this in a roughly two hour pairing session with one of our engineers. We’ll talk through the schema, load up some data, and run some queries. If we have time at the end it is always fun to load up the same data and queries into single node Postgres and see how we compare. After seeing this for years, I still enjoy seeing performance speed ups of 10 and 20x over a single node database, and in cases as high as 100x.

And the best part is it didn’t take heavy re-architecting of data pipelines. All it takes is just some data modeling, and parallelization with Citus.

Keep reading
Craig Kerstiens

Options for scaling from 1 to 100,000 tenants

Written byBy Craig Kerstiens | June 28, 2018Jun 28, 2018

When you first start out in building a SaaS application you talk about that day in the future when you will have scaling problems, how that’ll be the day, how that would be a good problem to have. You focus on getting the first few customers, making sure they have a great experience, and suddenly you’re at 10s of customers, then 100s. You’ve upgraded your app server to a larger one, then you’ve gone from one ec2 app server to multiple ones with ELB in front of things. You’ve upgraded your Postgres database from an r3.large on AWS, to r3.xlarge, now you’re eyeing that r3.2xlarge next month. In the back of your mind though, you’re starting to look at your plans for future growth of your SaaS app, and you’re wondering how much larger you can keep going. Your database is performing well at 100 tenants (tenants = customers), your back of the napkin math says you’ll be able to scale your app up to 1,000 tenants, but after that you know you’re going to have to explore some options.

What are those options and what are the trade-offs and benefits?

Keep reading

The Citus extension to Postgres allows you to shard your Postgres database across multiple nodes without having to make major changes to your SaaS application. Citus then provides performance improvements (as compared to single-node Postgres) by transforming SQL queries and distributing queries across multiple nodes, thereby parallelizing the workload. This means that a 2 node, 4 core Citus database cluster could perform 4x faster than single node Postgres.

With the Citus shard rebalancer, you can easily scale your database cluster from 2 nodes to 3 nodes or 4 nodes, with no downtime. You simply run the move shard function on the co-location group you want move shards for, and Citus takes care of the rest. When Citus moves shards, it ensures tables that are co-located stay together. This means all of your joins, say, from orders to order_items still work, just as you’d expect.

Keep reading

In both Citus Cloud 2 and in the enterprise edition of Citus 7.1 there was a pretty big update to one of our flagship features—the shard rebalancer. No, I’m not talking about our shard rebalancer visualization that reminds me of the Windows ‘95 disk defrag. (Side-node: At one point I tried to persuade my engineering team to play tetris music in the background while the shard rebalancer UI in Citus Cloud was running. The good news for all of you is that I was overwhelmingly veto'ed by my team. Whew.) The interesting new capability in the Citus database is the online nature of our shard rebalancer.

Keep reading
Craig Kerstiens

Database sharding explained in plain English

Written byBy Craig Kerstiens | January 10, 2018Jan 10, 2018

Sharding is one of those database topics that most developers have a distant understanding of, but the details aren’t always perfectly clear unless you’ve implemented sharding yourself. In building the Citus database (our extension to Postgres that shards the underlying database), we’ve followed a lot of the same principles you’d follow if you were manually sharding Postgres yourself. The main difference of course is that with Citus, we’ve done the heavy lifting to shard Postgres and make it easy to adopt, whereas if you were to shard at the application layer then there’s a good bit of of work needed to re-architect your application.

I’ve found myself explaining how sharding works to many people over the past year and realized it would be useful (and maybe even interesting) to break it down in plain English.

Keep reading
Craig Kerstiens

Five sharding data models and which is right

Written byBy Craig Kerstiens | August 28, 2017Aug 28, 2017

When it comes to scaling your database, there are challenges but the good news is that you have options. The easiest option of course is to scale up your hardware. And when you hit the ceiling on scaling up, you have a few more choices: sharding, deleting swaths of data that you think you might not need in the future, or trying to shrink the problem with microservices.

Deleting portions of your data is simple, if you can afford to do it. Regarding sharding there are a number of approaches and which one is right depends on a number of factors. Here we’ll review a survey of five sharding approaches and dig into what factors guide you to each approach.

Keep reading
Ozgun Erdogan

Principles of Sharding for Relational Databases

Written byBy Ozgun Erdogan | August 9, 2017Aug 9, 2017

When your database is small (10s of GB), it’s easy to throw more hardware at the problem and scale up. As these tables grows however, you need to think about other ways to scale your database.

In one way, sharding is the best way to scale. Sharding enables you to linearly scale your database’s cpu, memory, and disk resources by separating your database into smaller parts. In other ways, sharding is a controversial topic. The internet is full of advice on sharding, from “essential to scaling your database infrastructure” to “why you never want to shard”. So the question is, whose advice should you take?

Keep reading

One of the most common questions we get at Citus is how the rebalancer works–which is understandable. When you have an elastic scaled out database, how easy it is to scale is a key factor into how usable it will be. While we’ll happily take time for anyone that’s interested and live demo it, this walk through should give you a better idea of how it works for those of you that are curious.

Keep reading
Craig Kerstiens

A multi-tenant sharding tutorial

Written byBy Craig Kerstiens | March 9, 2017Mar 9, 2017

A number of SaaS applications have data models where they want to have their customers interact with only their data. At the enterprise end you have companies like Salesforce and Workday that fall into this bucket, but we see a ton of small ones as well. If you’re just getting started figuring out how you should approach your data so it can scale in the future, it doesn’t have to be hard.

Here we’re going to walk through an example data model that you can use as a basis for learning how you could apply the same to your own multi-tenant application.

Keep reading
Craig Kerstiens

Lessons learned from Postgres schema sharding

Written byBy Craig Kerstiens | December 18, 2016Dec 18, 2016

We talk with a number of Postgres users each week that are looking to scale out their database. First, we would never recommend scaling out until you truly have to, it’s always easier to scale your database up rather than out. It’s often not until over 100 GB of data that you need to think about sharding.

When you want to scale out though, you want it to be simple. For scaling a multi-tenant database, there’s three common approaches:

Keep reading

Page 1 of 2