- <50ms response time for 99% of queries
- Time to insight reduced from ~24 hours to near real time
- UserIQ supports more than 20M total users and hundreds of thousands of concurrent users today—and can easily scale for future growth
UserIQ helps SaaS businesses improve user adoption by providing tools to analyze user behaviors, simplify user onboarding, and increase in-app engagement.
Today, UserIQ serves a rapidly growing number of businesses and generates insights for more than 20 million users.
Supporting more than 500 million transactions per day from several hundred thousand concurrent users requires a relational database that can handle a demanding read and write workload with high concurrency.
The UserIQ team was using a single-node PostgreSQL database to support its platform. But it was projected that the single-node database would not maintain the required performance level as traffic increased.
“We were approaching the connection limits recommended by our hosted, single-node Postgres provider,” says Chris Morgan, director of software engineering for UserIQ. “Scaling would be challenging without drastically changing how we operate.”
UserIQ was committed to keeping latency low for the campaign delivery engine that supports critical, real-time, day-to-day activities for its customers. So as traffic grew, the UserIQ team had to make some quick decisions.
We had to deliver on our promises. But to consistently provide a full range of capabilities, we needed to solve our database scaling issues.”
The UserIQ team considered multiple approaches to solving its scaling problem, including using a NoSQL database. “There are a number of really good NoSQL databases. But the ones we evaluated were write-optimized, like Cassandra,” says Morgan. “We do use Cassandra to store the massive collection of analytics data, but we needed something that reads and writes faster.”
“We could have sharded the application and spun up databases horizontally, without a distributed architecture. But sharding that way would have required a lot of work—and at some point, that approach would have become unmanageable,” says Morgan. “If you’re going to provide a good product experience in a SaaS offering, a distributed, scale-out approach delivers faster performance and is a more forward-thinking solution.”
A Kafka developer who was friends with the UserIQ team suggested that UserIQ should evaluate Citus—an extension to Postgres that distributes data and queries across multiple nodes. “We discovered a local company that had moved to Citus, and we called the CTO,” says Morgan. “He told us about his positive experience with Citus, so we decided to investigate further.”
“We learned that Citus Data engineers had played key roles in the development of Heroku Postgres as well as the WAL-E continuous archive for Postgres, so we knew the Citus database was built and supported by a strong team,” says Whatley.
With Citus Cloud, we don’t have to worry about database problems. We can think about how we’re going to create new capabilities and offerings for UserIQ.Chris Morgan, Director of Software Engineering, UserIQ
Whatley and Morgan also liked that Citus is open source. “We would be wary of using a proprietary database,” says Morgan. “We don’t want to hitch our cart to a single vendor.”
Citus also has compliance certifications for strict data privacy regulations, such as Service Organization Control 2 (SOC 2 Type 2) and the EU’s General Data Protection Regulation (GDPR). “Citus makes compliance a breeze,” says Morgan. “We can delete a client’s data on request, easily, with a simple API call.”
After thoroughly evaluating Citus in a small development cluster, the UserIQ team decided to move forward. The initial Citus database cluster is a 3-node cluster with 750 GB of distributed data. UserIQ processes approximately 4TB of data per month in Citus and sends the analytic data to a data store for longer-term storage.
Migrating from Heroku Postgres to the Citus Cloud database was easy. “We used the Citus Warp migration feature to move data into Citus without a long maintenance period,” says Whatley.
By adopting Citus to distribute our database, we overcame the constraints of our single-node Postgres deployment—and reduced the time to insight for our customers. Now we can continue to deliver real-time results as we grow which would have been difficult without making the switch to Citus.Chris Morgan, Director of Software Engineering, UserIQ
UserIQ also took advantage of the Citus activerecord-multi-tenant Ruby library, which helps scale out applications that are built on Rails and that follow a multi-tenant data model. “The activerecord-multi-tenant gem made it easier to run our initial create distribute table scripts and record tables,” says Whatley. “And, more than anything else, we were able to complete operational tasks required to support sharding while writing fewer queries.”
Citus Data engineers also helped streamline the move. Using a Slack channel for fast communication and providing on-site assistance kept the project moving. “We find that developer-to-developer communications are often easier via chat rather than email or phone calls,” says Morgan.
“The Citus Data team helped us identify and quickly address potential issues,” says Whatley. “We had white-glove service.”
“Migrating from a single Postgres node to a distributed database might ordinarily take a quarter of a year or more,” says Morgan. “But with assistance from the Citus Data team, we finished the transition even faster than expected.”
We don’t do maintenance periods on our application—we do continuous deployment. Citus Cloud’s zero downtime shard rebalancer lets us avoid long maintenance periods. Having a database that can be updated, rebalanced, or otherwise improved without downtime is ideal.Lance Whatley, Lead Software Engineer, UserIQ
“By adopting Citus to distribute our database, we overcame the constraints of our single-node Postgres deployment—and reduced the time to insight for our customers,” says Morgan. “Now we can continue to deliver real-time results as we grow which would have been difficult without making the switch to Citus.”
For UserIQ, delivering that responsive, low-latency experience and real-time analytics to SaaS businesses is critical. “The faster our customers get their data, the more successful they are,” says Whatley.
“We rebalance twice per month so we can avoid overloading nodes,” says Whatley. “We want to make sure our smaller clients don’t suffer even as our larger clients grow.”
The shard balancer helps the UserIQ developers improve the company’s application without downtime. “We don’t do maintenance periods on our application—we do continuous development,” says Whatley. “The zero downtime shard rebalancer lets us avoid long maintenance periods. Having a database that can be updated, rebalanced, or otherwise improved without any downtime is ideal.”
By moving to Citus, the UserIQ team can spend more time envisioning the future of the company’s platform. says Morgan. “With Citus, we don’t have to worry about database problems. We can think about how we’re going to create new capabilities and offerings.”
UserIQ can continue to expand its clientele and handle rapidly rising transaction volumes. “One of the biggest benefits of a distributed database is that we can add nodes without having to do large database cutovers—that’s huge for us,” says Whatley.
UserIQ is also considering implementing tenant isolation in the future. “We have a few large-scale customers that might require their own nodes—with dedicated resources,” says Whatley. “With Citus, we can make those changes to accommodate client needs easily.”
Chris Morgan, Director of Software Engineering, UserIQ
By adopting Citus to distribute our database, we overcame the constraints of our single-node Postgres deployment—and we reduced the time to insight for our customers. “Now we can continue to deliver real-time results as we grow which would have been difficult without making the switch to Citus.
UserIQ empowers SaaS companies to deliver what each user needs to be successful in every moment, starting with adoption. As a result, you’ll be able to effectively scale onboarding, increase feature usage, accelerate time-to-value, and ultimately drive more revenue throughout the customer journey.