Citus 10.2 is out! 10.2 brings you new columnar & time series features—and is ready to support Postgres 14. Read the new Citus 10.2 blog.
Being a mobile app developer or a content publisher today is extremely challenging. Developers have to compete for attention amongst millions of other apps on the market and many that do the same thing. Of course, media companies have struggled for much longer. Trying to attract eyeballs in a world where readers have virtually unlimited access to free content online is brutal.
To solve this difficult circumstance for both mobile app developers and content publishers, Kixer connects the two. Kixer’s app recommendation engine helps mobile app developers to promote their apps to a highly engaged audience on premium content publishers’ websites and mobile apps. Developers only pay for results by installs, not clicks.
On the app developer side, Kixer counts top companies like Amazon, Experian and Credit Karma as advertisers. On the channel side, companies like Tronc (LA Times and Chicago Tribune), Daily Mail and Fox Sports are premium content publishers.
As a critical tool for mobile app developers to generate new users and content publishers to earn more revenue, supplying up-to-date and accurate analytics is super important.
Kixer works with thousands of web publishers and app developer accounts. At the peak, there can be up to 100 concurrent users accessing analytical reports.
“The data flowing into our system is on the order of a billion events at a terabyte of data per day,” said Cody Koeninger, Chief Architect, Kixer.
Eventually, it was clear that Kixer needed a new database solution to handle the scaling limits of their existing single Amazon RDS database instance. “We were approaching the limits of what could be stored in RDS, even with a read replica,” said Koeninger.
Settling on a database solution was a critical decision for Kixer. Although it wouldn’t be the data store used for serving live advertising traffic, it would be used for scaling analytics and monitoring of business performance. Also as the system of record for financial accounting, reliability was key.
Here’s a talk Cody gave at Spark Summit this year, where he mentions using Citus:
Because Citus is open source and the metadata is accessible in normal Postgres tables, we were able to integrate it with our existing Spark infrastructure in a straightforward way.Cody Koeninger, Chief Architect, Kixer
There were a number of deciding factors for Kixer to choose Citus to solve their database scaling needs.
Firstly, Citus offered massive scale out capability for both reads and writes. Secondly, being able to continue using Postgres made the team feel more comfortable from a reliability and operational perspective. Thirdly, having a reasonable integration path for Spark was also important. Lastly, Citus’ distribution and scaling model was fundamentally sound and clearly documented.
Moving from RDS Postgres to Citus, we were able to linearly scale up write throughput. Our read latency remains good, with most queries averaging 1 millisecond of execution time or under.Cody Koeninger, Chief Architect, Kixer
Transitioning a critical part of infrastructure like a database is a serious task. But Koeninger explains the process was simple.
“Because Citus is open source and the metadata is accessible in normal Postgres tables, we were able to integrate it with our existing Spark infrastructure in a straightforward way. We’ve open sourced the custom Spark partitioner we developed in order to write data directly to Citus workers while maintaining transactional semantics.”
After moving to Citus, Kixer has experienced some impressive results. The database is able to store aggregate analytics data in tables with several billion rows. Peak read transactions on the Citus worker nodes is around 3,000 per second.
“Moving from RDS Postgres to Citus, we were able to linearly scale up write throughput. Our read latency remains good, with most queries averaging 1 millisecond of execution time or under,” said Koeninger.
With Citus, Kixer plans to provide faster and more detailed analytics to customers.
But the most memorable impact for Kixer is saving Christmas.
The biggest impact so far has been availability. We hit hard size limits on RDS sooner than anticipated, on Christmas morning in fact. Thankfully we already had a Citus system developed and in final testing. We were able to promote it to production with minimal downtime and zero loss of data.Cody Koeninger, Chief Architect, Kixer
Cody Koeninger, Chief Architect, Kixer
Because Citus is open source and the metadata is accessible in normal Postgres tables, we were able to integrate it with our existing Spark infrastructure in a straightforward way.
Kixer created a leading advertising technology platform focused on optimizing and driving new mobile revenue streams for content publishers. Kixer’s app recommendation engine helps mobile app developers to promote their apps to a highly engaged audience on premium content publishers’ websites and mobile apps, across a portfolio of over 400 web publications.