It used to be that if you were a company that’s been accumulating tons of data about your business, it would take a day or two (or longer) to sift through and crunch those numbers to discover ways to run your business more efficiently.
Nowadays, however, as data processing technology has matured, companies are looking to analyze their data in real time as opposed to having to wait days. If you’re a retail company selling shirts and your inventory is going down fast, you want to know what is happening immediately so you can take action. Time is of the essence.
It’s this need for speed that Mountain View, Calif.-based Confluent, a startup specializing in data processing, wants to address, and it plans to announce on Wednesday that it raised $24 million to help do so. With the funding, the startup, founded in September 2014, now has $31 million in total investment.
Confluent’s founders were all engineers from the professional social network LinkedIn. The three co-founders helped LinkedIn build an internal system known as Apache Kafka that managed the 800 billion events the website sees on a given day, explained Jay Kreps, Confluent’s co-founder and CEO. At LinkedIn, each time someone clicks on someone’s profile, shares an article, or likes a post, the action triggers what is known as an event.
These events are essentially data points that need to be constantly updated so that LinkedIn’s website and application doesn’t feel stagnant and inactive. If someone likes an article you just shared, LinkedIn (LNKD) wants you to know that it happened as soon as possible.
Confluent’s business is built around this open-source technology and is free to download.
Kafka helped LinkedIn coordinate all these different data points, explained Kreps, and is wired to work with the latest in a type of data-crunching technology known as stream processing. Instead of data having to be spooled into a data warehouse to be analyzed, like how companies traditionally processed data, stream-processing technology allows for the data to be analyzed almost instantaneously.
These stream-processing technologies like Apache Spark have been gaining a lot of steam as of late for their ability to stream data and automatically run number-crunching queries that data scientists historically had to run on their own when the information was stored in a repository like an Oracle (ORCL) data warehouse.
Kafka coordinates all the activities of stream-processing technologies like Storm or Apache Samza that a company may have up and running in its infrastructure. You can think of stream-processing technologies as being the fire hoses attached to the fire truck that is Kafka.
As Kafka apparently worked well at LinkedIn, Kreps and his team felt they could take the technology and build a business around it for companies looking to improve their data infrastructure.
The company is currently working on making sure Kafka can work with the hundreds of different data systems companies often use throughout their business. Essentially, the Confluent team wants its product to be the data coordinator that helps manage all those disparate systems, whether they be Hadoop, relational databases, or others.
As far as building a business, Confluent is modeling itself after other popular companies like Cloudera that are selling proprietary tools based on open-source software. These include management tools, enterprise support, or security related features, among others, Kreps explained.
There is an opportunity to sell these tools to the current large scale users of Kafka, which include Netflix, Goldman Sachs, Uber, and Cisco, among others.
Building proprietary features based on open-source technology is a familiar strategy that many enterprise startups like Docker are taking nowadays. Kreps said that by being open source, it’s easier for companies to take a chance on the technology, which then provides the opportunity to upsell proprietary features.
“We are replacing the foundation of how data flows,” Kreps said. “It is a big change.”
Index Ventures drove the funding round along with participation from previous investor Benchmark Capital. Index Ventures partner Mike Volpi will join Confluent’s board.
Subscribe to Data Sheet, Coins2Day’s daily newsletter on the business of technology.
For more on data, check out the following Coins2Day video: