FONDEF IDeA CA12i10314, 2013 – 2015

Real-time Stream Processing

This project proposes research aimed at building software products for efficient and scalable processing of large event streams in real time. Feasible applications of this technology include domains as diverse as systems for crisis management and finance. In many cases, these applications require efficient parallelization and distribution of multiple tasks to achieve response times of the order of a fraction of a second per event. This as a result of the ever increasing availability of massive event sources such as message channels or geographically distributed sensors. The implementation of these complex systems is very demanding in terms of software development. Consequently, this project takes advantage of an unique opportunity for research collaboration and international visibility from a product called open-source S4 (Simple Scalable Streaming System), which facilitates development of such applications. Our project proposes to extend S4 to deliver better performance and enhance it with additional software tools that represent a clear advantage over other competing products. S4 is currently an incubation project at the Apache Software Foundation, a fact that directs monetization of technology to the open-source industry under Apache licensing. This is a market of billions of dollars that has been growing steadily in the last years. Companies monetize through consulting, training, application development, and services with premium versions of open-source products and/or licensing of related software tools.