On September 17, the Apache Software Foundation (ASF) voted to graduate Apache Storm to a top-level project (TLP). This represents a major step forward for the project and represents the momentum built by a broad community of developers from not only Hortonworks, but also Yahoo!, Alibaba, Twitter, Microsoft and many other companies.
What is Apache Storm and why is it useful?
Apache Storm is a distributed, fault tolerant, and highly scalable platform for processing streaming data. Storm supports a wide range of use cases, including real-time analytics, machine learning, and continuous computation. It is also extremely fast, with the ability to process over a million records per second per node on a cluster of modest size.
With the explosion of data sources in recent years, many Apache Hadoop users have recognized the necessity to process data in real time while also maintaining traditional batch and interactive data workloads. Apache Storm fills that real-time role and (through YARN, the data operating system of Hadoop) it can be tightly integrated with many of the tools and technologies commonly found in the Hadoop ecosystem.
Apache Storm is deployed in production by hundreds of organizations including Twitter, Yahoo!, Cisco, Spotify, Xerox PARC, and WebMD where it forms the backbone of the company’s real-time data processing architecture. When combined with other Hadoop components and YARN as the architectural center that manages resources across all of those components, Storm represents an integral part of any big data strategy.
What does “graduation” mean for Apache Storm?
Ultimately, an open source project is only as good as the community that supports it. The community improves the quality of the project and adds features needed by end users. The ASF has been established to do exactly this: to set governance frameworks and build communities around open source software projects. Further, the ASF believes that the real life and future of a project lies in the vibrancy of its community of developers and users.
Apache Storm’s graduation is not only an indication of its maturity as a technology, but also of the robust, active community that develops and supports it. Apache Storm’s vibrant community ensures that Storm will continue to evolve to meet the demands of real-time stream processing and computation.