Apama (software)
Apama is a Complex Event Processing (CEP) and Event Stream Processing (ESP) engine, developed by Software AG. Apama serves as a platform for performing streaming analytics over a range of high volume/low latency inputs and applications, such as IoT devices, financial exchanges, fraud detection, social media and similar.[1] Users can define data patterns to listen for and actions to take when these patterns are found, which are defined in the provided Domain Specific Language called the Event Processing Language (EPL). The core Apama engine is written in C++; the process can also optionally contain a JVM for interacting with user created Java code. Apama focuses on high throughput, low latency and memory efficient performance; used in both Intel benchmarks[2] and smaller machines such as the Raspberry Pi,[3] routers and other Edge/IoT[4] devices. It is particularly noteworthy within the CEP space as being one of the earliest projects, a long term market leader,[5][6][7][8] and innovator of many patents.[9][10][11][12][13][14][15][16]
Developer(s) | Software AG |
---|---|
Stable release | 10.5.0
|
Written in | C++ and Java (programming language) |
Operating system | Cross-platform |
Type | Complex Event Processing, Event Stream Processing, IoT, Data Analytics |
License | Commercial / Freemium |
Website | apamacommunity |
History
Apama Ltd. was founded in 1999 by Dr John Bates, Dr Giles Nelson and Dr Mohamad Afshar, who met while undertaking research at the Cambridge University Department of Computer Science and Technology.
In 2005, Apama Limited was acquired by Progress Software for $25 million.[17]
In 2013, Apama was acquired by Software AG from Progress for an undisclosed amount.[18]
In 2016, a freemium version, Apama Community Edition, was released,[19] alongside supporting forums and GitHub spaces.
Overview
Event Processing Language
Applications for Apama are authored in the Event Processing Language (EPL). EPL contains language features designed purely for Event Driven Programming,[20] including:
- Events - Simple data structures containing fields and actions
- Monitors - Similar to classes in other languages, monitors contain listeners and actions and are loaded upon injection
- Listeners - Declarative patterns defining interesting conditions/combinations of events. Aggregation, temporal and filtering operators are supported
- Actions - Imperative functions, usually run by listeners upon activation or monitors upon load
- Contexts - Similar to threads, monitors exist inside contexts
- Channels - Mechanism for communicating between contexts and external systems
- Streams - Ordered partitions of events with support for aggregation and SQL-like queries
The EPL C++/Java-like syntax is designed to create CEP applications succinctly in conjunction with the above features, an example of which can be found below:
event MyEvent {
integer myInt;
sequence<string> myListOfStrings;
boolean myBool;
}
monitor MyMonitor {
action onload() {
spawn worker() to context("mySideThread");
}
action worker() {
on all MyEvent(myInt>10) as myCoassignedEvent -> MyEvent(myInt>20) or MyEvent(myBool=true) within 30.0 {
if(doWork(myCoassignedEvent.myListOfStrings) != true) {
log "Problem!";
}
}
}
action doWork(sequence<string>) returns boolean {
// do work on the list
return true;
}
}
Apama Queries
Apama Queries is an alternate language for creating rules over defined data sets, such as the past five minutes of received data or past twenty events received, in an automatically multi-threaded environment that scales across machines. Apama Queries are better suited for monitoring very large sets, such as bank transactions. Typically, each entity (i.e. account holder) will be partitioned and processed independently of the rest of the set. Queries and Monitors can communicate via channels. An example of the previous scenario can be found below:
query FindSuspiciousWithdrawals {
parameters {
float DURATION;
float THRESHOLD;
}
inputs {
Withdrawal() key cardNumber within DURATION;
}
find every Withdrawal as w
where w.amount > 100
select last(w.transactionId) as tid
select last(w.cardNumber) as cid
having last(w.amount) > THRESHOLD * avg(w.amount) {
send SuspiciousTransaction(tid, cid) to SuspiciousTransactionManager;
}
}
Designer
The Software AG Designer is an Eclipse based IDE with special support for developing and deploying Apama applications, which is included in a standard installation. Features include code assistance, package management, profiling, GUI development and deployment management.
Plugins
Apama supports the ability for users to create plugins to extend capability. These plugins come in two forms, EPL plugins (to add features to EPL) and Connectivity plugins (for communicating with the outside world). C++ and Java APIs are available for both plugin types. Connectivity plugins come in two forms, codecs (which translate data from one format to another, such as the internal Apama representation to JSON) and transports (which handle sending/receiving to/from external systems, such as JMS). Connections to external systems are made by composing chains of codecs and one transport to achieve the desired topology; these connections are defined by a YAML configuration file at start-up or through dynamic management with the provided tools. Several pre-built plugins for popular technologies (e.g. R, MQTT, Kafka, HTTP, MatLab) as well as connectivity to other Software AG products (e.g. Universal Messaging, Terracotta, Cumulocity) are shipped with Apama installations.
Persistence
The Apama process is entirely in memory and supports an optional SQLite based in-built persistence system. Additionally, the connectivity API supports reliable messaging to systems that support it, such as JMS. Finally, a pre-built distributed memory store plugin is provided with standard installations for use with appropriate back-ends.
Visualization
Apama provides an in-built dashboarding technology developed from within the Software AG Designer, as well as native connections to Software AG's MashZone NextGen.
See also
- Complex event processing (CEP) - A related technology for building and managing event-driven information systems.
- Data Stream Management System (DSMS) - A type of software system for managing and querying data streams
- Event correlation
- Event-driven architecture — (EDA) is a software architecture pattern promoting the production, detection, consumption of, and reaction to events.
- Event stream processing — (ESP) is a related technology that focuses on processing streams of related data.
- Operational intelligence — Both CEP and ESP are technologies that underpin operational intelligence.
- Pattern matching
- Real-time business intelligence — Business Intelligence is the application of knowledge derived from CEP system
References
- "Apama Streaming Analytics & Event Processing Platform". Retrieved 2018-01-16.
- "Apama Streaming Analytics* with the Intel® Xeon® Processor E7 v3". Intel. Retrieved 2018-01-16.
- "Apama Community Edition Apama 9.12 Community Edition release announcement". www.apamacommunity.com. Retrieved 2018-01-16.
- "Software AG introduces Apama EagleEye for an updated, AI-enabled market surveillance solution - IoT Now - How to run an IoT enabled business". IoT Now - How to run an IoT enabled business. 2017-11-23. Retrieved 2018-01-16.
- "Sell-Side Technology Awards 2013: Best Sell-Side Complex-Event Processing (CEP) Technology ─ Progress Apama - WatersTechnology.com". WatersTechnology.com. 2013-06-03. Retrieved 2018-02-08.
- "Magic Quadrant for Intelligent Business Process Management Suites". www.gartner.com. Retrieved 2018-01-19.
- "The Forrester Wave™: Streaming Analytics, Q3 2017". www.forrester.com. Retrieved 2018-01-19.
- "Streaming analytics platforms – Bloor Research". www.bloorresearch.com. Retrieved 2018-01-19.
- , Nelson, Giles & John Bates, "Method and apparatus for evaluating queries against received event information"
- , Mitchell, Robert Scott; Mark K. Horsburgh & Richard M. Bentley, "Complex event processing system having multiple redundant event processing engines"
- , Bates, John; Gareth Smith & Richard M. Bentley, "Event-based process configuration"
- , Bates, John; Gareth Smith & Richard M. Bentley, "Automated construction and deployment of complex event processing applications and business activity monitoring dashboards"
- , Reed, Christopher & Mark Horsburgh, "System and method for managing the allocating and freeing of objects in a multi-threaded system"
- , Smith, Gareth, "Trade surveillance and monitoring systems and/or methods"
- , Smith, Gareth, "Dynamically adaptable real-time customer experience manager and/or associated method"
- , SMITH, Leighton & Gareth Smith, "Large venue surveillance and reaction systems and methods using dynamically analyzed emotional input"
- "Progress Software Announces Acquisition of Apama". Retrieved 2018-01-16.
- "Progress Software Announces Agreement to Sell Apama Solution to Software AG (NASDAQ:PRGS)". investors.progress.com. Archived from the original on 2018-02-09. Retrieved 2018-01-16.
- "Apama Community Edition Introducing Apama Community Edition". www.apamacommunity.com. Retrieved 2018-01-16.
- http://www.apamacommunity.com/wp-content/uploads/2016/06/SAG_The_Apama_Platform_20PG_WP_Jun16-Web.pdf
External links
- Apama Community Edition
- Apama Commercial Edition
- Software AG
- Forrester Wave: Streaming Analytics, 2017Q3
- Disruptive Analytics by Thomas W. Dinsmore, Apress (ISBN 978-1-4842-1312-4), a book about emerging trends in analytics, including Apama
- PrEstoCloud, a research paper featuring Apama