POLOKA Project Proposal

Abstract

Poloka vision is creation of an open source messaging broker that would act as a building block of a reliable and scalable network of messaging brokers. The network of Poloka brokers will function as fault tolerant, load balanced, and secure message-oriented middleware with Complex Event Processing capability. As an element of a enterprise infrastructure such a network would act as a federated Enterprise Service Bus. As an element of open internet infrastructure Poloka broker network will allow effective information dissemination. Poloka project will focus on reliability, high availability, and high performance while providing transparent support for content-based and topic-based publish/subscribe semantics. Our initial goal is to implement a standalone reference implementation of the WS-BaseNotification, WS-Topics and the WS-BrokeredNotification standards. In the future we would like to develop federated broker capability for WS-Eventing, AMQP, JMS, and other messaging protocols and standards.

Proposal

Poloka will provide a reference implementation of WS-BaseNotification, WS-Topics and WS-BrokeredNotification standards. The project will provide a clear separation of functionality required by the existing standards from additional functionality provided as an enhancement and elaboration on the standards

Poloka will:
1. not be tightly coupled with WS-RF and WSDM standards
1. implement the WS-BrokeredNotification specification absent in the Apache Muse project
1. will implement a network of brokers that will provide scalability, reliability and fault tolerance
1. feed implementation and design experiences into the OASIS standards process and might lead to new revisions of the WS-Notification stack of standards

Background

Poloka is a second generation of the research project going on for the last five years at the University of Toronto’s Middleware Systems Research Group under the PADRES project. During these years the group developed a stable code base that will form a base for the first release of Poloka.

The PADRES project has developed a number of new technologies that allow for a scalable and reliable federation of publish/subscribe brokers. The PADRES federation mechanisms allow for redundant message routing, load balancing, complex event processing and other useful features that become available to the federated network of the WS-BrokeredNotification brokers.

The project was started before WS-BrokeredNotification standard was created. The initial release of Poloka will contain existing PADRES code that does not use any WS-* compliant interfaces. However that will change with the future releases. PADRES brokers will be enriched with the WS--Notification interfaces on the notification producer and notification consumer side. The Broker-to-Broker communication side of communication is not covered by any existing standards to date and it would remain non-WS based.

The team engaged in the development of Poloka involves two former participants of the OASIS committee that produced WS-BaseNotification, WS-Topics and WS-BrokeredNotification standards. Their engagement would serve dual purposes: to serve as source if first-hand authoritative knowledge of the standards and as a conduit for the standards use experience that might result in future evolution of the WS-BaseNotification, WS-Topics and WS-BrokeredNotification standards.

A note about the project name; we are inspired by the impact WikiWiki made on the world of web applications. We followed the naming exampled set by author of the Wiki and simply translated word “Broker” using a Hawaiian dictionary. Poloka simply means “Broker”

Rationale

Current adoption of the WS-Notification set of standard is impeded by absence of a standalone reference implementation. For example WS-Notification implementation in Apache Muse is incomplete and tightly coupled with programming model of WS-RF specification. It is also packaged within WSDM implementation. Poloka aims to provide a “clean” implementation not tied to other standards as a part of WS-Commons.

The PADRES group have over the years received a number of requests to release existing code into open source. We are looking forward to Apache foundation and its organizing principles as a vehicle for creation of a successful community that would connecting all people already interested in the release of the code into open source.

The project will follow new standards and specifications that are coming up in the space of web-services based messaging.

Initial Goals

1. Initial release of the PADRES codebase to Apache Poloka
1. Implementation of the WS-Base Notification, WS-Topics and WS-Brokered Notification interfaces
1. Creation of a reference implementation of the standards with non-federated brokers
1. Implement Poloka federation based on the functions of the PADRES brokers

Current Status

The PADRES project has developed a stable code base and a number of systems are built using this code base. These systems are deployed on PlanetLab and within the PADRES lab infrastructure (cluster with 25+ machines). Future deployments on the Amazon Computing Cloud are under investigation. This code will become a base for the initial release of Poloka. The team conducted a survey of licensed code used within the existing code base. The code base can be brought up in compliance with Apache licensing policy without much effort. One of the members of our team is already working on the implementation of the WS-BaseNotification, WS-Topics and WS-BrokeredNotification interfaces. Remaining team members are preparing for the code release.

There is one patent application submitted to the US patent office that covers use of the federated publish/subscribe infrastructure for Enterprise Job Scheduling. The team is currently working with the respective IP owners on the patent to ensure compliance with the Apache licensing policy.

Meritocracy

The project was originally started in 2002 by Prof. Hans-Arno Jacobsen and Serge Mankovski under sponsorship of CA Inc (formerly Computer Associates), Ontario Centers of Excellence (formerly known as CITO) and the National Science and Engineering Research Council of Canada. Since then the project is growing and it became known and respected in the area of distributed event processing and messaging middleware.

PADRES code base has been developed through constant interaction and continuous integration. The codebase is open to everybody in the project and everyone can potentially make changes to everybody else's code. In fact this is happening a lot when a student is preparing to graduate or past graduation. We expect that there will be more of that once the project is open sourced. The students would charge ahead working on their ideas. This code will be visible to the community and anybody would have ability to contribute to that direction if it makes sense to them.

Community

Over the years a number of companies expressed interest in this work. At the moment project research activities are conducted in collaboration with CA Inc (formerly Computer Associates), IBM, Sun, and Bell Canada, and the Chinese Academy of Science. The project is managed and organized within the Middleware Systems Research Group at the University of Toronto.

The community is built around the papers and publications produced by the research community and also by conferences and workshops where the project is presented.

Domain Experts

  • Dr. Hans-Arno Jacobsen is the head of the Middleware Systems Research Group and he is leading all current research activities.
  • Serge Mankovski is mentoring the project on issues of Network and Systems Management, Security, Governance and Complex Event Processing.
  • Mark Weitzel provides guidance to make sure that the output is appropriate for the industrial participants.
  • Peter Niblett is one of the editors of the WS-BaseNotification, WS-Topics and WS-BrokeredNotification standards committee and mentoring the team on these technologies.
  • Dr. Kirk Wilson is a member of the WS-BaseNotification, WS-Topics and WS-BrokeredNotification standards committee and mentoring the team on these technologies.

Core Developers

  • Dr. Balasubramaneyam Maniymaran is developing mechanisms for Complex Event Processing using the PADRES system
  • Vinod Muthusamy is developing publish/subscribe matching algorithms and routing protocls,
  • Alex Cheung is developing load balancing and scalability features of the broker federation.
  • Guoli Li is developing composite subscription matching, message routing and historic data access
  • Reza Sherafat is developing reliability, scalability, fault tolerance and self-healing features of the broker federation
  • Naweed Tajuddin is developing WS-BaseNotification, WS-Topics and WS-BrokeredNotification interfaces.
  • Dr. Songlin Hu is developing business process automation solutions using the PADRES system

Alignments

Apache goals, mode of operation and development focus of the WS-Commons project provide an attractive framework for the inevitable release of the existing code base to Open Source communities. We are planning to use Apache Axis2 code as the base for our WS-enabled broker. The existing team of core developers and project supporters include representatives of several companies and Apache license is conducive to their future engagement in the project. There is an existing contact and working relationships with the Apache Muse project as well.

Known Risks

Orphaned Projects

Initial contributors are from major software companies and from the university research community. The development of the related code base has been steadily growing over the last five years and there is low risk of it being abandoned. There are a number of requests from various individuals for the Open Source code developed by the project. We expect that these individuals will join this project once it becomes publicly available. Current interest in the project gives us confidence that the community will be able to broaden the committer base in the future. Inexperience with Open Source

The core development team has varied degrees of experience with open source. Some of the developers released open source code in the past. We are hoping that Apache PMC and Mentors will help us to get us started and “hold our hand” until we gained sufficient experience.

The Middleware Systems Research Group has released other open source projects, such as the AspectC project (about 3500 downloads to date over a period of 2 years and 8 releases.) AspetC.net is released under GPL (it is based on GPLed components.)

Homogeneous Developers

Most of the current core developers do not work for a commercial entity. They are working on the project because it is a subject of their research and will result in PhD and Master Degrees at some point in the future. Other core developers are working for CA and IBM and the Chinese Academy of Sciences.

The team is collecting a list of persons and institutions that expressed interest in open source implantation of PADRES over the years. We expect that this list would give us a good start in building the community during the incubation period.

Documentation

Available in form of publications here http://padres.msrg.toronto.edu/Padres/

Initial Source

Initial source is currently in the closed source state.

Source and Intellectual Property Submission Plan

The project is currently conducted under terms of a collaboration agreement between the University of Toronto, CA Canada Company, Sun Microsystems, and the Ontario Centers of Excellence, and includes a clause to permit the release of the project IP into open source under the Eclipse license. However, under current considerations, we think that the Apache community is more closely aligned with the needs and goals of the project. Since the Apache license offers very similar IP terms, it would be acceptable to release the IP under the Apache license. The team will produce a Certificate of Originality in respect to all the code released to Apache.

External Dependencies

Currently PADRES code relies on Jess expert system shell distributed under a commercial license by Sandia National Labs. The team will remove this dependency before the initial release into incubation by implementing our own version of RETE matching engine.

Other code dependencies are:

  • openjms.jar Exolab License
  • exolabcore 0.3.5.jar: Exolab license
  • j2ee.jar: Apache license v2.0
  • commons-collections.jar: Apache license v2.0 It uses jung-1.7.6.ja
  • concurrent.jar: SUN license It uses jung-1.7.6.jar.
  • junit.jar: Common Public License v1.0
  • log4j-1.2.13.jar: Apache license v2.0
  • mysql-connector.jar: GNU General Public license
  • pg73jdbc3.jar: BSD license

Dependency on Exolab, GNU, BSD and Sun licensed code is localized and non-essential.

Cryptography

Some of the aspects of secure message transport include message encryption

Required Resources

  • Mailing Lists – there are two mailing lists
  • Subversion Directory – it exists, but it is not available outside of the Middleware Systems Research Group
  • Issue Tracking – it exists, but it is not available outside of the Middleware Research Systems Group
  • Other Resources - Poloka Wiki

Initial Committers

  • Arno Jacobsen
  • Serge Mankovski
  • NaweedTajuddin
  • Balasubramaneyam Maniymaran
  • Vinod Muthusamy
  • Alex Cheung
  • Guoli Li
  • Reza Sherafat

Affiliations

  • Hans-Arno Jacobsen – University of Toronto
  • Serge Mankovski – CA Labs
  • Kirk Wilson – CA Labs
  • Peter Niblett – IBM
  • Mark Weitzel - IBM
  • Naweed Tajuddin – University of Toronto
  • Balasubramaneyam Maniymaran - University of Toronto
  • Vinod Muthusamy - University of Toronto
  • Alex Cheung - University of Toronto
  • Guoli Li - University of Toronto
  • Reza Sherafat - University of Toronto
  • Shuang Hou – Academy of Science of China


Sponsors

  • Champion TBD
  • Nominated Mentors TBD
  • Sponsoring Entity TBD, but we think we should be somewhere in WS-Commons
  • No labels