[cgl_discussion] Event Notification API proposal

Joe DiMartino joe at osdl.org
Fri Dec 20 15:18:25 PST 2002

This is a request for comments on a proposed API for Event Notification
in a cluster.  This draft was written for the Open Cluster Framework
(www.opencf.org) and recently sent to the OCF mailing list for comments.

The intended use for the event notification API is to distribute cluster
events to interested software components and/or applications.  Some
examples of the types of events are cluster connectivity and cluster
node membership.

In addition to the proposed draft, I have included the introductory
mail that was sent to the OCF mailing list.  The introduction may be
a bit out of context for this CGL mailing list, but I hope more
information is better than less.

Any comments or questions will gladly be fielded.

Happy Holidays!
- Joe DiMartino <joe at osdl.org>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: intro-0.4pre
Type: text/x-troff-man
Size: 3155 bytes
Url : http://lists.linux-foundation.org/pipermail/cgl_discussion/attachments/20021220/4fe3e908/intro-0-0001.bin
-------------- next part --------------


		Draft version pre-0.4

1. Introduction

	The Cluster Event Notification API defines an asynchronous
	delivery framework for cluster events.  This document describes
	the OCF Cluster Event Notification framework.

	The event service uses a publish/subscribe model, where one
	or more publishers can post events that will be delivered to
	all subscribers interested in (subscribed to) events about
	certain topics.

	In this model, event data that passes through the event service
	is opaque to the event service itself and only makes sense in
	the context of a particular event topic.

	Event topics are administratively created.  Some topics will
	be pre-created to align with common architectural components
	of a cluster.

2.  Overview

	In the picture below, the cluster software components on the
	left are responsible for generating events and subsequently
	publishing these events to the event service for further
	distribution to all subscribers.  A subscriber can be either
	an individual application or a set of topic specific data

                      |                          |
                      V                          |
        +----------------+       +-----------+   |
        |                |------>|p         s|---+
        | group comms    |------>|u         u|
        |   +------------+       |b         b|------>  to subscribers
        |   | +----------+       |l  event  s|------>  ....
        |   | |membership|------>|i service c|
        +---+ +----------+       |s         r|---------------+
        +----------------+       |h         i|               |
        |  connectivity  |------>|          b|---+           |
        +----------------+       +-----------+   |           |
                                                 |           |
                                        ---------+           |
                                        |                    |
                                        V                    V
                                  +------------+      +------------+
                                  |connectivity|      | membership |
                                  | event      |      | event      |
                                  | filters    |      | filters    |
                                  +------------+      +------------+

	Data passing through the event service is opaque to the event

3.  Event delivery API

	This section describes the API for cluster event notification
	service.  Unless otherwise specified, errors are indicated by
	non-zero return values.  Errors with specific meanings are
	listed below.  Common errors are listed here once for brevity.

		EPERM	need suitable privileges to subscribe

		ENOMEM	insufficient memory to complete request

3.0  Event Message Format

	All events share a common message format that includes an
	event header, some event properties and the actual event data.

	Although each event topic uses its own data format, each supply
	a common event header that describes the event.  These are the
	components of an oc_ev_event_t.

	An Event Message:
		size	int32	size in bytes of this event message,
				including the event header, and all
				event data

		offset	int32	the offset in bytes from the beginning
				of the event message to the event data

		ev_id	int32	event id

		re_id	int32	reference id that links this message to
				a particular event id

		seq	int32	sequence number for this message in
				this topic

		topic	string	the topic name for reference

		<<<XXX add other header fields here>>>

		data	bytes	topic specific data, format specified

	Each event topic has a corresponding data format.  All events
	in a given topic supply data in the appropriate format.

3.1  Event service subscription

	int oc_ev_subscribe(char *topic, const oc_ev_on_event_t *fn);

	topic	the topic to associate with 'fn'

	fn	is the on_event function (see description below).
		If NULL, must use oc_ev_get_event().

	Specific errno values:
		EINVAL	specified topic is invalid

	This is the initial call to subscribe for notification of
	events in any topic.  A separate subscription is required
	to enable notification for events in each topic of interest.
	Multiple topics can be active at the same time.  Any limit
	on concurrent open topics is imposed by an implementation,
	not by this specification.

	Events are delivered only for event topics with an active
	subscription.  A subscription can be termintated at any time
	[see oc_ev_unsubscribe()].

	A process can choose between two event delivery mechanisms.
	When an "on_event" function is supplied, an asynchronous
	callback method will be used.  When the "on_event" function
	is NULL, a subscriber is responsible for fetching events on
	their own [see oc_ev_get_event()].

	Upon success, callers receive an event descriptor that must
	be passed into subsequent event service calls.  At user-level,
	the event descriptor can be converted into a file descriptor
	suitable for poll or select [see oc_ev_get_fd()].

	Failure returns a -1 and sets errno.

3.1.1  on_event Callback Function

	typedef oc_ev_on_event_t void fn(const oc_ev_event_t *event);

	event	The event data varies based on the event topic.
		Events are allocated by the event service and are
		valid until oc_ev_event_done() is called.  Delivery
		of subsequent events may be inhibited while another
		event is outstanding.

	An on_event function is optionally passed into oc_ev_subscribe()
	to indicate the use of an automatic callback.

3.2  Event service termination

	int oc_ev_unsubscribe(int ed);

	ed	is the event service descriptor obtained
		from call to oc_ev_subscribe().

	Specific errors:
		EINVAL	ed not recognized by event service

	Calling oc_ev_unsubscribe() will cancel the subscription to
	for this event descriptor.  This routine can be safely called
	from an on_event routine.

	Upon successful return, no further events will be delivered.  If
	called from an on_event routine, cleanup occurs after on_event
	completion is indicated by a call to oc_ev_event_done().

3.3  Converting an event descriptor to a file descriptor

	int oc_ev_get_fd(int ed);

	ed	is the event service descriptor obtained
		from call to oc_ev_subscribe().

	Specific errors:
		EINVAL	ed not recognized by event service

	When a select/poll is required, an event descriptor can be
	converted into a file descriptor.

3.4  Automatic Event Delivery

	int oc_ev_handle_events(void);

	Specific errors:
		ENOENT	on_event function not specified

	After subscribing with on_event functions to all desired topics,
	this function will pass control of the calling thread to the
	event service.  The event service will monitor the subscribed
	event topics and call the appropriate on_event function.

	This function will return when there are no more subscribed topics.

	NOTE: This function does nothing for kernel clients.

3.5  Manual Event Extraction

	oc_ev_event_t *oc_ev_get_event(int ed, int timeout);

	ed	is the event service descriptor obtained
		from call to oc_ev_subscribe().

	timeout	time to wait (in milliseconds) for a new event
		to arrive.  If the timeout value is negative,
		oc_ev_get_event() will wait forever for a new

	Specific errors:
		EINVAL	ed not recognized by event service

		EEXIST	an on_event function was specified

	As an alternative to using the on_event callback method of
	event delivery, a subscriber may extract events at their
	leisure.  A user-level process determines that an event is
	pending using select/poll on the file descriptor acquired by

	If no event is pending

3.6  Event Completion

	int oc_ev_event_done(const oc_ev_event_t *event);

	event	the event message address passed to an on_event
		callback function or returned from oc_ev_get_event().

	Specific errors:
		EINVAL	address not recognized by event service

	It is necessary to inform the notification service that callback
	processing is complete for each event.  Upon successful return,
	any data associated with this completed callback is no longer

3.7  Version number

	int oc_ev_get_version(oc_ver_t *ver);

	ver	the version number of the service.

	Specific errors:

	This is a synchronous call to return the event notification
	service version number.  It is safe to call anytime.
	(XXX what is an oc_ver_t - i.e., what does version data look like?)

More information about the cgl_discussion mailing list