Event Platform/Stream Configuration

Stream configuration refers to configuration that distributed producers or consumers of a stream might want, e.g. the sampling rate or the schema title of the events that are allowed in the stream. Stream configuration was originally a requested feature of Event Platform for Product engineers, so they could more easily vary some event stream producer setting without having to do code deploys. It has since become a critical part of Event Platform, used by multiple services.

EventStreamConfig

EventStreamConfig is a MediaWiki extension that implements PHP and HTTP API for requesting stream configuration. Streams configuration entries are declared in the $wgEventStreams global list in mediawiki-config.

This centralized EventStreamConfig is used by several services to automate discovery and configuration of stream producer and consumer clients:

• EventGate service clusters uses stream config to restrict which types of events are allowed in which streams via tha schema_title setting.
• The MediaWiki EventLogging extension uses stream config to vary things like event stream sampling rate.
• The Analytics Cluster uses stream config to automate ingestion of streams into Hive.
• EventStreams uses stream config to discover streams and auto-generate OpenAPI docs.

Usage

See the EventStreamConfig README.