You are browsing a read-only backup copy of Wikitech. The primary site can be found at

Event Platform/Stream Configuration

From Wikitech-static
< Event Platform
Revision as of 20:16, 12 August 2020 by imported>Ottomata
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Stream configuration refers to configuration that distributed producers or consumers of a stream might want, e.g. the sampling rate or the schema title of the events that are allowed in the stream. Stream configuration was originally a requested feature of Event Platform for Product engineers, so they could more easily vary some event stream producer setting without having to do code deploys. It has since become a critical part of Event Platform, used by multiple services.


EventStreamConfig is a MediaWiki extension that implements PHP and HTTP API for requesting stream configuration. Streams configuration entries are declared in the $wgEventStreams global list in mediawiki-config.

This centralized EventStreamConfig is used by several services to automate discovery and configuration of stream producer and consumer clients:

  • EventGate service clusters uses stream config to restrict which types of events are allowed in which streams via tha schema_title setting.
  • The MediaWiki EventLogging extension uses stream config to vary things like event stream sampling rate.
  • The Analytics Cluster uses stream config to automate ingestion of streams into Hive.


See the EventStreamConfig README.