You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
Event Platform/Stream Configuration
Stream configuration refers to configuration that distributed producers or consumers of a stream might want, e.g. the sampling rate or the schema title of the events that are allowed in the stream. Stream configuration was originally a requested feature of Event Platform for Product engineers, so they could more easily vary some event stream producer setting without having to do code deploys. It has since become a critical part of Event Platform, used by multiple services.
EventStreamConfig is a MediaWiki extension that implements PHP and HTTP API for requesting stream configuration. Streams configuration entries are declared in the $wgEventStreams global list in mediawiki-config.
This centralized EventStreamConfig is used by several services to automate discovery and configuration of stream producer and consumer clients:
- EventGate service clusters uses stream config to restrict which types of events are allowed in which streams via tha schema_title setting.
- The MediaWiki EventLogging extension uses stream config to vary things like event stream sampling rate.
- The Analytics Cluster uses stream config to automate ingestion of streams into Hive.
See the EventStreamConfig README.