You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Metrics Platform: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Will Doran
No edit summary
imported>Emil Chetty
 
(3 intermediate revisions by 3 users not shown)
Line 1: Line 1:
The '''Metrics Platform''' is <section begin="excerpt" />a suite of services, standard libraries, and APIs for producing and consuming data of all kinds from Wikimedia Foundation products.<section end="excerpt" />.


The '''Metrics Platform''' is <section begin=excerpt/>a suite of services, standard libraries, and APIs for producing and consuming data of all kinds from Wikimedia Foundation products.<section end=excerpt/> For data producers, it aims to simplify the task of designing, implementing, and maintaining data-producing code (also called ''instrumentation''), while offering better guarantees of quality, rigor, and safety. For data consumers, it aims to streamline the process of defining a new dataset and offer a rich set of tools to answer questions and generate insights with data.
The Metrics Platform provides standardized algorithms, behaviors, and basic necessities for web and app instrumentation, including:


The Metrics Platform:
* standardized session ID generation, consistent across MediaWiki, Android, and iOS
* standardized session expiry
* enriching events with contextual data
* determining which events are in-sample or out-of-sample based on a specific identifier (currently: pageview, session, or app install ID)


* specifies how clients work with the [[Event Platform]]
== Quick Start ==
* provides standardized algorithms, behaviors, and basic necessities for web and app tracking, including:
Currently, the JS and PHP Metrics Platform Clients only work within [[mw:Extension:EventLogging|the EventLogging extension]]. Before we get started defining streams and creating instruments, you will need to have an up-to-date MediaWiki development environment. If you do not have one, then please consider using [[mw:MediaWiki-Docker|MediaWiki-Docker]].
** standardized session ID generation, consistent across MediaWiki, Android, and iOs
** attaching the necessary metadata to logged events such as client-side timestamp recording when the event was generated
** determining which events are in-sample or out-of-sample based on a specific identifier (pageview, session, device)


=== Topics Directory ===
=== Setup and Hello, World! ===
* [[Metrics Platform/Client/Implementations]] - Differences between the various client implementations
See [[mw:MediaWiki-Docker/Configuration_recipes/EventLogging#Event_Platform|this Event Platform configuration recipe for MediaWiki-Docker]] for detailed instructions about setting up an Event Platform (ish) environment alongside your MediaWiki development environment. Afterwards, add this to <code>LocalSettings.php</code>:<syntaxhighlight lang="php">
* [[Metrics Platform/Client]] - Specification of interfaces, data structures, algorithms for conforming clients
$wgEventStreams = [
* [[Metrics Platform/Event]] - Specification of format and properties of an event
    'test.all' => [
* [[Metrics Platform/Event Context Attributes]] - Specification of context attributes
        'schema_title' => '/analytics/mediawiki/client/metrics_event',
* [[Metrics Platform/Event Schema]] - Specification of compatible schema
        'producers' => [
* [[Metrics Platform/Event Stream Configuration]] - Specification of stream configuration
            'metrics_platform_client' => [
*[[Metrics Platform/Demos]] - Biweekly demo recordings
                'events' => [ '' ], // Matches all event names
            ],
        ],
    ],
];
$wgEventLoggingStreamNames = [ 'test.all' ];
</syntaxhighlight>Navigate to http://localhost:8080/wiki/Main_Page and run this in the console:<syntaxhighlight lang="javascript">
mw.eventLog.dispatch( 'test.hello_world', {
    hello: 'World!'
} );
</syntaxhighlight>


=== Development Status ===
== Rationale ==
{| class="wikitable" text-align:left; margin-left:40px; margin-top:20px; margin-bottom:40px;"
Previously, different teams implemented their own analytics solutions in isolated from one another. Those solutions were typically based on the Legacy EventLogging pipeline and, more recently, [[Event Platform|the Event Platform]]. The Metrics Platform is an effort to unify that previous work and to establish uniformity and consistency across platforms.
|-
|+Current product platform support
|-
! Version
! Release Date
! style="width:60px; text-align:center;" | MediaWiki JS
! style="width:60px; text-align:center;" | MediaWiki PHP
! style="width:60px; text-align:center;" | Wikipedia Android
! style="width:60px; text-align:center;" | Wikipedia iOS
! style="width:60px; text-align:center;" | Wikipedia KaiOS
|-
! style="text-align:left;" | 1.0 (in development)
| 05-30-2021
| style="text-align:center;" |
| style="text-align:center;" |  
| style="text-align:center;" |
| style="text-align:center;" |
|
|-
! style="text-align:left;" | 0.2
| 12-30-2020
| style="text-align:center;" | x
| style="text-align:center;" | x
| style="text-align:center;" | x
| style="text-align:center;" | x
|
|-
! style="text-align:left;" | 0.1
| 06-30-2020
| style="text-align:center;" | x
|
|
| style="text-align:center;" | x
|
|-
|}


=== History and Rationale ===
It will be easier to leverage data from multiple platforms to yield insights into how our users use our whole ecosystem of products in unison.
Previously, different teams implemented their own EventLogging-based analytics solutions, isolated from each other. The Metrics Platform is an effort to unify that previous work and to establish consistency across platforms. That uniformity and consistency makes it possible to leverage data from multiple platforms to yield insights into how our users use our whole ecosystem of products in unison.


It also enables analysts to support teams which are not their primary teams – to be more portable. The legacy system, in which every instrumentation has its own quirks and naming is inconsistent, places a heavy burden on each analyst to learn and remember the specifics of their assigned teams' data; and if another analyst had to come in as back-up, they too would need to learn those specifics.
It will be easier for analysts to support teams which are not their primary teams. Previously, every instrument has its own quirks and conventions, which need to be remembered by analyst that works with the data produced by the instrument – including analysts that are providing temporary support.
 
It will be easier to implement and maintain instruments. The Metrics Platform will provide the APIs and protocols that MediaWiki developers need to create sophisticated instruments in as few lines of code as possible while maintaining quality, rigor, and safety.
 
== Further Reading ==
 
=== FAQ ===
[[Metrics Platform/FAQ]]
 
=== Getting Started ===
[[Metrics Platform/Getting Started/Creating An Instrument]]
 
[[Metrics Platform/Getting Started/Hacking On Metrics Platform]]
 
=== How To ===
[[Metrics Platform/Creating a Stream Configuration]]
 
=== Reference Guides ===
[[Metrics Platform/Implementations]] - The APIs provided by each Metrics Platform Client implementation and the key differences between them
 
[[Metrics Platform/Identifiers|Metrics Platform/Identifiers]]
 
[[Metrics Platform/Event Schema]] - The specification of the monoschema
 
== Archive ==
The following pages were archived while this page and its subpages were rewritten:
* [[Metrics Platform/Demos]]
* [[Metrics Platform/Client]] - Specification of interfaces, data structures, algorithms for the Metrics Platform Clients

Latest revision as of 12:11, 28 November 2022

The Metrics Platform is <section begin="excerpt" />a suite of services, standard libraries, and APIs for producing and consuming data of all kinds from Wikimedia Foundation products.<section end="excerpt" />.

The Metrics Platform provides standardized algorithms, behaviors, and basic necessities for web and app instrumentation, including:

  • standardized session ID generation, consistent across MediaWiki, Android, and iOS
  • standardized session expiry
  • enriching events with contextual data
  • determining which events are in-sample or out-of-sample based on a specific identifier (currently: pageview, session, or app install ID)

Quick Start

Currently, the JS and PHP Metrics Platform Clients only work within the EventLogging extension. Before we get started defining streams and creating instruments, you will need to have an up-to-date MediaWiki development environment. If you do not have one, then please consider using MediaWiki-Docker.

Setup and Hello, World!

See this Event Platform configuration recipe for MediaWiki-Docker for detailed instructions about setting up an Event Platform (ish) environment alongside your MediaWiki development environment. Afterwards, add this to LocalSettings.php:

$wgEventStreams = [
    'test.all' => [
        'schema_title' => '/analytics/mediawiki/client/metrics_event',
        'producers' => [
            'metrics_platform_client' => [
                'events' => [ '' ], // Matches all event names
            ],
        ],
    ],
];
$wgEventLoggingStreamNames = [ 'test.all' ];

Navigate to http://localhost:8080/wiki/Main_Page and run this in the console:

mw.eventLog.dispatch( 'test.hello_world', {
    hello: 'World!'
} );

Rationale

Previously, different teams implemented their own analytics solutions in isolated from one another. Those solutions were typically based on the Legacy EventLogging pipeline and, more recently, the Event Platform. The Metrics Platform is an effort to unify that previous work and to establish uniformity and consistency across platforms.

It will be easier to leverage data from multiple platforms to yield insights into how our users use our whole ecosystem of products in unison.

It will be easier for analysts to support teams which are not their primary teams. Previously, every instrument has its own quirks and conventions, which need to be remembered by analyst that works with the data produced by the instrument – including analysts that are providing temporary support.

It will be easier to implement and maintain instruments. The Metrics Platform will provide the APIs and protocols that MediaWiki developers need to create sophisticated instruments in as few lines of code as possible while maintaining quality, rigor, and safety.

Further Reading

FAQ

Metrics Platform/FAQ

Getting Started

Metrics Platform/Getting Started/Creating An Instrument

Metrics Platform/Getting Started/Hacking On Metrics Platform

How To

Metrics Platform/Creating a Stream Configuration

Reference Guides

Metrics Platform/Implementations - The APIs provided by each Metrics Platform Client implementation and the key differences between them

Metrics Platform/Identifiers

Metrics Platform/Event Schema - The specification of the monoschema

Archive

The following pages were archived while this page and its subpages were rewritten: