You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

X-Analytics: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Neil P. Quinn-WMF
(Remove what I just added—probably isn't correct)
imported>Krinkle
No edit summary
Line 13: Line 13:
{| class="wikitable"
{| class="wikitable"
|-
|-
! Key !! Possible Values ||  Who sets this value || Use Case !! Team !! Contact Person
! Key !! Value ||  Origin
!Since
!Until!! Team !! Contact|| Use case
|-
|-
   | mf-m
   | mf-m
   | <code>b</code>, <code>amc</code>,  <code>b,amc</code>,  or not set
   | <code>b</code>, <code>amc</code>,  <code>b,amc</code>,  or not set
   | appserver  
   | appserver  
   | If set, then the value <code>b</code> indicates that the user is opted into [[mw:Reading/Web/Mobile Beta|the beta mode (of the mobile site)]] (<code>mf-m=b</code>), the value <code>amc</code> indicates [[Mw:Reading/Web/Advanced mobile contributions|Advanced Mobile Contributions (<code>mf-m=amc</code>)]], and <code>b,amc</code> indicates both (<code>mf-m=b,amc</code>). See [[phab:diffusion/EMFR/browse/master/includes/MobileContext.php;895d6cf2eadd7affa2e38ec2d8be3e789483f0a0$1114|MobileContext.php]].
|?
  | Readers Web
|Current
  | [[User:Phuedx]]
| Readers Web
|-
   | [[User:Phuedx|Phuedx]]
  | php
| If set, then the value <code>b</code> indicates that the user is opted into [[mw:Reading/Web/Mobile Beta|the beta mode (of the mobile site)]] (<code>mf-m=b</code>), the value <code>amc</code> indicates [[Mw:Reading/Web/Advanced mobile contributions|Advanced Mobile Contributions (<code>mf-m=amc</code>)]], and <code>b,amc</code> indicates both (<code>mf-m=b,amc</code>). See [[phab:diffusion/EMFR/browse/master/includes/MobileContext.php;895d6cf2eadd7affa2e38ec2d8be3e789483f0a0$1114|MobileContext.php]].
  | <code>zend</code>, or <code>hhvm</code>
  | appserver
  | If set, marks the used PHP implementation.
 
This tag was only set between September 2014 and January 2015 during the migration from Zend to HHVM.
(See {{Gerrit|I46ff99c198892ade54eef6ffae7ebd264304a681}}, and {{Gerrit|I75b30bef2c146b9241d2bc2f4ad6c1415a04e5d5}})
 
  | Ops
  | _joe_
|-
  | zero
  | MCC-MNC of a [https://wikimediafoundation.org/w/index.php?title=Mobile_partnerships#Where_is_Wikipedia_free_to_access.3F zero carrier], e.g. '''<code>404&#8209;01</code>'''.
| varnish
  | If set, indicates that this request has been associated with the given carrier. It does not mean that the request qualifies as page view.
 
This marker is only set for requests from the mobile varnishes (so no api, bits, upload, ...) and, at this time, only if carrier's whitelisting settings match this request, such as domain (EN is whitelisted, FR is not), HTTPS support (most carriers don't), Proxy (some carriers only whitelist if traffic is via Opera Mini proxy, some - only if direct, some allow both).
 
* [https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/templates/varnish/zero.inc.vcl.erb;f13f911e1f8f85d00fe803eae41aa93d42ed291c zero.inc.vcl.erb].
* [https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/templates/varnish/mobile-frontend.inc.vcl.erb;f13f911e1f8f85d00fe803eae41aa93d42ed291c$19 mobile-frontend.inc.vcl.erb].
  | Wikipedia Zero
  | [[User:Yurik]]
|-
  | zeronet
  | Subdivision of a carrier, e.g. '''<code>b</code>'''
  | varnish
  | Used of disambiguate between different parts/configurations of a single carrier. Like broadband vs. special access points.
  | Wikipedia Zero
  | [[User:Yurik]]
|-
|-
   | proxy
   | proxy
   | Proxy name, e.g. '''<code>Opera</code>'''
   | Proxy name, e.g. '''<code>Opera</code>'''
   | varnish
   | varnish
   | If set, indicates that this request has been received via one of the trusted proxies such as Opera Mini servers.
|?
|Current
| Wikipedia Zero
   | [[User:Yurik|Yurik]]
| If set, indicates that this request has been received via one of the trusted proxies such as Opera Mini servers.


Currently, the following proxies can be expected:
Currently, the following proxies can be expected:
Line 74: Line 52:
| Internet.org (set by  analytics.inc.vcl in puppet)
| Internet.org (set by  analytics.inc.vcl in puppet)
|}
|}
  | Wikipedia Zero
  | [[User:Yurik]]
|-
|-
   | https
   | https
   | <code>1</code>
   | <code>1</code>
   | varnish
   | varnish
   | If set, will be equal to "1", indicating [[HTTPS]] protocol. Currently set for the vast majority of requests, including all that are served with content from [[HTTPS#For_the_Foundation's_canonical_domainnames|canonical WMF domains]]. If it is missing and the HTTP status is 301, the request was sent using HTTP and met with a HTTP redirect response, most likely to the corresponding HTTPS URL. For other response codes <400 (non-errors), it is assumed that the absence of this field also indicates a HTTP request. For some rare cases involving response codes >= 400, it may be possible that this field is not set even though the request was over HTTPS. ([[phab:T188807|More details]])
|?
  | Traffic (Ops)
|Current
  | [[User:BBlack]]
| SRE Traffic
   | [[User:BBlack|BBlack]]
| If set, will be equal to "1", indicating [[HTTPS]] protocol. Currently set for the vast majority of requests, including all that are served with content from [[HTTPS#For_the_Foundation's_canonical_domainnames|canonical WMF domains]]. If it is missing and the HTTP status is 301, the request was sent using HTTP and met with a HTTP redirect response, most likely to the corresponding HTTPS URL. For other response codes <400 (non-errors), it is assumed that the absence of this field also indicates a HTTP request. For some rare cases involving response codes >= 400, it may be possible that this field is not set even though the request was over HTTPS. ([[phab:T188807|More details]])
|-
|-
   | wmfuuid
   | wmfuuid
   | unique value per installation, conforming to [[:w:en:Universally unique identifier#Version 4 (random)|RFC 4122 version 4]], spanning multiple requests
   | [[:en:Universally_unique_identifier#Version_4_(random)|UUID v4 value]]
   | varnish
   | varnish
   | If set, will be equal to a hyphen separated value, and indicates a unique app installation. The ID is generated at app install time, using the appropriate library ([https://docs.oracle.com/javase/7/docs/api/java/util/UUID.html#randomUUID() Java], [https://developer.apple.com/library/ios/documentation/Foundation/Reference/NSUUID_Class/ Objective C]), and conforms to [[:w:en:Universally unique identifier#Version 4 (random)|RFC 4122 version 4]].
|?
|Current
| Mobile apps
   | [[User:dr0ptp4kt|dr0ptp4kt]]
| If set, will be equal to a hyphen separated value, and indicates a unique app installation. The ID may span multiple requests, as it is generated once, at app install time, using an appropriate library ([https://docs.oracle.com/javase/7/docs/api/java/util/UUID.html#randomUUID() Java], [https://developer.apple.com/library/ios/documentation/Foundation/Reference/NSUUID_Class/ Objective C]), and conforms to [[:w:en:Universally unique identifier#Version 4 (random)|RFC 4122 version 4]].


Older versions of the app may contain an ''appInstallID'' parameter in the request URL instead, or may contain both the ''appInstallID'' parameter in the URL as well as the ''wmfuuid'' X-Analytics value. Later versions of the software should only contain the ''wmfuuid'' X-Analytics value and not the ''appInstallID'' parameter in the URL.
Older versions of the app may contain an ''appInstallID'' parameter in the request URL instead, or may contain both the ''appInstallID'' parameter in the URL as well as the ''wmfuuid'' X-Analytics value. Later versions of the software should only contain the ''wmfuuid'' X-Analytics value and not the ''appInstallID'' parameter in the URL.


Requests from the app will not contain this header if the user has turned off "Send usage reports" in the settings menu of the app.
Requests from the app will not contain this header if the user has turned off "Send usage reports" in the settings menu of the app.
  | Mobile apps ("Wikipedia" for Android app, "Wikipedia Mobile" for iOS app)
  | [[User:dr0ptp4kt]]
|-
|-
   | WMF-Last-Access
   | WMF-Last-Access
   | Date of site last access
   |dd-MMM-yyyy, e.g. <code>06-May-2015</code>
   | varnish
   | varnish
   | If set will be equal to the latest date when a device issued a request to the specific host in dd-MMM-yyyy format(Eg: 06-May-2015) and an expiration date set to ~31 days in the future. More explanation at [[Analytics/Unique_clients/Last_access_solution]].
|?
  | Analytics (Infrastructure)
|Current
  | [[User:Milimetric]] or [[User:Madhuvishy]]
| Analytics (Infrastructure)
   | [[User:Milimetric|Milimetric]]
| Date of site last access. If set will be equal to the latest date when a device issued a request to the specific host in dd-MMM-yyyy format(Eg: 06-May-2015) and an expiration date set to ~31 days in the future. More explanation at [[Analytics/Unique_clients/Last_access_solution]].
|-
|-
|preview
|preview
   | Whether this is a preview request (not present if not)
   | 1
   | client  
   | client  
   |At the time of this writing, preview requests by mobile apps are not consider pageviews.  
|?
|
| Analytics (Infrastructure)
   | [[User:Milimetric|Milimetric]]
|Whether this is a preview request (not present if not). At the time of this writing, preview requests by mobile apps are not consider pageviews.  
Expected value is preview=1
Expected value is preview=1
  | Analytics (Infrastructure)
  | [[User:Milimetric]] or [[User:Nuria]]
|-
|-
|pageview
|pageview
   | Whether this request should be counted towards pageviews metric
   | 1
   | client  
   | client  
   | If set it will count the request in question as a pageview regardless of other attributes of request.
|?
  | Analytics (Infrastructure)
|Current
  | [[User:Milimetric]] or [[User:Nuria]]
| Analytics (Infrastructure)
   | [[User:Milimetric|Milimetric]]
| If set it will count the request in question as a pageview regardless of other attributes of request.
|-
|-
|nocookies
|nocookies
   | Whether this is a request that came in without any cookies.
   | 1
   | varnish
   | varnish
   | If set it will tag the request in question as a nocookie request. This means that either this is a fresh browser session, a user browsing with cookies disabled or possibly a bot request.
|?
We expect that the majority of requests tagged with nocookies will belong to bots. Please see: https://gerrit.wikimedia.org/r/#/c/244626/
|Current
  | Analytics (Infrastructure)
| Analytics (Infrastructure)
  | [[User:Madhuvishy]] or [[User:Nuria]]
   | [[User:Madhuvishy|Madhuvishy]] or [[User:Nuria|Nuria]]
| If set it will tag the request in question as a nocookie request. This means that either this is a fresh browser session, a user browsing with cookies disabled or possibly a bot request.
We expect that the majority of requests tagged with nocookies will belong to bots. Please see: [[gerrit:#/c/244626/|change 244626]].
|-
|-
|loggedIn
|loggedIn
|1
|1
|WikimediaEvents extension
|appserver (WikimediaEvents)
|?
|
|WMDE-Analytics
|[[User:Addshore|Addshore]]
|If set, will be equal to "1", and indicates that the request came from a logged in user (see also [[phab:rEWMVa3d93b5cbf4ba02b14b028b61879fd4f2e409419|code]]).
|If set, will be equal to "1", and indicates that the request came from a logged in user (see also [[phab:rEWMVa3d93b5cbf4ba02b14b028b61879fd4f2e409419|code]]).
|WMDE-Analytics
|[[User:Addshore]]
|-
|-
|page_id
|page_id
|The page ID
|Page ID
|WikimediaEvents extension
|appserver (WikimediaEvents)
|?
|
|WMDE-Analytics
|[[User:Addshore|Addshore]], [[User:Ori.livneh|Ori.livneh]]
|If set, will be a string of a positive integer.
|If set, will be a string of a positive integer.
|Ori & WMDE-Analytics
|[[User:Addshore]] & [[User:Ori.livneh]]
|-
|-
|ns
|ns
|The namespace ID
|Namespace ID
|WikimediaEvents extension
|appserver (WikimediaEvents)
|?
|
|WMDE-Analytics
|[[User:Addshore|Addshore]], [[User:Ori.livneh|Ori.livneh]]
|If set, will be a string integer (can be negative for negative namespace IDs)
|If set, will be a string integer (can be negative for negative namespace IDs)
|Ori & WMDE-Analytics
|[[User:Addshore]] & [[User:Ori.livneh]]
|-
|-
|special
|special
|String, resolved special page name
|Special page name
|WikimediaEvents extension
|appserver (WikimediaEvents)
|?
|
|WMDE-Analytics
|[[User:Addshore|Addshore]]
|Set for special pages only. This will be the base name of the special page, so if the user is browsing a page via an alias the actual page name will be here.
|Set for special pages only. This will be the base name of the special page, so if the user is browsing a page via an alias the actual page name will be here.
|WMDE-Analytics
|[[User:Addshore]]
|-
|-
|translationengine
|translationengine
|Identifier of intermediary, such as <code>GT</code>
|Identifier, e.g. <code>GT</code>
|varnish
|varnish
|If set, indicates request served through identified translation service. "GT" stands for Google Translate.
|Nov&nbsp;2018
|Audiences
|Current
|[[meta:User:ABaso (WMF)|User:ABaso (WMF)]]
|Product
|[[meta:User:ABaso (WMF)|ABaso]]
|If set, indicates request served through a known intermediary service for machine translations. "GT" stands for Google Translate.
Added in [[phab:T208795|T208795]].
|-
|-
|wprov
|wprov
|<3_char_feature>
<1_char_platform><major_version>
e.g. <code>srpw1</code> for SRP, Web, v1.
|client ''or'' varnish
|?
|Current
|?
|?
|see [[Provenance]]
|see [[Provenance]]
|client ''or'' Varnish servers
|-
|
|debug
|
|1
|
|varnish
|Jan&nbsp;2021
|Current
|Analytics (Infrastructure)
|[[User:Milimetric|Milimetric]]
|Added in [[phab:T263683|T263683]].
|-
|client_port
|medium-size int
|varnish
|Jan 2021
|Current
|Analytics (Infrastructure)
|JAllemandou
|Added in [[phab:T271953|T271953]].
|-
|public_cloud
|1
|varnish
|May&nbsp;2021
|Current
|SRE
|[[User:CDanis|CDanis]]
|Added in [[phab:T279380|T279380]].
|}
 
=== Former keys ===
{| class="wikitable"
|-
! Key !! Value ||  Origin
!Since
!Until!! Team !! Contact|| Use case
|-
  | php
  | <code>zend</code>, or <code>hhvm</code>
  | appserver
|?
|Jan&nbsp;2015
| SRE
  | _joe_
| If set, marks the used PHP implementation.
 
This tag was only set between September 2014 and January 2015 during the migration from Zend to HHVM. (See {{Gerrit|I46ff99c198892ade54eef6ffae7ebd264304a681}}, and {{Gerrit|I75b30bef2c146b9241d2bc2f4ad6c1415a04e5d5}})
|-
|zero
|MCC-MNC of a [https://wikimediafoundation.org/w/index.php?title=Mobile_partnerships#Where_is_Wikipedia_free_to_access.3F zero carrier], e.g. '''<code>404&#8209;01</code>'''.
|varnish
|?
|July 2019
|Wikipedia Zero
|[[User:Yurik|Yurik]]
|If set, indicates that this request has been associated with the given carrier. It does not mean that the request qualifies as page view.
 
* [[phab:diffusion/OPUP/browse/production/templates/varnish/zero.inc.vcl.erb;f13f911e1f8f85d00fe803eae41aa93d42ed291c|zero.inc.vcl.erb]].
* [[phab:diffusion/OPUP/browse/production/templates/varnish/mobile-frontend.inc.vcl.erb;f13f911e1f8f85d00fe803eae41aa93d42ed291c$19|mobile-frontend.inc.vcl.erb]].
Removal in [[phab:T213769|T213769]].
|-
|zeronet
|Subdivision of a carrier, e.g. '''<code>b</code>'''
|varnish
|?
|July&nbsp;2019
|Wikipedia Zero
|[[User:Yurik|Yurik]]
|Used of disambiguate between different parts/configurations of a single carrier. Like broadband vs. special access points.
Removal in [[phab:T213769|T213769]].
|}
|}


Revision as of 22:38, 22 June 2021

X-Analytics is a general purpose header for measurement purposes, which is used in the Cache log format, and recorded in e.g. the webrequest table.

Format

The X-Analytics header is a a list of zero or more key=value pairs, separated by semicolons. So for example mf-m=b, or zero=123-45;mf-m=b. If a key occurs multiple times in an X-Analytics header, it is undefined which one takes precedence.

The special value “-” has to be interpreted as the empty string.

Keys

Looks like X-Analytics extension might be adding these values if used: [1]

Key Value Origin Since Until Team Contact Use case
mf-m b, amc, b,amc, or not set appserver ? Current Readers Web Phuedx If set, then the value b indicates that the user is opted into the beta mode (of the mobile site) (mf-m=b), the value amc indicates Advanced Mobile Contributions (mf-m=amc), and b,amc indicates both (mf-m=b,amc). See MobileContext.php.
proxy Proxy name, e.g. Opera varnish ? Current Wikipedia Zero Yurik If set, indicates that this request has been received via one of the trusted proxies such as Opera Mini servers.

Currently, the following proxies can be expected:

Value Description
Opera Opera mini proxy
Nokiaprod Nokia Xpress Production Proxy Servers
Nokiaqa Nokia Xpress QA Lab Proxy Servers
IORG Internet.org (set by analytics.inc.vcl in puppet)
https 1 varnish ? Current SRE Traffic BBlack If set, will be equal to "1", indicating HTTPS protocol. Currently set for the vast majority of requests, including all that are served with content from canonical WMF domains. If it is missing and the HTTP status is 301, the request was sent using HTTP and met with a HTTP redirect response, most likely to the corresponding HTTPS URL. For other response codes <400 (non-errors), it is assumed that the absence of this field also indicates a HTTP request. For some rare cases involving response codes >= 400, it may be possible that this field is not set even though the request was over HTTPS. (More details)
wmfuuid UUID v4 value varnish ? Current Mobile apps dr0ptp4kt If set, will be equal to a hyphen separated value, and indicates a unique app installation. The ID may span multiple requests, as it is generated once, at app install time, using an appropriate library (Java, Objective C), and conforms to RFC 4122 version 4.

Older versions of the app may contain an appInstallID parameter in the request URL instead, or may contain both the appInstallID parameter in the URL as well as the wmfuuid X-Analytics value. Later versions of the software should only contain the wmfuuid X-Analytics value and not the appInstallID parameter in the URL.

Requests from the app will not contain this header if the user has turned off "Send usage reports" in the settings menu of the app.

WMF-Last-Access dd-MMM-yyyy, e.g. 06-May-2015 varnish ? Current Analytics (Infrastructure) Milimetric Date of site last access. If set will be equal to the latest date when a device issued a request to the specific host in dd-MMM-yyyy format(Eg: 06-May-2015) and an expiration date set to ~31 days in the future. More explanation at Analytics/Unique_clients/Last_access_solution.
preview 1 client ? Analytics (Infrastructure) Milimetric Whether this is a preview request (not present if not). At the time of this writing, preview requests by mobile apps are not consider pageviews.

Expected value is preview=1

pageview 1 client ? Current Analytics (Infrastructure) Milimetric If set it will count the request in question as a pageview regardless of other attributes of request.
nocookies 1 varnish ? Current Analytics (Infrastructure) Madhuvishy or Nuria If set it will tag the request in question as a nocookie request. This means that either this is a fresh browser session, a user browsing with cookies disabled or possibly a bot request.

We expect that the majority of requests tagged with nocookies will belong to bots. Please see: change 244626.

loggedIn 1 appserver (WikimediaEvents) ? WMDE-Analytics Addshore If set, will be equal to "1", and indicates that the request came from a logged in user (see also code).
page_id Page ID appserver (WikimediaEvents) ? WMDE-Analytics Addshore, Ori.livneh If set, will be a string of a positive integer.
ns Namespace ID appserver (WikimediaEvents) ? WMDE-Analytics Addshore, Ori.livneh If set, will be a string integer (can be negative for negative namespace IDs)
special Special page name appserver (WikimediaEvents) ? WMDE-Analytics Addshore Set for special pages only. This will be the base name of the special page, so if the user is browsing a page via an alias the actual page name will be here.
translationengine Identifier, e.g. GT varnish Nov 2018 Current Product ABaso If set, indicates request served through a known intermediary service for machine translations. "GT" stands for Google Translate.

Added in T208795.

wprov <3_char_feature>

<1_char_platform><major_version>


e.g. srpw1 for SRP, Web, v1.

client or varnish ? Current ? ? see Provenance
debug 1 varnish Jan 2021 Current Analytics (Infrastructure) Milimetric Added in T263683.
client_port medium-size int varnish Jan 2021 Current Analytics (Infrastructure) JAllemandou Added in T271953.
public_cloud 1 varnish May 2021 Current SRE CDanis Added in T279380.

Former keys

Key Value Origin Since Until Team Contact Use case
php zend, or hhvm appserver ? Jan 2015 SRE _joe_ If set, marks the used PHP implementation.

This tag was only set between September 2014 and January 2015 during the migration from Zend to HHVM. (See I46ff99, and I75b30b)

zero MCC-MNC of a zero carrier, e.g. 404‑01. varnish ? July 2019 Wikipedia Zero Yurik If set, indicates that this request has been associated with the given carrier. It does not mean that the request qualifies as page view.

Removal in T213769.

zeronet Subdivision of a carrier, e.g. b varnish ? July 2019 Wikipedia Zero Yurik Used of disambiguate between different parts/configurations of a single carrier. Like broadband vs. special access points.

Removal in T213769.

Code

https://github.com/wikimedia/mediawiki-extensions-XAnalytics

Extension documentation

mw:Extension:XAnalytics

Tasks and Bugs

https://phabricator.wikimedia.org/tag/xanalytics/

See also

Retrieved from "https://wikitech-static.wikimedia.org/w/index.php?title=X-Analytics&oldid=581502"