You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Analytics/Data Lake/Mediawiki history/Presentation

From Wikitech-static
Jump to navigation Jump to search

ː What is the audience?

Everybody at WMF (Grantmaking, Community liasons, infrastructure, editing, reading, research, C levels, legal, communications, ...) and some community members.

ː What is the message at the high level?

Mediawiki data supports us all in making decisions and strategy. We have data in 700 separate databases with missing important information. It's hard to ask questions and get quick answers. Our solution to this problem was to clean, reconstruct, and unify all of our data. Decision support questions are easier to ask and the system responds much faster.

Currently, data based decision is very difficult because our data lives in a complicated schema in over 700 separate databases, and some data was lost over time. Now we build a unified and coherent easy to access data environment which can be used for better decisions in every level of our movement. Mediawiki data supports decision making at all levels of our movement.

ːː What is the detailed message (an example)?

(Let's assume we are interested in the dynamics of the creation process.) How many edits are made using our labs infrastructure, to any wiki? This query took three engineers one week to answer and now it's a simple 20 minute query. How many active editors do we have de-duplicated across all our projects? This question took 3 weeks of processing to answer in the past, now it's a few minutes. How many editors are surviving to continue editing the month after they register? This query used to be really hard to write, now we can take a look together (show surviving̠ new editor).

ː What is the level of the presentation?

Summarize advantages for community, C-level team, directors, and analysts.

ː Links

https://pivot.wikimedia.org/#pageviews-daily/line-chart/2/EQUQLgxg9AqgKgYWAGgN7APYAdgC5gQAWAhgJYB2KwApgB5YBO1Azs6RpbutnsEwGZVyxALbVeAfQlhSY4AF9kwYhBkdmeANroVazsApU6jFmw55uOfABtSYag2LWqANycBXcV2DMwxBmC8AEwADACMAGwAtCHRQWFwISG4SSkhAHRJIQBaRuQAJsHh0SEAnFFhCanVGVm5isBgAJ5YXsBwAJIAsiASAEoAggByAOIgCoo6quz6xAVG9EysMxaYVgQkhkrGS2aclrwCQqJt7sQSAEYMGADuzA4S/KKk1k0KSror3p/m+BguDmsxBw20Wpi+BxsdgcTlcHi86HuYDgLTaAGU4H0OqMjNZqGJyGANLhNMAEIRrnIALoNZqtXhokBwCbyeQ05DaGhg5a/SF8aiCJTCOT4M6Xa53B4iYgAKwwDHePnlYAG015yjV+mYyoWJh5+zWhwFxxFwBcpGoNwkEAw7kJivypCYmt4+RYEGoBQoAHMJmguXq9qsePgjkKTpJpLJxA0Lu4IABragql3fVONaMAIXjScCSny7kcel4AAUwgARRXagKq4tputKgK63YQw2h43h01SGRyBqO50N4jMD1e8i+1lU5DkdzWaxKc2W622+0cs0Wq02u2BSeaSfT2f+x0EoOi85XW73BgSaVyhVKWwiOx4ACs8iAA===

https://pivot.wikimedia.org/#mediawiki-history-beta/line-chart/2/EQUQLgxg9AqgKgYWAGgN7APYAdgC5gQAWAhgJYB2KwApgB5YBO1Azs6RpbutnsEwGZVyxALbVeAfQlhSY4AF9kwYhBkdmeANroVazsApU6jFmw55uOfABtSYag2LWqANycBXcV2DMwxBmC8AEwADACMAGwAtCEAnDFhcCEhuMmpIQB0ySEAWkbkACbB4dFhIVEAzIlpNZnZeYrAYACeWF7AcACSALIgEgBKAIIAcgDiIAqKOqrs+sSFRvRMrLMWmFYEJIZKxstmnJa8AkKi7dQu1ORgEpcyLQpKuqveT+b4GBcM1sQ4O0umz0ONjsDicrg8XnQzGoYDgrXaAGU4P1OmMjNZqGIrhpcJo+OdSPtgABdRotNq8BEgOCTeRTZQzN7KBZ/EwrN5AzZkSisvaA9ZHaiCJTCOT4LDEADm1AkopYEogMsJEggHHsVweDL0a1e+g+Dm+vxo/3ZBwFwPsjmcSjc1k8a2hsPhlORqNG6MxtxxeLCJLJzvwVJpdLppOQ2mNbKJ3h4+GOItOvHOtwk7mhDAkyoARhhrlnmrLE41mBgAoNGWbdbwSwFFlH+bH8cLgHLePMOM0RBg0zcLldU+mNI0CqQmBXeAUWIrChRJZM0JG+RzzU2TmLgFIZHJGln3BAANYw8val7j/Bb6gAIT3h8CSgK7kcJ+AAAUwgARTU1sDH/lV/DfnWS5mo28Ytom+CbrI4jDqO1BnsozDTiO5BzqGyDkO41jWDsfZgBo4Y0HhGjEuGpFEV6KDoCOWLRhR/Zpg4mbMBIOZ5gWrZKLYIh2HgACs8hAA