You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Performance/WebPageReplay

From Wikitech-static
< Performance
Revision as of 21:10, 4 December 2017 by imported>Phedenskog (Added emulated mobile examples)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

In the path to have more stable metrics in our synthetic testing we have been trying out mahimahi and WebPageReplay to record and replay Wikipedia. For mahimahi we have used patched version fixed by Gilles over Benedikt Wolters HTTP2 version of https://github.com/worenga/mahimahi-h2o. With WebPageReplay we use the default version. The work has been done in T176361.

We have put mahimahi on ice because it is too much of hack and WebPageReplay is more a finished product. However right is our main concern that we only can use Chrome, we should make sure we could also run with Firefox.

The current version run that collect the data for https://grafana.wikimedia.org/dashboard/db/webpagereplay is a Docker container with this setup:

https://github.com/soulgalore/browsertime-replays/tree/master/webpagereplay and the setup looks like this:

WebPageReplay setup.png

Running on AWS (instance type c4.large) we get stable metrics. On desktop we can use 30 frames per second for the video and then get a change of 33 ms for first visual change. Speed Index got more variance but still ok (less than 50 points but it depends on the content). For emulated mobile we can run 60 frames per second and get the same first visual change and Speed Index somewhere of lower 50 points. We run the both desktop and mobile on 100 ms latency.

WebPageTest vs WebPageReplay

Let us compare what the metrics looks like comparing WebPageTest vs WebPageReplay.

Compare emulated mobile First Visual Change on Obama
Compare emulated mobile Speed Index on Obama
First Visual Change on Desktop using WPT vs WebPageReplay
Compare Speed Index on Desktop using WPT vs WebPageReplay