imported>Nuria |
imported>Milimetric |
| (One intermediate revision by one other user not shown) |
| Line 1: |
Line 1: |
| The consumer side of event logging can be easily tested on Beta Cluster.
| | #REDIRECT [[Analytics/Systems/EventLogging/TestingOnBetaCluster]] |
| | |
| == Instance ==
| |
| | |
| The instance name is configured here: https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/CommonSettings-labs.php
| |
| Note that this might change at any time but other than the instance the rest of the info on this document should apply regardless of the instance
| |
| | |
| Note that you need <code>sudo</code> on this instance to see logs, any user trying to test stuff on Beta Cluster should ask for sudo on <code>deployment-eventlogging03.eqiad.wmflabs</code>.
| |
| It is unfortunate that sudo is required but that is the state of affairs right now.
| |
| | |
| == How to create test events ==
| |
| | |
| === How to log a client-side event to Beta Cluster directly ===
| |
| Just hit the varnish endpoint on labs, for example:
| |
| | |
| http://bits.beta.wmflabs.org/event.gif?%7B%22event%22%3A%7B%22mobileMode%22%3A%22stable%22%2C%22name%22%3A%22hamburger%22%7D%2C%22revision%22%3A10742159%2C%22schema%22%3A%22MobileWebUIClickTracking%22%2C%22webHost%22%3A%22en.m.wikipedia.beta.wmflabs.org%22%2C%22wiki%22%3A%22enwiki%22%7D;
| |
| | |
| === How to log via the website ===
| |
| Use http://en.m.wikipedia.beta.wmflabs.org/wiki/Main_Page to create events in mobile, for example.
| |
| | |
| === How to load test with a bunch of events ===
| |
| There's a script that may be handy. It's in the same eventlogging codebase:
| |
| | |
| https://github.com/wikimedia/eventlogging/blob/master/bin/eventlogging-load-tester
| |
| | |
| == How to verify events ==
| |
| You can tail the files in the <code>/srv/log/eventlogging</code> on <code>deployment-eventlogging03.eqiad.wmflabs</code> to verify if your event is coming through.
| |
| Unless noted otherwise, the files mentioned in this section and the subsections are in this directory.
| |
| | |
| ssh deployment-eventlogging03.eqiad.wmflabs
| |
| cd /srv/log/eventlogging
| |
| | |
| === Validated events ===
| |
| * <code>all-events.log</code>: validated events appear in this file
| |
| Tail this file while you use the website and emit server or client side events. If your events are valid your events should be there after a short while (seconds).
| |
| If they don't appear then check the next section.
| |
| | |
| === Raw stream of events (including unvalidated events) ===
| |
| * <code>client-side-events.log</code>: client side events appear in this file (valid and not)
| |
| | |
| If events do not appear they might not be valid, check <code>/var/log/upstart/</code> for either the client-side processor logs.
| |
| eventlogging_processor-client-side-events.log
| |
| | |
| | |
| Validation errors will appear on those logs and they are very descriptive.
| |
| | |
| === Where is eventlogging code? ===
| |
| | |
| /srv/deployment/eventlogging/analytics/eventlogging
| |
| | |
| == Database ==
| |
| The mysql server is storing events just like it is in production, in order to see events you can use the eventlogging user whose user and password
| |
| are listed at:
| |
| | |
| /etc/eventlogging.d/consumers/mysql-m2-master
| |
| | |
| If you have sudo on the machine the mysql password for the root user is 'secret', otherwise:
| |
| | |
| mysql -h 127.0.0.1 --user=eventlogging --password=68QrOq220717816UycU1
| |
| (it's labs, the password is not really a secret.)
| |
| | |
| | |
| If mysql needs a re-start:
| |
| | |
| /etc/init.d/mysql start
| |
| | |
| | |
| TokuDB shoudl be enabled by default, otherwise try:
| |
| /etc/init.d/mysql start--default-storage-engine=tokudb
| |
| | |
| The mysql setup on beta leaves much to be desired, if mysql does start check /var/log/mysql.err
| |
| | |
| This might be of help: [http://www.chriscalender.com/disabling-transparent-hugepages-for-tokudb/]
| |
| | |
| == Admin ==
| |
| | |
| === Give people access ===
| |
| Add them to the lists on these wikis (you need to be an admin to do that)
| |
| Asking in #wikimedia-labs might be a way to get help.
| |
| | |
| [[Nova_Resource:Deployment-prep]] | |
| | |
| [[Special:NovaProject]] -> add users to deployment-prep
| |
| | |
| === How to deploy code ===
| |
| | |
| <source lang="bash">
| |
| # Log into the beta deploy server
| |
| ssh deployment-tin.deployment-prep.eqiad.wmflabs
| |
| | |
| # cd to the EventLogging analytics deploy source
| |
| cd /srv/deployment/eventlogging/analytics
| |
| | |
| # Deploy using scap3 in the beta environment
| |
| deploy -e beta
| |
| </source>
| |
| | |
| You can run puppet with
| |
| | |
| puppet agent -tv
| |
| | |
| === Restart EventLogging ===
| |
| Check:
| |
| | |
| /etc/init/eventlogging/init.conf
| |
| | |
| | |
| Run:
| |
| | |
| sudo eventloggingctl restart
| |
| | |
| | |
| Stop completely:
| |
| sudo eventloggingctl stop
| |
| | |
| The config applied to create logs and such by upstart is at:
| |
| | |
| /etc/eventlogging.d/consumers/
| |
| | |
| == Kafka ==
| |
| | |
| If you're testing Kafka stuff on the beta cluster, you'll need a zookeeper. You can pass <code>--zookeeper deployment-zookeeper01:2181/kafka/deployment-kafka</code>. Or you can just do <code>export ZOOKEEPER_URL=deployment-zookeeper01:2181/kafka/deployment-kafka</code>
| |