You are browsing a read-only backup copy of Wikitech. The live site can be found at

Analytics/EventLogging/TestingOnBetaCluster: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
(One intermediate revision by one other user not shown)
Line 1: Line 1:
The consumer side of event logging can be easily tested on Beta Cluster.
#REDIRECT [[Analytics/Systems/EventLogging/TestingOnBetaCluster]]
== Instance ==
The instance name is configured here:
Note that this might change at any time but other than the instance the rest of the info on this document should apply regardless of the instance
Note that you need <code>sudo</code> on this instance to see logs, any user trying to test stuff on Beta Cluster should ask for sudo on <code>deployment-eventlogging03.eqiad.wmflabs</code>.
It is unfortunate that sudo is required but that is the state of affairs right now.
== How to create test events ==
=== How to log  a client-side event to Beta Cluster directly ===
Just hit the varnish endpoint on labs, for example:;
=== How to log via the website ===
Use to create events in mobile, for example.
=== How to load test with a bunch of events ===
There's a script that may be handy. It's in the same eventlogging codebase:
== How to verify events ==
You can tail the files in the <code>/srv/log/eventlogging</code> on <code>deployment-eventlogging03.eqiad.wmflabs</code> to verify if your event is coming through.
Unless noted otherwise, the files mentioned in this section and the subsections are in this directory.
ssh deployment-eventlogging03.eqiad.wmflabs
cd /srv/log/eventlogging
=== Validated events ===
* <code>all-events.log</code>: validated events appear in this file
Tail this file while you use the website and emit server or client side events. If your events are valid your events should be there after a short while (seconds).
If they don't appear then check the next section.
=== Raw stream of events (including unvalidated events) ===
* <code>client-side-events.log</code>: client side events appear in this file (valid and not)
If events do not appear they might not be valid, check <code>/var/log/upstart/</code> for either the client-side processor logs.
Validation errors will appear on those logs and they are very descriptive.
=== Where is eventlogging code? ===
== Database ==
The mysql server is storing events just like it is in production, in order to see events you can use the eventlogging user whose user and password
are listed at:
If you have sudo on the machine the mysql password for the root user is 'secret', otherwise:
mysql -h --user=eventlogging --password=68QrOq220717816UycU1
(it's labs, the password is not really a secret.)
If mysql needs a re-start:
/etc/init.d/mysql start
TokuDB shoudl be enabled by default, otherwise try:
  /etc/init.d/mysql start--default-storage-engine=tokudb
The mysql setup on beta leaves much to be desired, if mysql does start check /var/log/mysql.err
This might be of help: []
== Admin ==
=== Give people access ===
Add them to the lists on these wikis (you need to be an admin to do that)
Asking in #wikimedia-labs might be a way to get help.
[[Special:NovaProject]] -> add users to deployment-prep
=== How to deploy code ===
<source lang="bash">
# Log into the beta deploy server
ssh deployment-tin.deployment-prep.eqiad.wmflabs
# cd to the EventLogging analytics deploy source
cd /srv/deployment/eventlogging/analytics
# Deploy using scap3 in the beta environment
deploy -e beta
You can run puppet with
puppet agent -tv
=== Restart EventLogging ===
  sudo  eventloggingctl restart
Stop completely:
  sudo  eventloggingctl stop
The config applied to create logs and such by upstart is at:
== Kafka ==
If you're testing Kafka stuff on the beta cluster, you'll need a zookeeper.  You can pass <code>--zookeeper deployment-zookeeper01:2181/kafka/deployment-kafka</code>.  Or you can just do <code>export ZOOKEEPER_URL=deployment-zookeeper01:2181/kafka/deployment-kafka</code>

Revision as of 14:13, 7 April 2017