You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
Accounts & Privileges to get
- Live - https://civicrm.wikimedia.org
- Staging - https://civicrm.frdev.wikimedia.org (db will be the same as live one, while will update later)
- Local - https://wmff.localhost:32353/civicrm?reset=1
- Payment processor consoles
- Gerrit, our code review system
- Add to FailMail recipient list
- SSL client certificate to access Superset, our data analyst dashboards.
- Phabricator for task management
- Set a dashboard with this style(Optional)
- Packagist to update PHP repository managed by us's version
- FILE SERVER we want the Fundraising folder available, request access to IT service
- Write your name inside the team contact list with phone number and email address, this will add you to Gerrit as project's git reviewer.
- Get IRC join team #wikimedia-fundraising and #xxxx(ask your team member). Choose your own irc connection methods, limechat or irccloud or online works
- Manage gmail filters
- Add a team calendar for online meetings.
- Know more about FR team & Fundraising, with lots links for payments and test links
- Install your own IDE, JetBrains tool is free with wiki email if you use phpStorm.
- Join Asana with gmail and join team “online fundraising” by searching coworker’s email.
- Check Fr-tech priorities meeting notes for each spring's task overview
Set up local Env
- Refer to the Readme, get all the code we are working on in local env
- install git-review for peer review
- use Zuul to check code reveiw by Jenkins.
- use any GUI you like, with the Docker up, get all the code and the database, Redis and so on based on containers
- if use the MysqlWorkBench, set useSSL=0 to advance others to enable no ssl for standard TCP/IP.
- Yubikey will be requested by your manger.
- With the Yubikey set up, able to go to the dev server with ssh access,
- To set up your Yubikey, you needs to be corporate with Dallas
- With ssh access, you can perform deploy(see below, ask peers to do it together at your first time.
- We have three projects (Smashpig, payments, civicrm) which smashipig is working for both payments and civicrm.
- Whenever we has some package version updates, need to update the correspondent package version for donor interface and smashpig
- the access could be granted by Elliott
Some account might not need at this moment but good to have later
- Wikimedia developer account
- Cloud VPS Projects
- Horizon: Manage tech development server, now just forwarding things, and also manger central notice staging server
- Silverpop: now acoustic, could be used for FailMail forget me, for email list.
- Participate the Onboarding meeting and set up some account
- Get to know the team and set up some 1:1 with your onboard buddy
- Get accesses per above list
What is Gerrit and get to know code in general
- Register Gerrit and get your username, use it to download code from Gerrit and run setup.sh with Gerrit username as reviewer name to set docker up;
- After you got the yubikey, you can pull the config-private from frpm1001:/var/lib/git/fundraising-dev-private.git, also for this folder, we use git push instead of git review
- All the live version config files can be found from server set up your local env and start to get to know the code
yourname@frdev1001:/srv/www/org/wikimedia/listeners/SmashPig/local-config$ ls -a
Get familiar with the payment process
- After install docker successfully, go to https://localhost:9001 (if not able to open localhost in https with Chrome, enable this chrome://flags/#allow-insecure-localhost
- Make test payment with adyen or ingenico test card, it will perform auth first, in local env, we need to manually check payment from Redis, then use queue testing doc to consume queue to db to actually capture payments.
./queues-redis-cli.sh (this will log you in redis) Keys * (this will show all the keys) lrange donations 0 -1 (this will show all the jobs in donations queue) cd <roor dir> (go back to root dir) ./civicrm-drush.show (this will logi you in civi) drush --user-1 -v -r /srv/civi-sites/wmff/drupal qc (run queue consume to exec the queue to db) docker@civicrm:/srv/civi-sites/wmff/drupal$ drush --user=1 -v afqc (this is consume fraud contributions)
After consume the queue, we will be able to see the transaction from
- CiviCrm database
- payment method's back office with payment details.
How to monitor production log
ssh frlog1001 cd /var/log/remote tail -f payments-ingenico (Check out how many ingenico payments) tail -f payments-listener-smashpig (Check out payments in general which smashpig handle auth) grep <keyword> (to find the keyword from log)
What is Irc
- Register your nickname
- Set up cloak
Add google calendar and know agile meetings
- Daily Autopsy (to check if there is any blocker)
- Backlog Refinement (repeat every week to clear the backlog)
- Spring Review, retro and planing ( Repeat every 2 weeks, to plan what should we do in next spring)
- Talk Talk (Solve blocker together or sync up recent issues, one repeat every week and one repeat every two weeks)
- Analytics sync up (repeat every 2 weeks to make sure we are sync in each Quarters)
- Civi Fortnightly (Talk about Civi with whole advancement team, repeat every 2 weeks)
- Alerts Retro (text based meeting to look at recent Failmail and alerts, repeat every 2 weeks)
- some 1:1 meetings
Get familiar with local test process:
- A: queue’s testing
- B: queue Message
- c: Make payment with Redis queue monitor to observe all the transaction stepsIf smashpig has too many pending transactions, which might due to fraud, then clear them with this cmd,
cd <rootdir> ./queues-rediscli.sh MONITORTo clear the pending transactions with a certain gateway and exist for more than 1 days
docker-compose exec smashpig bash (go inside docker server first) cd /srv/smashpig/ (go to the right dir ) php Maintenance/ConsumePendingQueue.php (this put all fraud into pending table first) php Maintenance/DeleteExpiredPendingMessages.php --gateway=ingenico --days=1
- clear drush cache and system flush to clear any cache, then run the PendingTable.consume also in local source code open xdebug.ini inside config civicrm, update xdebug.remote_autostart=on then start debug.
docker@civicrm:/srv/civi-sites/wmff/drupal$ drush cc docker@civicrm:/srv/civi-sites/wmff/drupal$ drush cvapi System.flush docker@civicrm:/srv/civi-sites/wmff/drupal$ drush cvapi PendingTable.consume version=4 gateway=ingenico
Understand each dev test URLs
- Payments URL: https://localhost:9001 payments forms
- Payments http URL: http://localhost:9009 same as above, but over http (used to receive requests via the routable URL)
- Payments test routable URL: https://paymentstest1.wmcloud.org (see README.md) For Apple Pay testing
- WMF CiviCRM install URL: https://wmff.localhost:32353/civicrm CiviCRM with all of our drupal modules and custom Civi modules
- Generic CiviCRM install (based on upstream master) URL: https://dmaster.localhost:32353/civicrm Core CiviCRM
- Civicrm user/password: admin/admin
- Mailcatcher - mails sent from CiviCRM core code: http://localhost:1080 For civi, email display like the EOY
- CiviProxy URL: https://localhost:9005 do not need to worry about this now
- SmashPig IPN listener routable URL: https://paymentsipntest1.wmcloud.org (see README.md) test adyen apple pay
- E-mail Preference Center URL: https://localhost:9002/index.php/Special:EmailPreferences not working on that
- PrivateBin read-only URL: https://localhost:9007 fancy page for contributor to see their activities
- PrivateBin read-write URL: https://localhost:9008 donor activities with write access
- install git-review
- submitting changes with git-review
- Git pull from master
- Git checkout -b <newbrach>
- Make modification to the code, then use git status to check which code is been modified and will we add it or use ui to add certain change files
- Git add <right paths of the code file>
- Git commitif not satisfied, then
WIP:<comment> <more desc for this change> Bug: <Gerrit Tag Name>
- Git commit –amend (if your want to update exit with :wq)
- Git review (this part submit the code for review and auto run by Jenkins with +2 if did not break anything, then all the commit will automatically linked with Gerrit and Phabricator).
- Note: you can always reply "recheck" to your commit to let jenkins run the check again
- Modify you commit
- Beside the code you commit, you will find a button as "UPDATE CHANGE"
- Click that one, then you can find the 4 steps instruction as
- Checkout this change locally and make your desired modifications to the files: `git fetch https://gerrit.wikimedia.org/r/xxxxxx && git checkout FETCH_HEAD
- Update the local commit with your modifications using the following command: `git add . && git commit --amend --no-edit` (Leave the "Change-Id:" line of the commit message as is)
- Push the updated commit to Gerrit: `git push origin HEAD:refs/for/master`
- Refresh this page to view the the update.
- Review other’s commits
- Go to the Gerrit dashboard and find the patches you want to review
- git-review -d <tagName> to checkout that branch or from the Gerrit ui, click download, use anonymous http to checkout the new branch or just checkout to the patch branch.
- If everything is ok, leave a comment below of the Phabricator task and then use Zuul to run reviewed code with Jenkins. If passed, then +2 this code is ready to move to production.
- During debug, could use export PHP_IDE_CONFIG="serverName=civi"to enable xDebug for phpStorm (where depending on your IDE you may wish to set the environment variable PHP_IDE_CONFIG.For phpstorm it should hold a value like "serverName=wmff" - where wmff is configured as a deployment server in phpStorm. If you are using drush then the site alias will set this variable - e.g drush @wmff will run drush in the context of the wmff.localhost site, including the PHP_IDE_CONFIG.serverName key)
Update package at Packagist
- 1: Login to root dir where you have docker code, fundraising handle three main repos:
cd src/payments/extensions/DonationInterface git remote -v (if return https then convert to ssh) git remote set-url origin ssh://<reviwer>@gerrit.wikimedia.org:29418/mediawiki/extensions/DonationInterface git pull (run git remote -v again to check if remote set to ssh then pull the newest code, handle all the payment methods)
cd ~/<rootdir>/src/smashpig git pull (to get the most updated version of smaship which handle both civicrm -when donor is doing recurring payment and payments flow)
cd ~/<rootdir>/src/civi-sites/wmff git pull (get the most updated version of civicrm which manage the customer backoffice) git remote
Upgrade drupal 8 to 9 in year 2020, and in the future remove drupal just save civi
In wmff > drupal > sites> all (those are the old drupal module)
Git remote should be /crm
- 2: update tag to Packagist if necessary: (after pull most updated code from master, we have two packages which are wikimedia/smash-pig and wikimedia/donation-interface)
- git tag (to check current tag)
- git tag v.x.x.x.x (to mark current repo to this tag version)
- git push --tags (to push new tag version)
- Go to Packagist, login, find the package, then hit update
- 3: update packages then update it in code:
- docker-compose exec civicrm bash (go to local civicrm server)
- cd src/civi-sites/wmff (go to wmff repo)
- composer self-update (if necessary)
- composer update wikimedia/donation-interface wikimedia/smash-pig (must update them at the same time)
- Update the new tag version number in DonationInterface composer.json
Understand redis queue
./queues-redis-cli.sh (login to redis) Keys * (check all the keys in redis)
1: contribution-tracking -see under current session, actual payment tried how many times with detail
2: payments-init -payment in pending before it finished
3: sequence_contribution-tracking -hold the number of contribution tired
4: payments-antifraud -contain all fraud transactions
5: pending -not completed
6: donations - all complete, user redirect to thank you page (this is the donation we just made in json)
you can either use Redis desktop manager to view key value pairs or just use lrange <keyName> 0 -1 to view value with keyName.
Lint for JS
- Payment DonationInterface uses the mediawiki rules:
- Under /src/payment/extension/DonationInterface Run `npm install -D stylelint-config-wikimedia`, then you will find your esLint enabled
- CiviCrm is using civilint rule, also here is the doc civi has for phpStorm config.
Code style for PHP
- Payment DonationInterface use the MediaWiki rule:
- download MediaWiki extension jar from this link https://plugins.jetbrains.com/plugin/7439-mediawiki-support, then in phpStorm setting, select plugin -> Install Plugin from Disk -> then set from choose "mediaWiki" from code style -> php.
- CiviCrm is using Drupal rule, https://docs.civicrm.org/dev/en/latest/tools/phpstorm/
XDEBUG for phpStorm
- Go to Preferences -> PHP -> Servers, add three servers with
- path mapping: /src/payment <--> /var/www/html;
- host: localhost;
- port: 9001;
- Debuger: Xdebug.
- path mapping: /src/citi-sites <--> /srv/citi-sites;
- host: wmff.localhost;
- port: 32353;
- Debuger: Xdebug.
- path mapping: /src/smashpig <--> /srv/smashpig;
- host: localhost;
- port: 9006;
- Debuger: Xdebug.
- Edit configuration -> add PHP Remote Debug -> check Filter debug connection by IDE key
- Server: payment-wiki
- IDE Key: PHPSTORM
- Server: civi
- IDE Key: PHPSTORM
- Server: smashpit
- IDE Key: PHPSTORM
- In addition to the above, you would have to update the Xdebug web on payments and civicrm because there's a slight issue with docker on mac that affects Xdebug. Paste the following lines of code in <rootdir>/config/payments/xdebug-web.ini & <rootdir>/config/smashpig/xdebug-cli.ini xdebug.remote_autostart=on xdebug.remote_host=docker.for.mac.localhostIn <rootdir>/config/civicrm/xdebug-cli.ini
xdebug.remote_autostart=on xdebug.remote_host=docker.for.mac.localhostthen restart docker’s smashpig and payments container to enable the change
- Test payment-wiki
- add a breakpoint at /src/payments/extensions/DonationInterface/gateway_forms/Mustache.php setGateWay function
- Then select Payment-wiki as configuration
- click on the green bugs
- open localhost:9001 then click any payment links, you will see your xdebug is working
- Test civi
- open phpstorm preferences -> php -> debug -> break at first line in php scripts
- add breakpoint at src/civi-sites/wmff/drupal/sites/default/civicrm/extensions/org.wikimedia.smashpig/CRM/Core/Payment/SmashPigRecurringProcessor.php
- go to the civicrm docker container (./civicrm-drush.sh)
- run You might get the error saying that you should have the php_ide_config set up
docker@civicrm:/srv/civi-sites/wmff/drupal/sites/default/civicrm/extensions$ drush --user=1 -v -r /srv/civi-sites/wmff/drupal qc
- then inside the civicrm docker, run the serverName will be the one you have set up at phpStorm server, the one have right path mapping
- skip the first break point, then it will work
- Test smashpig
- set a breakpoint at src/civi-sites/wmff/vendor/wikimedia/smash-pig/Maintenance/ConsumePendingQueue.php execute function's new PendingQueueConsumer.
- inside the smashpig container, to go /srv/smashpig/Maintenancethen
docker@smashpig:/srv/smashpig/Maintenance$ php ConsumePendingQueue.php
Understand the error messages, permission will be added by Jeff,
e.g. Check forget me processing failed with code 1
- After check Acoustic > data -> contact list > view's each db with the email you saw from civi forget me, if no matches then just delete it.
- Use this to can delete where id is equal to the most recent id as above xxxx
- Go this page, hit execute, we should see a large number id in result.
- everything you need will be here
- Otherwise just ask your buddy to do that with you for the first time, and it’s always good to have someone else's recent deployment from Gerrit to use as ref.
- After Jenkins gives you all the good feedback, you can use this link
- Log in to the active fundraising deploy server frpm1001
- Run fundraising_code_update to refresh the deploy server's staging area
fundraising_code_update -p civicrm (if payment use payment-wiki)
- Confirm the reported commit changes match what you expect, you can inspect the contents of the project staging directory which is listed by
- Run rsync_blaster to propagate this staged code to the individual servers
rsync_blaster ALL:civicrm (if payment use payment-wiki)
- Log action into #wikimedia-fundraising by typing the below (where xyz & abc are the commits) into IRC !log Updating civicrm from xyz to abc (if payment use payment-wiki)
- Turn your stuff back on and make sure it works.
- We do civicrm upgrade if only the core got upgraded, and all you need to know is this link
- For the customized civicrm upgrade, we need to have the submodule update as well
- For payment-wiki you just need to follow the deployment link above and replace the civicrm to payment-wiki
- For fraud filter deployment, since it's private files, not in Gerrit, so just do directly to frpm1001. Nano the localSettings, update it and deploy it, once you are done, test a real world donation and see from the log if everything is going well.
- Deploy to staging site: Fundraising/Cluster/Civicrm staging server
(Usually Elliot will run the cron job, rest of the team will watch the whole FailMail):
To update these numbers (safely because this CiviCrm is the slave db):
- Run `ssh frdev1001`
- Type `mysql`
- Run `use civicrm`
SELECT year, status, COUNT(*) FROM wmf_eoy_receipt_donor GROUP BY year , status;
To see them live (using prod mysql access: it's in the .my.cnf file in your home directory)
- Run `ssh civi1001`
- Change directory to `/srv/org.wikimedia.civicrm/drupal`
- Run `drush civicrm-sql-cli`
SELECT year, status, COUNT(*) FROM wmf_eoy_receipt_donor GROUP BY year , status;
To preview an EOY email locally:
admin@admins-MBP ~ % cd ~/wiki admin@admins-MBP wiki % ./civicrm-drush.sh
Render email to preview
docker@civicrm:/srv/civi-sites/wmff/drupal$ drush @wmff cvapi EOYEmail.Render version=4 limit=1 year=2022 contactID=210
docker@civicrm:/srv/civi-sites/wmff/drupal$ drush @wmff cvapi EOYEmail.Send version=4 limit=1 year=2022 contactID=210
Go then go to localhost:1080 (MailCatcher) we will see the test mail.
Refer to this Gerrit Ticket
Update the policy inside frpm1001:/localsettings/payment-wiki/LocalSettings.php, and do the deploy:
fundraising_code_update -p payment-wiki
then go to the frlog1001:/var/log/remote to check the corespondent payment method log, then always try to make a real donation and check the log to make sure nothing is been break due to the new policy we add.
Since this is private file, which we do not have code publish to Gerrit, so there is no way to do the git review process, just ask you peer to do that with you if you are not very sure about the whole process.
- Fundraising On-Call Manual
- Fundraising - Outages Guide
- How to disable CentralNotice campaigns
- Fundraising Shutting the pipeline down
- Payment Gateway Contacts
- Fundraising Emergency Datacenter Cutover Procedure
- local test site: https://localhost:9002/index.php/Special:EmailPreferences/emailPreferences?contact_hash=2261039855&contact_id=6
- local test with json response example: https://localhost:9005/rest.php?entity=civiproxy&action=getpreferences&key=SITE_KEY&version=3&json=1&api_key=API_KEY&hash=2261039855&contact_id=6
- Queue consume files:
- Css file:
- html file:
- i18n file:
- php file:
- Cmd to consume
- drush cvapi Preferencesqueue.consume time_limit=1000 max_batch_size=1
Currently we have Ingenico and Adyen those two methods to support the Monthly Convert, which we would love to use a Modal to ask our one time donor do they want to do the monthly donation after they one time payment.
Monthly convert Ingenico local test links:
Set breakpoint @ src/payments/extensions/DonationInterface/gateway_common/RecurringConversion.api.php
FR-tech-dev cloud project to set up amazon local env
Config the payments.fr-tech-dev.eqiad1.wikimedia.cloud for local sandbox amazon testing
First add below to .ssh/config
Then add the paired public key to this site:
Then run `ssh -vv payments.fr-tech-dev` to test if it is working, other wise need to contact your team to add you account as admin
then go the project root dir, run `./proxy-forward.sh`
Then you can use the paymentstest1.wmcloud.org as the payment site to do amazon testing now
Amazon sandbox testing credential is:
email@example.com / amazon
How to watch log with logdog:
1: ssh to civi1001 or frlog1001
2: copy this logdog.sh to your home dir
3: from your home dir, run: `bash logdog.sh -d <date> <keyword>` then all the qualified gz files will go to your home-directory/logdog/<keyword>
5: go to that directory contains all the keywords, then run something like `grep -r --color=always "fraud"` (which is equivalent to ‘grep --color=always "fraud" *’)then you will be able to see all other keywords for those files.
Or without logdog, you can also zgrep keyword:
xxx@frlog1001:/srv/archive/frlog1001/logs$ zgrep --color=always <keyword> *<date>.gz, to fetch all the gz files at date xxx see if they have the keyword xxx with keyword highlighted
p.s. can use `echo "<XML XXXXX>" | xmllint --format -` to view formated xml