Fundraising
This is the homepage for Fundraising-tech documentation . If you can't find what you are looking from from here then take a look at our documentation plan and add the appropriate header & links.
- If you want to understand what fr-tech does, who we are and how to contact us head over to mw:Fundraising tech
External-facing
Mediawiki extensions
The following Mediawiki extensions related to fundraising are installed on the payments wiki :
DonationInterface
Renders donation forms and handles donor interaction, redirecting donors to payment processors when necessary and then either presenting an error or redirecting to a thank you page.
Extension documentation on mediawiki.org
FundraisingEmailUnsubscribe
Allows a donor to unsubscribe from fundraising-related emails.
Extension documentation on mediawiki.org
Central Notice
Extension documentation on mediawiki.org
Fundraiser Landing Page
Extension documentation on mediawiki.org
Landing Check
Extension documentation on mediawiki.org
Public reporting (frdata.wikimedia.org)
We export some extremely aggregated datasets in CSV format at
https://frdata.wikimedia.org/
. They are generated once daily (previously every half hour) by the
public_data_export process-control
job running the
fundraiser_public_data_export
and
fundraiser_public_data_mover
child jobs.
fundraiser_public_data_export
runs the
FundraiserStatisticsGen/fundstatgen.py script from the tools repo
, then
fundraiser_public_data_mover
just rsyncs the data to the frdata server.
Internal-facing
WMF fundraising uses CiviCRM to track donor data.
There is a guide to dealing with outages and upgrades .
We manually send these in early January, summarising all donations for the year, broken into Endowment & Foundation. See technical details
We get data from HEP with information on companies that match donations.
Libraries
Service Providers
Minfraud
Maxmind geolocation
Currency rates
Employer matching gifts
Cluster layout, deployment, codebases
Analytics
Jobs for analytics processing run on
fran1001
. The primary database server for analytics data is
frdb1003
. Jobs include:
- DjangoBannerStats , a legacy python codebase that ingresses data about banner impressions and Donate Wiki landing page impressions to the pgehres database. For details, see DjangoBannerStats documentation.
Staging server
Grafana
- Located at frmon.wikimedia.org
- Routes donors to the correct payment processor based on the country, currency, and payment method.
-
Incoming links from donatewiki and banners hit this special page on paymentswiki rather than going directly to individual gateways
- This allows us to disable a gateway on paymentswiki and have traffic flow immediately to the backup
We have the ability to use several payment processors for online donations. Currently, we route most credit card donations to Adyen.
- Adyen - Main credit card processor, also has iDEAL, ApplePay, and GooglePay
- Amazon - A widget on our page, integrated using Login and Pay with Amazon.
- dLocal - A payment processor specializing in the local payment methods of South America and India
- Ingenico - Ingenico has the ability to handle payments from multiple international systems including: credit card, direct debit, real time bank transfer, eWallets and more. We're currently only using them for cards.
See also, Ways to Give for our recommended donation methods according to country.
Payment Methods
Some methods are shared across processors:
- Apple Pay
- Venmo
Acoustic is the service we use to send out bulk emails. They are able to handle high volumes of emails and are responsible for managing server reputation to improve deliverability. Acoustic also provide tools for A-B testing to see which emails perform better. In order to be able to use our donor information from Acoustic we have a nightly upload job. We also re-import information from acoustic - for details go to /Data and Integrated Processes/Acoustic Integration
Note that prior brandings of the Acoustic platform may still linger - ie Silverpop, WCM, Watson Campaign Manager, or sometimes just 'IBM'
In 2024 we introduced SMS solicitation via the remind me later form. The process is described at /Data and Integrated Processes/SMS
See Process control README for details on scheduling and running jobs.
Monthly convert is a pop up that appears after a one time donation asking if the donor would like to set up a recurring donation.
Audits are our way of checking the data at the payment processor with our data in civi. Depending on the payment processor we may also have IPNs to triple check the data. They also are how refunds and chargebacks get back to us.
Details are in Fundraising/Data and flow/Audits - including information about how we use these to drive our Intact integration
So many different ways to identify transactions!
Fraud filtering
Refer to docs at https://collab.wikimedia.org/wiki/Fundraising/Engineering/Fraud
Fail mail refers to the emails received by fr-tech when processes fall over. Diagnostic details are in the Failmail zoo
How we work (Team Processes)
Fundraising Tech is a Scrum team. We use Phabricator to manage our backlog and we work in 2-week sprints.
Fundraising Emergencies
Fundraising Engineering Documentation has with system information and emergency response protocols. Or more specifically Shutting the pipeline down details how/when to disable banner campaigns or other fundraising/payment services.
Fundraising On-Call documentation
See on-call documentation for instructions for on-call duty.
Feature / Bug Trackers
There's loads of information about how fr-tech triages bugs at Fundraising/Bug Triaging Not sure what to do next? See Fundraising Tech's Phabricator Workboard
PCI Compliance
Payment Card Industry rules we have to follow to keep accepting credit cards.
Daily standup
We have a browser extension that generates a randomly-ordered list of Google Meet participants for standup.
Development Tools
Getting Started
Local setup for cluster SSH access
Docker
Testing locally
Gerrit
Gitlab
CI
Logdog helps you hunt through the logs.
PHPStorm / Intellij
This section is for configuration notes for when we lose our PHPStorm config :-) (like erm today - thanks JetBrain!)
Tip 1 - make sure that if you have symlinks to your project directory you do not use them when opening your project...
Tip 2 - the python plugin can mess with your ability to run unit tests through the IDE
UML diagrams
https://gitlab.wikimedia.org/repos/fundraising-tech/fr-tech-diagrams
All Subpages
- AI
- Annual PCI form
- Bug Triaging
- CiviCRM Reporting
- Cluster/Civicrm staging server
- Cluster/Deployments
- Cluster/DjangoBannerStats
- Cluster/Grafana
- Cluster/Logging
- Cluster/PCI
- Cluster/Payments staging server
- Cluster/Process control
- Cluster/Puppet
- Cluster/Redis
- Cluster/Upgrades
- ContributionTracking
- Data and Integrated Processes/Acoustic Integration
- Data and Integrated Processes/Acoustic Integration/Scheduled Jobs
- Data and Integrated Processes/Acoustic Integration/Snooze
- Data and Integrated Processes/SMS
- Data and flow
- Data and flow/Audits
- Data and flow/Database cheatsheet
- Data and flow/Database schemas
- Data and flow/Exchange rates
- Data and flow/Failmail zoo
- Data and flow/Fraud filtering
- Data and flow/Geolocation
- Data and flow/IPN listener
- Data and flow/Intacct
- Data and flow/Monthly convert
- Data and flow/PSP integrations
- Data and flow/PSP integrations/Adyen Checkout
- Data and flow/PSP integrations/Amazon
- Data and flow/PSP integrations/Braintree
- Data and flow/PSP integrations/Gateway Chooser
- Data and flow/PSP integrations/Gravy
- Data and flow/PSP integrations/Ingenico
- Data and flow/PSP integrations/PayPal Express
- Data and flow/PSP integrations/dLocal
- Data and flow/Payment methods/ACH
- Data and flow/Payment methods/Apple Pay
- Data and flow/Pending transaction resolver
- Data and flow/Queues
- Data and flow/Queues/Queue performance log
- Data and flow/Queues/Testing
- Data and flow/Recurring
- Data and flow/Stats pipeline
- Data and flow/Transaction IDs
- Data and flow/URL parameters
- Development tools/Docker
- Development tools/Fundraising-dev
- Development tools/HTML Email Development
- Development tools/Logdog
- Development tools/Testing
- DonationForm
- External-facing/CiviProxy
- External-facing/Donor Data Delivery Tool
- External-facing/Donor Portal
- External-facing/E-mail preference center
- External-facing/Payments
- Internal-facing/CiviCRM
- Internal-facing/CiviCRM/CiviCRM tokens
- Internal-facing/CiviCRM/ContactMatching
- Internal-facing/CiviCRM/Coworker
- Internal-facing/CiviCRM/Create Custom Formbuilder pages to monitor tables in other WMFF Databases
- Internal-facing/CiviCRM/Creating extensions in Civicrm to manage tables in other WMFF Databases
- Internal-facing/CiviCRM/Debugging deadlocks
- Internal-facing/CiviCRM/Exporting FormBuilder Pages into an extension
- Internal-facing/CiviCRM/Exporting SearchKit Search into an extension
- Internal-facing/CiviCRM/Imports
- Internal-facing/CiviCRM/SearchKit
- Internal-facing/CiviCRM/Triggers
- Internal-facing/Donor Portal
- Internal-facing/E-mail preference center
- Internal Endpoints/CiviCRM extensions
- Internal Endpoints/Drupal modules
- Internal Endpoints/End of year emails
- Internal Endpoints/Matching gifts
- Normalized donation messages
- Queue wrangling
- RFC/Abstract CRM import
- RFC/Direct Mail Appeal campaign management
- Roadmap
- SAL
- Service providers/Fundraise up
- SmashPig
- Syslog Settings
- Team processes
- Team processes/Campaign Readiness Checklist
- Team processes/Chaos crew handover
- Team processes/DRI
- Team processes/DRI/Chores
- Team processes/DRI/Rotation
- Team processes/Definition of Done
- Team processes/How we use Phabricator
- Team processes/Mid-Sprint Checkup
- Team processes/New integration manual
- Team processes/On-call guide
- Team processes/Onboarding
- Team processes/SprintLog
- Team processes/Sprint Retro
- Team processes/Tech talks
- Team processes/Training
- Translation
- analytics
- tech
- tech/Alerts
- tech/Apache httpd
- tech/Coinbase
- tech/Currency conversion sample
- tech/Installing Drupal and CiviCRM
- tech/Spikes
- tech/Transformer
- tech/access
- tech/crm
- tech/crm/Add a New Gateway
- tech/crm/Quirks
- tech/crm/Running tests
- tech/crm/Upgrade CiviCRM
- tech/crm/Upgrade Drupal
- tech/deployments
- tech/git
- tech/payments1004
- tech/ssh config
- techops
- techops/docs/analytics stack
- techops/docs/analytics stack/dbt
- techops/docs/analytics stack/dlt
- techops/docs/analytics stack/env vars
- techops/docs/analytics stack/how to guides
- techops/docs/analytics stack/pipelines
- techops/docs/analytics stack/troubleshooting
- techops/docs/dependencies-production
- techops/docs/frack ssh access
- techops/docs/fundraising access request
- techops/docs/fundraising cluster deploy
- techops/docs/ssl client authentication
- techops/procedures/servers-decommissioning
- techops/procedures/servers-out of band management access
- techops/procedures/servers-procurement and install
- techops/procedures/servers-role-build
- techops/procedures/servers-roles
- techops/procedures/servers-server build standard
- techops/procedures/servers-server policy
- techops/procedures/servers-standard software vulnerability management
- techops/procedures/services-analytics datacenter swap
- techops/procedures/services-backups
- techops/procedures/services-civicrm database trigger updates
- techops/procedures/services-civicrm staging db refresh
- techops/procedures/services-datacenter cutover procedure
- techops/procedures/services-kafkatee
- techops/procedures/services-kerberos kdc install
- techops/procedures/services-kerberos new replica setup
- techops/procedures/services-kerberos user management
- techops/procedures/services-lets encrypt setup
- techops/procedures/services-mariadb clone
- techops/procedures/services-mariadb mariabackup
- techops/procedures/services-mariadb primary swap
- techops/procedures/services-mariadb tzinfo
- techops/procedures/services-metabase upgrade
- techops/procedures/services-minio upgrade
- techops/procedures/services-payments-settings
- techops/procedures/services-redis-frqueue
- techops/procedures/services-superset upgrade
- techops/procedures/services-system-authentication
- techops/procedures/services-trino upgrade
- techops/procedures/users-access roles
- techops/procedures/users-departing user offboarding checklist
- techops/procedures/users-fr tech managed accounts
- techops/procedures/users-new user checklist
- techops/procedures/users-ssl client certs yearly renewal