You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Fundraising/Fundraising-dev: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Cstone
imported>Cstone
 
Line 1: Line 1:
Fundraising-dev is a set of docker images coordinated by docker-compose configuration and a setup script. Its purpose is to let all members of the Fundraising Tech team develop against the same setup of PHP, mediawiki, and CiviCRM, along with their ancillary tools and databases. It also aims to speed team onboarding by making the install process as quick and painless as possible.
#REDIRECT [[Fundraising/Development tools/Fundraising-dev]]
 
=Quickstart=
* Clone the repo from https://gerrit.wikimedia.org/r/wikimedia/fundraising/dev
* Run setup.sh
 
=Detailed Start=
Things you need before starting:
1. Docker installed
2. Your SSH key added to gerrit
3. A yubikey set up and access to frpm
 
 
1. clone above
 
2. Install docker https://docs.docker.com/engine/install/ubuntu/
 
3. Install docker-compose https://docs.docker.com/compose/install/
 
4. If you get an error <code>docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))</code>
Add your user to the docker group <code>sudo gpasswd -a $USER docker
newgrp docker</code>
https://stackoverflow.com/questions/64662372/docker-compose-up-error-while-fetching-server-api-version-connection-aborte
 
4.5 Add your SSH key to gerrit https://www.mediawiki.org/wiki/Gerrit/Tutorial#Set_Up_SSH_Keys_in_Gerrit
If this isn't added ./setup.sh will fail with permission errors:
 
5. If the gitclone steps fail because of a permission error
<code>fatal: Could not read from remote repository.
 
Please make sure you have the correct access rights
and the repository exists.</code>
 
=Components installed=
==CiviCRM==
Two sites are created. 'wmff' is meant to resemble as close as possible the CiviCRM instance installed in WMF production, with all of our custom Civi extensions and drupal modules. 'dmaster' is built directly on core CiviCRM code and is meant to facilitate sending patches upstream.
 
==Payments-wiki==
The fundraising branch of MediaWiki core, with all the same extensions that are deployed to payments.wikimedia.org, and a similar configuration. The default appeal from production (text shown next to the donation form) is imported during setup to make forms look as similar to production as possible. The main page has handy links to payment forms for various combinations of country, currency, and payment method.
 
==Email Preference Center==
Another MediaWiki instance on the same branch as payments-wiki but with different configuration. Payment processor forms are not enabled, but email preference forms are, and networking is configured to allow tightly controlled access to some CiviCRM data.
 
==CiviProxy==
An API proxy that allows tightly controlled access to CiviCRM data. We use it to mediate between the Email Preference Center and the CiviCRM database so that only one specific API request is allowed, and that the request's parameters are validated before being forwarded to CiviCRM. ([https://github.com/systopia/CiviProxy CiviProxy Github site])
 
==IPN listeners==
A standalone [[Fundraising/SmashPig|SmashPig]] install, with Apache configured to send requests to the smashpig_http_handler endpoint. In payment processor sandbox configuration pages you can request IPNs be sent to e.g. https://paymentsipntest1.wmcloud.org/adyen (see FR-tech-dev cloud project, below).
 
==PrivateBin==
Used to comply with donor data requests. We have a donor-facing URL that only allows downloading and an internal-facing URL that allows uploading.  ([https://privatebin.info/ PrivateBin project site])
 
==Mailcatcher==
A development tool to catch any outbound mail sent by the other services and show it in a web UI.
 
==Redis==
Used as a queue server to allow communication between other components, as in production
 
==MariaDB==
Database server, as in production
 
=Running stuff on the docker VMs=
TODO
 
=FR-tech-dev cloud project=
This is a [[Help:Cloud Services Introduction|Wikimedia Cloud Services]] project with a single VM that primarily serves to allow access to our development machines via routable URLs (similar to the commercial service NGrok). Payment methods such as Apple Pay and Amazon Pay require forms to be served on URLs that can be crawled by the controlling entities. For any payment processor, testing IPNs requires a routable URL.
 
The setup.sh script has you select a 'proxy forwarding ID' - this ID determines which URLs and ports are reserved for you. To connect the routable URLs to your local machine, run <pre>./proxy-forward.sh</pre> in your local fundraising-dev root directory.
 
==Domain Names and forwarding==
The VM has twelve domain names pointing to it via the WMCS https web proxy. paymentstest1.wmcloud.org through paymentstest6.wmcloud.org are for payments forms, and paymentsipntest1.wmcloud.org through paymentsipntest6.wmcloud.org are for IPN testing.
 
The IPN test domains are configured to point directly to ports that are served from developer machines. The paymentstest domains are served via nginx, which serves the .well-known directory from the VM itself and forwards any other requests to developer machines. This is necessary so that the .well-known files are available any time that Apple crawls the site to verify our ownership of the domains.
 
==Setup and Administration==
The project is configured via [https://horizon.wikimedia.org/ Horizon]. Log in with your Wikitech credentials and select the 'fr-tech-dev' project from the dropdown at top left. If not available, ask another fr-tech member to add you to the project.
 
To allow port forwarding, gateway_ports is enabled in ssh server settings. In Horizon, under Compute->instances->'payments'->Puppet Configuration, the Heira Config includes
<pre>
profile::base::ssh_server_settings:
  gateway_ports: true
</pre>
 
Proxied domain names are configured under DNS->Web Proxies. paymentstest1-6 proxy to ports 8051-8056 on the payments VM, and paymentsipntest1-6 proxy to ports 8101-8106 on the payments VM.
 
Apple Pay domain verification files are stored in /var/www/applepay/paymentstest[1-6]/.well-known directories.
 
Ngnix configuration to serve .well-known locally and forward other requests is as follows (example for paymentstest1):
<pre>
server {
        listen 8051;
        server_name paymentstest1.wmcloud.org;
        include custom_502;
 
        location /.well-known {
                alias /var/www/applepay/paymentstest1/.well-known;
        }
 
        location / {
                proxy_set_header Host $host;
                proxy_pass http://localhost:8001;
        }
}
</pre>
The custom_502 config file serves a default page when no development machine is answering on the proxied port:
<pre>
error_page 502 =200 /error-502.html;
 
location /error-502.html {
internal;
root /var/www/fallback;
}
</pre>
 
=Images=
Dockerfiles for the necessary images are checked into the Release Engineering team's [https://gitlab.wikimedia.org/repos/releng/dev-images dev-images repo], hosted on the WMF gitlab instance. After you get a change merged into the dev-images repo, you may have to ask a member of release engineering to compile the image and upload it to the WMF docker registry.
 
* fundraising-buster
* fundraising-buster-php73
* fundraising-civicrm-buster-php73
* fundraising-civicrm-buster-php73-apache2
* fundraising-civiproxy-buster-php73-apache2
* fundraising-mediawiki-buster-php73-apache2
* fundraising-privatebin-buster-php73-apache2
* fundraising-smashpig-buster-php73-apache3

Latest revision as of 22:36, 4 May 2022

Redirect to:

Retrieved from "https://wikitech-static.wikimedia.org/w/index.php?title=Fundraising/Fundraising-dev&oldid=590253"