You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Difference between revisions of "Spicerack"

From Wikitech-static
Jump to navigation Jump to search
imported>Volans
m (fix obsolete hostname)
imported>Jobo
m
 
Line 5: Line 5:




See the [https://doc.wikimedia.org/spicerack/master/ Spicerack documentation].
== Documentation ==
See the automatically generated [https://doc.wikimedia.org/spicerack/master/ Spicerack documentation].


=== Test newly released Spicerack features ===
== How to contribute ==
 
=== Bug fixes and small feature updates ===
For any trivial changes simply send a Gerrit request for review. Context is always in demand, so we'll greatly appreciate a Phabricator task with the <code>SRE-Tools</code> tag and include some background that would help us understand your request better.
 
In case of any uncertainty or questions, feel free to contact the [[SRE/Infrastructure Foundations/Contact|SRE Infrastructure Foundations team]].
 
=== Adding new module or change in core behaviour ===
For any bigger changes like '''adding a new module or changing core behaviour''', please create a Phabricator task with <code>SRE-Tools</code> tag documenting your proposal.
 
==== The ideal proposal would include ====
 
*; A problem statement
*: Here please detail the benefits that the proposed changes will bring or the issues they will assist with.
*: In case of a new or refactored module, describe the high level API definition including:
*:* Any new data structures
*:* All affected public methods signatures with type hints to clearly describe parameters and return types
*:* The spicerack accessor(s) to be added/modified to spicerack.Spicerack (how the cookbook will access the module features)
 
*; Third party dependencies
*: Any additional third party dependency that would be required with version constraints and if they are available as Debian packages.
 
*; Additional configuration
*: Any additional configuration file and its structure.
 
==== Alternatively ====
At an early stage of a project, you might not be sure about API structure, dependencies and configuration needed - that’s fine, in such a case just create a task with a problem statement and we can backfill the proposal template as we work through the solution together.
 
This will allow the SRE Infrastructure Foundations team to provide early feedback and guidance on where best in the Spicerack source tree the code should live, or if the code fits more logically in a different library such as [[Python/Wmflib|wmflib]].
 
We can also provide feedback on any current Spicerack interfaces which may assist with the implementation, either in their current form or with minor refactoring.  Finally it allows the SRE Infrastructure Foundations team to provide guidance on how best to organise the module to ensure its interface is consistent with the current Spicerack modules.
 
=== Automation and Tooling office hours ===
Further to this all are welcome to attend the Automation and Tooling office hour to discuss new ideas or specific proposals. Please ask us on [[SRE/Infrastructure Foundations/Contact|IRC]] for more details and an invite to the office hours meeting.
 
== Test newly released Spicerack features ==
In order to test changes included in a new Spicerack release, upgrade the <code>spicerack</code> package on a single host (usually <code>cumin2002</code>) and then within a Python shell (<code>sudo -i python3</code>) run:<syntaxhighlight lang="python3">
In order to test changes included in a new Spicerack release, upgrade the <code>spicerack</code> package on a single host (usually <code>cumin2002</code>) and then within a Python shell (<code>sudo -i python3</code>) run:<syntaxhighlight lang="python3">
import logging
import logging

Latest revision as of 10:50, 22 September 2021

Spicerack is a Python library to orchestrate tasks in the Wikimedia Foundation production environment. It comes with an easy API and a cookbook entry point script that allows to write simple Cookbooks to automate and orchestrate tasks.


Documentation

See the automatically generated Spicerack documentation.

How to contribute

Bug fixes and small feature updates

For any trivial changes simply send a Gerrit request for review. Context is always in demand, so we'll greatly appreciate a Phabricator task with the SRE-Tools tag and include some background that would help us understand your request better.

In case of any uncertainty or questions, feel free to contact the SRE Infrastructure Foundations team.

Adding new module or change in core behaviour

For any bigger changes like adding a new module or changing core behaviour, please create a Phabricator task with SRE-Tools tag documenting your proposal.

The ideal proposal would include

  • A problem statement
    Here please detail the benefits that the proposed changes will bring or the issues they will assist with.
    In case of a new or refactored module, describe the high level API definition including:
    • Any new data structures
    • All affected public methods signatures with type hints to clearly describe parameters and return types
    • The spicerack accessor(s) to be added/modified to spicerack.Spicerack (how the cookbook will access the module features)
  • Third party dependencies
    Any additional third party dependency that would be required with version constraints and if they are available as Debian packages.
  • Additional configuration
    Any additional configuration file and its structure.

Alternatively

At an early stage of a project, you might not be sure about API structure, dependencies and configuration needed - that’s fine, in such a case just create a task with a problem statement and we can backfill the proposal template as we work through the solution together.

This will allow the SRE Infrastructure Foundations team to provide early feedback and guidance on where best in the Spicerack source tree the code should live, or if the code fits more logically in a different library such as wmflib.

We can also provide feedback on any current Spicerack interfaces which may assist with the implementation, either in their current form or with minor refactoring.  Finally it allows the SRE Infrastructure Foundations team to provide guidance on how best to organise the module to ensure its interface is consistent with the current Spicerack modules.

Automation and Tooling office hours

Further to this all are welcome to attend the Automation and Tooling office hour to discuss new ideas or specific proposals. Please ask us on IRC for more details and an invite to the office hours meeting.

Test newly released Spicerack features

In order to test changes included in a new Spicerack release, upgrade the spicerack package on a single host (usually cumin2002) and then within a Python shell (sudo -i python3) run:

import logging
logging.basicConfig(level=logging.DEBUG)
from spicerack import Spicerack
s = Spicerack(verbose=True, dry_run=True)  # An instance of Spicerack with dry-run set to True
# ATTENTION: sreal will make read-write changes to the infrastructure!!!
sreal = Spicerack(verbose=True, dry_run=False)  # An instance of Spicerack with dry-run set to False

At this point s and sreal are two instances of Spicerack like the ones passed to the Spicerack/Cookbooks with the dry-run mode set to True and False respectively.