You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Logstash/Extended Retention

From Wikitech-static
Jump to navigation Jump to search

Policy

See Privacy Policy and Data Retention Guidelines.

Request Process

  1. Ensure the log stream you want to preserve has only non-personal information not associated with a user account.
  2. Create a request to increase retention for a log stream on Phabricator, tagging the Observability team.
  3. Once the audit is complete and approved, the log stream will be tagged for inclusion into long-term retention indexes according to cluster capacity.

Audit Process

  1. Audit the log stream to determine it is non-personal information not associated with a user account.
  2. If in doubt, consult with the Security and/or Legal teams. Notify these teams when necessary.

Common Logging Schema fields indicating PII

These fields (as of ECS 1.7.0) have been identified as likely containing personal information.

  • client.*
  • error.message
  • geo.*
  • http.request.body.content
  • http.response.body.content
  • http.headers.*
  • labels
  • log.original
  • message
  • source.*
  • tls.client.*
  • user.*
  • user_agent.*