You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Data Catalog Application Evaluation Rubric/data-catalog-evaluation server notes

From Wikitech-static
< Data Catalog Application Evaluation Rubric
Revision as of 23:28, 17 December 2021 by imported>Razzi (→‎Data Catalogs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

All files are currently in /home/razzi on data-catalog-evaluation.analytics.eqiad1.wikimedia.cloud.

Docker setup

Install docker: 

sudo apt-get install docker.io

Download docker-compose and move it to ~/.docker/cli-plugins to install it: 

sudo mkdir -p /usr/libexec/docker/cli-plugins/
wget https://github.com/docker/compose/releases/download/v2.2.2/docker-compose-linux-x86_64
sudo cp docker-compose-linux-x86_64 /usr/libexec/docker/cli-plugins/docker-compose

Nginx setup

Run nginx docker container in foreground (this will be served at data-catalog-evaluation.wmcloud.org): 

sudo docker run -it -p 80:80 nginx

Run nginx docker container as daemon: 

sudo docker run -d -p 80:80 -v $(pwd):/usr/share/nginx/html nginx
Contents of /home/razzi/static/index.html
The following content has been placed in a collapsed box for improved usability. 
<html>
<style>
body {
font-family: arial;
}
</style>
Welcome to <a href="/">data-catalog-evaluation.wmcloud.org</a>

<h2>Links:</h2>

<ul>
<li> <a href="#">Atlas (TODO)</a>
</ul>

<p><a href="https://wikitech.wikimedia.org/wiki/Data_Catalog_Application_Evaluation_Rubric">Read about the data catalog evaluation</a></p>

<p>
This file lives at data-catalog-evaluation.analytics.eqiad1.wikimedia.cloud:/home/razzi/static/index.html and the author (me, Razzi) gives you permission to edit it, wiki-style.
</p>
</html>
The above content has been placed in a collapsed box for improved usability.

Data Catalogs

Atlas

Clone apache atlas docker image (currently at /home/razzi/apache-atlas-docker) 

git clone https://github.com/sonnyhcl/apache-atlas-docker

Run atlas: 

sudo docker compose up

It is accessible from localhost (curl -u admin:admin http://localhost:21000/api/atlas/admin/version works) but not via the web proxy at atlas-demo.wmfcloud.org. This works with ssh forwarding:

ssh -NL 21000:data-catalog-evaluation.analytics.eqiad1.wikimedia.cloud:21000 data-catalog-evaluation.analytics.eqiad1.wikimedia.cloud

but it does not work with the proxy (configured as shown)

Highlighted text shows atlas-demo.wmcloud.org http://172.16.3.184:21000
Proxy looks to be configured properly but is not working.

Amundsen

TODO

DataHub

TODO

Egeria

TODO User:milimetric is working on this

Marquez

TODO

Data Sources

Retrieved from "https://wikitech-static.wikimedia.org/w/index.php?title=Data_Catalog_Application_Evaluation_Rubric/data-catalog-evaluation_server_notes&oldid=585494"