User:Cwhite/OpenSearch

From Wikitech-static
Jump to navigation Jump to search

OpenSearch is the core data storage component of the Observability Logging stack. It replaced ElasticSearch and Kibana in 2021.

Building

Easiest path to building locally is to use a container on a Debian Bullseye WMCS instance >= g3.cores2.ram4.disk20

Environment

# Install podman and rootlesskit
sudo apt update && sudo apt install -y podman rootlesskit
# Add user subuid/gid settings
echo $(whoami):100000:65536 | sudo tee -a /etc/subuid
echo $(whoami):100000:65536 | sudo tee -a /etc/subgid
# Add registry
echo 'unqualified-search-registries = ["docker.io"]' | sudo tee -a /etc/containers/registries.conf
# Reboot (https://github.com/containers/podman/issues/3890)
sudo reboot
# Once back up and logged in, enable unprivileged_userns_clone
sudo sysctl kernel.unprivileged_userns_clone=1

OpenSearch

# Clone repo
git clone https://github.com/opensearch-project/OpenSearch.git
# Enter OpenSearch repo
cd ~/OpenSearch
# Checkout 1.0.0
git checkout tags/1.0.0
# Give podman permissions on source dir
podman unshare chown -R 1000:1000 .
# Enter the container
podman run -it --rm --user gradle --net host --workdir /home/gradle/opensearch -v $(pwd):/home/gradle/opensearch:Z gradle:jdk11 /bin/bash

#  Build
./gradlew -Dbuild.snapshot=false distribution:packages:deb:build
#  Exit the container
exit

# Move the compiled deb package to home dir
sudo mv distribution/packages/deb/build/distributions/opensearch*.deb ~/
# Optionally change ownership to yourself
sudo chown $(id -u):$(id -g) ~/opensearch*.deb

OpenSearch Dashboards

# Clone repo
git clone https://github.com/opensearch-project/OpenSearch-Dashboards.git
cd ~/OpenSearch-Dashboards
# Checkout version 1.0.0
git checkout tags/1.0.0
# Give podman permissions on source dir
podman unshare chown -R 1000:1000 .
# Enter the container
podman run -it --rm --user root --net host -v $(pwd):/home/node/osd:Z node:10.24.1-buster /bin/bash

#   Install dependencies
apt update && apt install -y ruby ruby-dev rubygems build-essential
gem install --no-ri --no-rdoc fpm
#   Switch to unprivileged node user
su - node
#   Change to build dir
cd /home/node/osd
#   Bootstrap
yarn osd bootstrap
#   Build
node scripts/build --deb --all-platforms --release
#   Exit the container
exit

# Move the compiled deb package to home dir
sudo mv target/opensearch-dashboards*.deb ~/
# Optionally change ownership to yourself
sudo chown $(id -u):$(id -g) ~/opensearch-dashboards*.deb