Gitaly Service

• Service Overview
• Alerts: https://alerts.gitlab.net/#/alerts?filter=%7Btype%3D%22gitaly%22%2C%20tier%3D%22stor%22%7D
• Label: gitlab-com/gl-infra/production~“Service::Gitaly”

Logging

• Gitaly
• gitlab-shell
• system

Architecture

Chef

gprd

graph TD
    gprd-base-stor-gitaly-common-->gprd-base-stor-gitaly-cny
    gprd-base-stor-gitaly-common-->gprd-base-stor-gitaly
    gprd-base-stor-gitaly-->gprd-base-stor-gitaly-hdd
    gprd-base-stor-gitaly-->gprd-base-stor-gitaly-marquee

• gprd-base-stor-gitaly-common: The base configuration which is common for all Gitaly shards and stages.
• gprd-base-stor-gitaly-cny: Any extra configuration we might want to add for Gitaly shard=default, stage=cny.
• gprd-base-stor-gitaly: Any extra configuration we might want to add for Gitaly shard=default, stage=main.
• gprd-base-stor-gitaly-hdd: Any extra configuration we might want to add for Gitaly shard=hdd, stage=main.
• gprd-base-stor-gitaly-marquee: Any extra configuration we might want to add for Gitaly shard=marquee, stage=main.

gstg

graph TD
    gstg-base-stor-gitaly-->gstg-base-stor-gitaly-cny
    gstg-base-stor-gitaly-->gstg-base-stor-gitaly-hdd

• gstg-base-stor-gitaly: The base configuration which is common for all Gitaly shards and stages.
• gstg-base-stor-gitaly-cny: Any extra configuration we might want to add for Gitaly shard=default, stage=cny.
• gstg-base-stor-gitaly-hdd: Any extra configuration we might want to add for Gitaly shard=main, stage=main.

Monitoring/Alerting

GitalyServiceGoserverTrafficCessationSingleNode Playbook

GitalyOOMKillLoop Playbook: Gitaly on one node is OOM-killed and restarted repeatedly; do not restart the VM, find the amplifier.