Skip to content
Runbooks

AI-Assisted Service

Logging

Section titled “Logging”

Summary

Section titled “Summary”

AI Assisted is a dedicated Rails fleet that provides an AI Abstraction Layer in front of AI Gateway. It handles AI-specific requests that are often long running and depend on external services (such as Vertex AI and Anthropic). To prevent these requests from occupying Puma workers on the main fleet, potentially impacting performance, AI Assisted runs on an isolated fleet to safely serve these endpoints without affecting core GitLab traffic.

Architecture

Section titled “Architecture”

For diagram, refer to architecture blueprint.

Operational Roles and Responsibilities

Section titled “Operational Roles and Responsibilities”

Currently, all traffic served by the AI Assisted service supports Code Suggestions features, such as Code Generation and Completion, which are owned by the Code Creation Group. However, if additional feature teams begin contributing endpoints under /api/v4/ai_assisted in the future, service ownership may need to be reassessed.

The service is implemented across the following projects:

Deployment

Section titled “Deployment”

AI-Assisted Service is deployed in:

  • Staging
  • Production

Regions

Section titled “Regions”

AI-Assisted Service is currently deployed across 3 pods in the following regions:

  1. us-east1-b
  2. us-east1-c
  3. us-east1-d

Performance

Section titled “Performance”

AI-Assisted Service includes the following SLIs/SLOs:

Section titled “Links to further Documentation”