Skip to content
Runbooks

Remove Blobs

The Remove Blobs feature permanently deletes blobs that contain sensitive or confidential information.

Sequence

Section titled “Sequence”

  1. Blobs are removed by the RewriteHistoryWorker asynchronously in Sidekiq. Job logs can be found in Kibana.

  2. RewriteHistoryWorker calls RewriteHistoryService, which puts the project in a read only state before executing the call to rewrite history. Ongoing pushes to the repository may still go through, potentially causing remove blobs to fail with a source repository checksum altered error.

  3. This worker calls the rewrite_history Gitaly RPC. Gitaly logs can be found using the correlation dashboard. Use the correlation_id found in Sidekiq job logs.

  4. After this job completes, objects are left in a dangling state (not attached to any tags or branches). These can be cleaned up by running housekeeping, and pruning unreachable objects. These steps are documented in the Remove Blobs documentation.

Removing blobs on previously forked projects

Section titled “Removing blobs on previously forked projects”

A known issue exists when removing blobs for projects that have been previously forked, and the offending blobs are still part of the old object pool.

Diagnostics and a workaround is documented in this issue.

Contacting the team

Section titled “Contacting the team”

Remove Blobs is owned by Create:Source Code Management.

Requests for help can be submitted using the source code group template.

Urgent, or less formal requests can be made directly on Slack in one of our team channels: