Skip to content

Add PgBouncer search_path as default ignored startup parameter

Stan Hu requested to merge sh-ignore-search-path-pgbouncer into master

What does this MR do?

If a user attempts to backup GitLab through PgBouncer, this can cause a full site outage since pg_dump will alter the PostgreSQL search path, as documented in https://docs.gitlab.com/ee/administration/postgresql/pgbouncer.html#backups and https://docs.gitlab.com/ee/administration/backup_restore/backup_gitlab.html#back-up-and-restore-for-installations-using-pgbouncer.

To avoid causing an outage, make PgBouncer ignore the search_path startup parameter by default.

Related issues

Relates to https://github.com/pgbouncer/pgbouncer/issues/89

ZD: https://gitlab.zendesk.com/agent/tickets/518992

Testing

Before this patch

  1. Configure PgBouncer via https://docs.gitlab.com/ee/administration/postgresql/pgbouncer.html.
  2. In pgbouncer['databases'], set pool_size to something low, like 2.
  3. Also make sure pool_mode: 'transaction' is set for pgbouncer['databases'].
  4. Make sure GitLab Rails is talking to PgBouncer (/var/opt/gitlab/gitlab-rails/database.yml points to port 6432, not 5432).
  5. Run sudo gitlab-backup create SKIP=repositories.
  6. When the backup is done, attempt to sruf GitLab. You should see 500 errors with these errors in /var/log/gitlab/gitlab-rails/production.log, and this can't be recovered without a restart of Rails or PostgreSQL.

After this patch

You should not see this issue after the patch.

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • MR title and description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • Latest Merge Result pipeline is green.
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

  • The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
  • If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
  • If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
  • If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes.
  • Documentation created/updated.
  • Tests added.
  • Integration tests added to GitLab QA.
  • Equivalent MR/issue for the GitLab Chart opened.
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by Stan Hu

Merge request reports