Schedule - PGCon 2022

Back

Advanced database testing in CI/CD pipelines

Date: 2022-05-26
Time: 09:00–09:45
Room: Stream 1
Level: Intermediate

Usually, CI/CD pipelines involve stateless components – nobody likes to wait dozens of minutes or hours while a pipeline is running. So in most cases, the maximum volume of data generated or copied during testing is limited to gigabytes.

In this talk, we will explore an unusual but very promising land: testing using thin clones. Database Lab Engine (DLE), developed by Postgres.ai, is an open-source tool that allows to:

  • clone Postgres database of any size in just a few seconds using copy-on-write provided by ZFS or LVM, and
  • run dozens of thin clones on a single machine, performing dozens of independent experiments, development, and testing activities.

These abilities open new horizons. Now it is possible to:

  • have realistic tests using "recycled" full-size database clones,
  • perform regression testing to see if some business-critical SQL query is going to have performance degradation after a proposed change,
  • reach 100% coverage for automated testing of all DB changes using full-size databases.

I will present the most recent findings and experience of building CI/CD pipelines involving advanced DB testing in a fully automated fashion. I will also provide practical advice, covering various aspects of observability, security, and change management workflow.

Slides

The following slides have been made available for this session:

Speaker

Nikolay Samokhvalov