Don't stop the world

Thanks to advances in device technology, it becomes possible to use abundant hardware resources such as many core CPUs and several hundreds Giga- or Tera-bytes of memory, which is an amount of computing resources that can not be imagined only a decade ago, and thus software development has to move towards utilizing such hardware resources. In spite of this situation, PostgreSQL that is a stable DBMS for, at least has an architecture suitable for non-rich hardware, which may cause a performance bottleneck in a DB server with contemporary hardware. In particular, scalability bottleneck, which has a large adverse effect when increasing the number of CPU cores, is a serious problem. Typical one is lock contention that becomes prominent when the number of processes trying to execute a critical section protected by the lock increases; when severe lock contention occurs, it's like stop-the-world as most of transaction processing stop waiting for the critical section becomes available..I have identified that some lock contention can be resolved at the cost of increasing memory usage and other lock contention can be mitigated by prepare-in-advance strategy. Among the lock contention that occurred in a benchmark execution, that caused by CLogControlLock is the former type and is easily resolved. The increase in the memory usage is acceptable for contemporary DB server which has a large amount of main memory. On the other hand, typical bottleneck categorized as the latter type is that caused by the critical section for extending a particular relation. Simple countermeasure that the author implemented was not effective for performance improvement as it caused a side effect. Some architectural change seems necessary for PostgreSQL in order to resolve the bottleneck. In this lecture, I will describe the outline and countermeasures of these bottlenecks, their effects, and necessary architectural change in PostgreSQL.