PGCon2016 - 20180510
PGCon 2016
The PostgreSQL Conference
| Speakers | |
|---|---|
|
Alexander Korotkov |
| Schedule | |
|---|---|
| Day | Talks - Day 1 - 2016-05-19 |
| Room | DMS 1120 |
| Start time | 14:00 |
| Duration | 00:45 |
| Info | |
| ID | 920 |
| Event type | Lecture |
| Track | Hacking |
| Language used for presentation | English |
PostgreSQL extendability: Origins and new horizons
Towards pluggable storage engines
Postgres was initially designed to support access methods extendability. Well known citation about access methods in Postgres claims: "It is imperative that a user be able to construct new access methods to provide efficient access to instances of nontraditional base types" Michael Stonebraker, Jeff Anton, Michael Hirohama. Extendability in POSTGRES, IEEE Data Eng. Bull. 10 (2) pp.16-23, 1987
Initially, heap was just one of access methods as well as btree and hash indexes were. So, extendability of access methods implies pluggable storage engines support in modern terms. For now, only index access methods are defined in pg_am table of system catalog. Those index access methods also have well-defined interface. Therefore in order to meet initial design PostgreSQL need to support two features:
- Pluggable index access methods, i.e. ability to implement new index types by adding new tuples to pg_am;
- Pluggable storage engines, i.e. ability to implement completely different storages for tables without traditional heap.
Besides mechanical work like "CREATE ACCESS METHOD" command, extensible index access methods needs to be WAL-logged. For now, community doesn't want extensions to define their own WAL-records, because there is a chance to break both recovery and replication, and that is not acceptable. Another approach is to define generic WAL-records, that specify a difference between pages in generalized way.
There are only few DBMS which support pluggable storage engines now. MySQL is the most common example here. However, dealing with different storage engines in MySQL is like dealing with different DBMSes. This is not the way PostgreSQL should go in my view.
However, now PostgreSQL users realize benefits from other storage engines. Ideas of columnar and in-memory storage engines for PostgreSQL are very popular. Simultaneously, technical possibilities to implement them are growing. Writable FDW and custom nodes are arrived. Generic WAL and extensible index access methods are pending for 9.6. Much work in the direction of pluggable storage engines is already done if even it had different aims.
It's time for PostgreSQL core developers to think about native support of pluggable storages without kludges. Finally, we should get "CREATE STORAGE ENGINE name ..." command as legal extendability mechanism.
In this talk I will show current state on pluggable index access method and design of pluggable storage engines.