How to partition existing table in postgres?

Since #1 requires copying data from the master to the child while it is in an active production environment, I personally went with #2 (creating a new master). This prevents disruptions to the original table while it is actively in use and if there are any issues, I can easily delete the new master without issue and continue using the original table. Here are the steps to do it:

Create new master table.

CREATE TABLE new_master (
    id          serial,
    counter     integer,
    dt_created  DATE DEFAULT CURRENT_DATE NOT NULL
);

Create children that inherit from master.

CREATE TABLE child_2014 (
    CONSTRAINT pk_2014 PRIMARY KEY (id),
    CONSTRAINT ck_2014 CHECK ( dt_created < DATE '2015-01-01' )
) INHERITS (new_master);
CREATE INDEX idx_2014 ON child_2014 (dt_created);

CREATE TABLE child_2015 (
    CONSTRAINT pk_2015 PRIMARY KEY (id),
    CONSTRAINT ck_2015 CHECK ( dt_created >= DATE '2015-01-01' AND dt_created < DATE '2016-01-01' )
) INHERITS (new_master);
CREATE INDEX idx_2015 ON child_2015 (dt_created);

...

Copy all historical data to new master table

INSERT INTO child_2014 (id,counter,dt_created)
SELECT id,counter,dt_created
from old_master
where dt_created < '01/01/2015'::date;

Temporarily pause new inserts/updates to production database

Copy most recent data to new master table

INSERT INTO child_2015 (id,counter,dt_created)
SELECT id,counter,dt_created
from old_master
where dt_created >= '01/01/2015'::date AND dt_created < '01/01/2016'::date;

Rename tables so that new_master becomes the production database.

ALTER TABLE old_master RENAME TO old_master_backup;
ALTER TABLE new_master RENAME TO old_master;

Add function for INSERT statements to old_master so that data gets passed to correct partition.

CREATE OR REPLACE FUNCTION fn_insert() RETURNS TRIGGER AS $$
BEGIN
    IF ( NEW.dt_created >= DATE '2015-01-01' AND
         NEW.dt_created < DATE '2016-01-01' ) THEN
        INSERT INTO child_2015 VALUES (NEW.*);
    ELSIF ( NEW.dt_created < DATE '2015-01-01' ) THEN
        INSERT INTO child_2014 VALUES (NEW.*);
    ELSE
        RAISE EXCEPTION 'Date out of range';
    END IF;
    RETURN NULL;
END;
$$
LANGUAGE plpgsql;

Add trigger so that function is called on INSERTS

CREATE TRIGGER tr_insert BEFORE INSERT ON old_master
FOR EACH ROW EXECUTE PROCEDURE fn_insert();

Set constraint exclusion to ON
```
SET constraint_exclusion = on;
```
Re-enable UPDATES and INSERTS on production database
Set up trigger or cron so that new partitions get created and function gets updated to assign new data to correct partition. Reference this article for code examples
Delete old_master_backup

There is a new tool called pg_pathman (https://github.com/postgrespro/pg_pathman) that would do this for you automatically.

So something like the following would do it.

SELECT create_range_partitions('master', 'dt_created', 
   '2015-01-01'::date, '1 day'::interval);

How to partition existing table in postgres?

Tags:

Postgresql

Optimization

Partitioning

Related

Recent Posts