Migrating from Heroku Postgres

Migrating data from Heroku Postgres into Crunchy Bridge can be achieved one of two ways. For datasets larger than 100GB it is recommended to work with Crunchy Bridge support to use Heroku's physical backups and WAL files to create a replica of your Heroku database and cutover your apps in a coordinated event.

For datasets under 100 GB, it is generally simplest to migrate via a logical backup (pg_dump / pg_restore). Running the pg_dump / pg_restore on-dyno saves time by eliminating the need to download the backups, and all operations occur within the same datacenter. The pg_dump is performed using the Heroku default user's credentials, while pg_restore will use the Crunchy Bridge superuser role.

1. All Crunchy Bridge instances are created with database auditing enabled for the postgres role. Terefore, to reduce excessive logging, temporarily disable pgaudit for the postgres role. On your Crunchy Bridge instance as the postgres role. While connected to your Crunchy Bridge instance:

ALTER USER postgres RESET pgaudit.log;
Copy

2. Start a Heroku bash session in an app that has the Heroku database attached, and pipe the logical backup from pg_dump directly into pg_restore to your empty Crunchy Bridge instance. The dump is current as of the moment it is executed, so if this is being done for a cutover ensure that your application is in maintenance mode so that there are no changes to the data during the migration.

$ heroku run bash --app your-app-here

$ pg_dump -Fc $DATABASE_URL | pg_restore --no-acl --no-owner -d postgres://postgres:password@p.clusterid.db.postgresbridge.com:5432/postgres
Copy

3. Since the restore was executed using the postgres role, all the created tables are also owned by the postgres role, and so the Crunchy Bridge application role has no default permissions. To give the application role identical permissions to those that the Heroku role had, reset objects' owners to the application role. These statements will generate the commands used to grant those permissions - The output of these commands still need to be executed in order for ownership to change. This runs in your Crunchy Bridge instance as the postgres role:

-- Tables
SELECT 'ALTER TABLE '|| schemaname || '."' || tablename ||'" OWNER TO application;'
FROM pg_tables WHERE NOT schemaname IN ('pg_catalog', 'information_schema')
ORDER BY schemaname, tablename;

-- Sequences
SELECT 'ALTER SEQUENCE '|| sequence_schema || '."' || sequence_name ||'" OWNER TO application;'
FROM information_schema.sequences
WHERE NOT sequence_schema IN ('pg_catalog', 'information_schema')
ORDER BY sequence_schema, sequence_name;

-- Views
SELECT 'ALTER VIEW '|| table_schema || '."' || table_name ||'" OWNER TO application;'
FROM information_schema.views
WHERE NOT table_schema IN ('pg_catalog', 'information_schema')
AND NOT table_name IN ('pg_stat_statements', 'pg_stat_statements_info')
ORDER BY table_schema, table_name;

-- Materialized Views
SELECT 'ALTER TABLE '|| oid::regclass::text ||' OWNER TO application;'
FROM pg_class WHERE relkind = 'm'
ORDER BY oid;
Copy

4. Be sure to create any other needed roles and set appropriate permissions depending on how they are used (ie, read-only roles used by your application). Note also that the Application role is intended for use by your application while the postgres / superuser account is intended for administration (refer to User Management Concepts for details).
5. Re-enable pgaudit on the postgres user by, on your Crunchy Bridge instance as the postgres role:

ALTER USER postgres SET pgaudit.log='all';
Copy

6. Once data is migrated, apps can be updated with the new, Crunchy Bridge, connection string and taken out of maintenance mode. If you wish to continue to use $DATABASE_URL in your app, be sure to detach the database URL from your app so that Heroku doesn't overwrite the Crunchy Bridge URL with the old Heroku Postgres URL, ie:

$ heroku addons:detach DATABASE --app your-app-hereCopy

It is also advisable to validate the restore process in a staging environment prior making changes to your production application.