Logging and Monitoring your Crunchy Bridge Cluster

PostgreSQL emits messages that show server behavior. These messages not only indicate when a bug is encountered or something unexpected has occurred so that you can dig deeper and resolve issues; over time, they also allow you to measure and keep track of how your database is performing.

If you’re already using a logging service for other components in your tech stack, you’ll probably want to direct your Postgres message log there as well. This makes your logs easier to manage, since you won’t have to view nor analyze them individually per source. Having your logs aggregated and viewable in one place also lets you have a better overview of your systems, and be able to synthesize related information across components.

Logging in Crunchy Bridge

Crunchy Bridge clusters can export its logs via syslog.

To integrate your Crunchy Bridge Postgres cluster logs to a third-party logging service, you’ll need an access key associated with your service account. It might be referred to as an “ingestion key,” “API key,” or “token,” just to give a few examples. Your logging service documentation should have the details available for retrieving that key.

Configure logging integration

  1. On your Crunchy Bridge dashboard, select the database cluster, and navigate to the logging tab.

  2. Click “Add Log Destination.”

  3. In the “Create log destination” window, fill in the following fields based on your logging service:

    • Host: Logging destination hostname.
    • Port: Destination port number.
    • Message template: Format for ingestion as specified in our Logging section of the documentation, or the syslog-ng instructions for your logging service. The message template usually where you add your access key.
    • Description: For your reference.

    Click “Submit” to save.

Verify integration

Once a destination is configured, Crunchy Bridge will start sending log messages to your service. These logs will contain PostgreSQL server messages as well as other syslog output.

PostgreSQL messages include a log line prefix string based on the following format:

%m [%p][%b][%v][%x] %q[user=%u,db=%d,app=%a]

Syslog-ng messages will include a roll-up of log statistics, collected and emitted periodically. Error and other high-severity level messages are also emitted by syslog-ng.

You’ll verify that the integration was set up correctly by accessing your logging service account, and checking that log messages are visible from the Crunchy Bridge cluster host/source.

LogDNA example

This example walks you through the exact steps to configure and verify your LogDNA integration for Crunchy Bridge via the UI.

  1. In your LogDNA account, open the Manage Organization page and retrieve your Ingestion Key under “API Keys.”

  2. In the “Create log destination” window for your Crunchy Bridge cluster, enter the following values in the respective fields:

    Host: syslog-a.logdna.com

    Port: 6514

    Message template (add your Ingestion Key): <${PRI}>1 ${ISODATE} ${HOST} ${PROGRAM} ${PID} ${MSGID} [[email protected] key=\"INSERT-YOUR-INGESTION-KEY-HERE\"] $MSG\n

    Description: A brief description for your reference.

    Hit “Submit” to save. This adds your Crunchy Bridge PostgreSQL cluster as an ingestion source for your organization in LogDNA.

  3. In your LogDNA app, you’ll soon start to see syslog-ng statistics and Postgres log messages in your log viewer.

    This is an extract from the log viewer for both a syslog-ng periodic roll-up message as well as a Postgres error (where there was an attempt to connect to a database that does not exist):

    Mar 18 15:08:03 c6p3j5s7evattiqewf7al52hzu postgres err [16-1] 2021-03-18 19:08:03.819 GMT [715322][client backend][4/6762675][0] [user=postgres,db=fakedb,app=[unknown]] FATAL:  database "fakedb" does not exist
    Mar 18 15:15:18 c6p3j5s7evattiqewf7al52hzu syslog-ng info Log statistics; processed='global(sdata_updates)=0', processed='source(s_egress)=38524', processed='global(payload_reallocs)=292', queued='global(scratch_buffers_bytes)=0', processed='global(msg_clones)=0', processed='global(internal_queue_length)=0', dropped='dst.network(d_whwgdlo27va3tlpvqis2m5c4iy#0,tls,[syslog-a.logdna.com:6514](http://syslog-a.logdna.com:6514/))=37513', processed='dst.network(d_whwgdlo27va3tlpvqis2m5c4iy#0,tls,[syslog-a.logdna.com:6514](http://syslog-a.logdna.com:6514/))=38524', queued='dst.network(d_whwgdlo27va3tlpvqis2m5c4iy#0,tls,[syslog-a.logdna.com:6514](http://syslog-a.logdna.com:6514/))=0', written='dst.network(d_whwgdlo27va3tlpvqis2m5c4iy#0,tls,[syslog-a.logdna.com:6514](http://syslog-a.logdna.com:6514/))=8679', processed='src.internal(s_egress#0)=38487', stamp='src.internal(s_egress#0)=1616094318', processed='center(received)=38524', processed='destination(d_diag)=327', queued='global(scratch_buffers_count)=60129542144', processed='destination(d_whwgdlo27va3tlpvqis2m5c4iy)=38524', processed='center(queued)=38851'
    
  4. By default, the LogDNA viewer displays all logs from all your organization’s ingestion sources. You can filter on:

    • Hosts (Ingestion source)
    • Apps (App or program), e.g. postgres, syslog-ng
    • Levels (e.g. INFO, NOTICE, DEBUG, WARN, ERR)

(Click the following image to enlarge) LogDNA viewer with Crunchy Bridge Postgres and syslog messages

  • Crunchy Bridge is identified as a source/host with a character string, e.g. c6p3j5s7evattiqewf7al52hzu

  • On the app level, you can filter on just postgres to view only Postgres server messages.

  • Finally, you can filter on message severity levels.

You can save the filter selections as a new view.

To learn more about working with and viewing logs, visit the LogDNA docs.