Monitoring Psycopg2

Psycopg2 is a popular PostgreSQL database adapter for Python. It is an API which allows Python to interface with Postgres databases. It has a number of features such as thread safety, and was designed with concurrency in mind(supporting large numbers of simultaneous operations to a single database).

While thread safety protects against common parallel programming errors, there are still things that can go wrong. Parallel connections increase memory and CPU utilization and can cause difficult-to-catch bugs. The Psycopg2 quickstart automatically monitors your Psycopg2 instances and provides instantaneous feedback on key metrics that affect performance.

New Relic Psycopg2 quickstart features

The Quickstart offers a number of visual dashboards that display the following data:

• CPU Utilization
• Memory heap used
• Garbage collection CPU time
• Top 5 slowest transactions
• Throughput reports
• Most popular transactions
• And more…

It also offers alerts which will notify you when any of the metrics you’re monitoring fall into a critical range, or if an error arises. Alerts include

• Apdex score
• CPU utilization
• Transaction error

New Relic - The complete Psycopg2 dashboard tool

Any Python application which interfaces with a Postgres database is likely to use Psycopg2. This spans a huge array of applications from web backends to machine learning tools to general data stores. Regardless of the specific application, working with Psycopg2 on large-scale projects requires instantiating multiple simultaneous connections with the database. When this many connections are active at once, applications running Psycopg2 become prone to slowdowns or failure.

This observability pack can help you detect those before they become an issue. For example, the New Relic dashboards and alerts relating to CPU utilization provide notifications when a CPU running Psycopg2 nears full capacity. This can be used as a diagnostic tool to identify which sections of an application might need refinements or refactoring.

Another great tool are the alerts on transaction errors. This alert quick catches errors related to database transactions before they derail the rest of an application. They also speed up the debugging process as visibility into the sources of errors.