Keeping track of a Python application’s performance

Preamble

Why Python application monitoring is important

There are many advantages to monitoring Python programs, and these advantages also apply to programs written in other programming languages. It’s important to think about these benefits in the big picture because larger applications often use a lot of different tools and programming languages. It’s important to know the benefits of monitoring for each layer of your application and the types of monitoring tools you’ll use for each layer, whether you’re using Python with other tools in a larger distributed system or Django for an MVC application.

Presentation layer

The presentation layer is made up of the user interface and every part of your app that a user can see and interact with. In the presentation layer, there can be many problems, such as pages that load slowly or not at all, forms that don’t work right, or parts of a page that don’t look right. This could also involve coding errors in your views if you’re using Django or another framework.

Synthetic monitoring, which navigates your site using a headless browser, and real-user monitoring (RUM), which tracks actual user behavior on your site, are both frequently used to track the presentation layer.

Commercial layer

The business layer is where application logic is located. The logic of a monolithic app might be in a single codebase, but the business layer is becoming more and more made up of loosely connected microservices that, if one fails, can have big effects on the others. Because many Python applications use JavaScript or JavaScript single-page apps (SPAs) like React for the presentation layer, a Python business layer is typically an API, whether built with the Django REST framework or another tool. In the business layer, problems can happen with failed or slow API calls, problems with customer bank transactions (like on an e-commerce site), and problems with authentication and authorization services that can keep your customers from accessing your site or, worse, leave your application vulnerable.

Most of the time, you’ll use an APM (application performance monitoring) tool for the business layer. This tool gives you important metrics about application performance, like error rate, throughput, and transaction time.

Benefits of Python application monitoring

Monitoring Python applications has many other benefits, such as making developers more productive and reducing the average amount of time it takes to fix a problem. Let’s look at a few of these benefits:

• Bottlenecks and other performance problems should be quickly located and diagnosed. These metrics help you decrease your mean time to detection (MTTD) and mean time to resolution (MTTR) of issues. A performance monitoring tool includes metrics such as throughput, average transaction time, and error rate. You can easily see when topline metrics are affected, then drill down further to find where problems are occurring, such as by looking at the top five slowest transactions.
• Boost the efficiency of your application. If you want to ensure that pages that end users load quickly or make more significant architectural decisions about the scalability of your application, detailed metrics give you important insights into what your teams should focus on.
• Be aware of important performance indicators and metrics. Without alerts, it’s difficult to know when something is wrong with your application, which frequently results in you learning about problems from your customers rather than taking steps to lessen their impact before your customers are impacted.
• Increase the output of developers. Python is already well-known for being a language that increases developer productivity, but every minute you have to spend locating bugs and other issues in live code detracts from your ability to code and create new features.
• Enable cross-team cooperation, including DevOps. A good application monitoring tool includes opportunities for collaboration, ranging from alerts that can be sent to multiple teams to in-application debugging and triaging of issues when they arise. While many teams are cross-functional, with larger applications, different teams may be siloed.

Metrics you should monitor in a Python application

The application layers and some of the metrics used to track each layer were covered in the last section. Now let’s look at some of the metrics, what they mean, and why they are important for your Python application.

A good application monitoring tool will automatically measure these things about your app:

• Response time: also known as latency, is one of the four golden signals and is a crucial indicator of user satisfaction. A response time of less than a second is considered acceptable (with 200 milliseconds or less being ideal), and when the average response time is above a second, your end users are much more likely to be dissatisfied with the performance.
  Ultimately, the exact threshold depends on your specific use case. A potential starting point is to alert if the median page load time is more than 10 seconds for 5 minutes. This indicates that there are serious issues with your website that need to be addressed right away. If your alert threshold is too ambitious (for example, set at 1 second), you might get too many alerts, resulting in a lot of wasted time.

Distributed tracing with Python

Metrics give you a high-level overview of your application’s performance, but what should you do if they show slow requests or other problems? If the request is simple, you might be able to find the source of the problem by looking into it by hand. What happens, though, if the request passes through numerous services? In that case, you require more in-depth trace data that tracks requests as they move through the system. Distributed tracing can help with that. Distributed tracing lets you keep track of requests as they move through your application. This tells you a lot about how your services are working.

Distributed tracing can process and store a lot of data, but it provides you with detailed information about the requests in your system. Because of this, sampling traces are often used. These are traces of only some of the requests that go through your application. When using tail-based sampling, you can choose to only trace a certain subset of requests, like all requests that include an error message, or you can choose to trace a certain percentage of requests, like 25%.

In a Python application, you can use a wide range of tools, including open-source tools like OpenTelemetry, for distributed tracing requests.

Tools for monitoring Python applications

Let’s take a look at some open-source tools that you can use to monitor your application.

• OpenTelemetry, which is part of the Cloud Native Computing Foundation (CNCF), is a collection of open source tools, APIs, and SDKs for working with telemetry data. You can use it to create, collect, and export your data, including metrics, logs, and traces. Because it’s vendor neutral, you can use it with any language or framework, including Python and Python frameworks. You can easily install the API and SDK packages with pip, then use OpenTelemetry’s tracer to collect data from your application. Because OpenTelemetry is part of the CNCF, it will always be open source, and it benefits from a strong community of developers. However, while you can do automatic instrumentation of your Python applications with OpenTelemetry, setting it up throughout your application will take some manual work.
• Prometheus, which is also a CNCF open source project, collects metrics data by scraping HTTP endpoints and then stores that data in a time series database that uses a multidimensional model. It’s a powerful tool for gathering metrics about your application and it also includes alerting functionality that you can use to notify your teams when issues come up. Prometheus includes a client library for Python.
• Jaeger is an open source distributed tracing tool. It can store trace data in both Cassandra and Elasticsearch.
• Zipkin, which was developed by Twitter, is an open source tool for distributed tracing that can also be used to troubleshoot latency issues in your application. While Zipkin is Java-based, py_zipkin is an implementation for Python.
• Logging is a built-in Python library that provides flexible event logging. You can easily import it into your Python application by adding import logging to the top of a file where it’s needed.
• Structlog is an open source Python tool for adding structure to your log messages. You can use it with Python’s built-in logging tool to better organize your logs.

You can use open source to keep an eye on your apps, but you will probably need to use and understand a number of tools. To fully monitor every part of your application, such as the server side, the client side, cloud-based services, and more, you need custom implementations. As your applications grow, it gets harder to build and keep up a full-stack custom observability solution.

About Enteros

​​​​Enteros offers a patented database performance management SaaS platform. It finds the root causes of complex database scalability and performance problems that affect business across a growing number of cloud, RDBMS, NoSQL, and machine learning database platforms.