Welcome to NOBL9
and Your First SLO!
Thank you for trying out NOBL9! NOBL9 is a tool to help you create and monitor service level objectives (SLOs), and notify you when things are going wrong. SLOs allow you to define the reliability of your products and services in terms of customer expectations. You can create SLOs for user journeys, internal services, or even infrastructure. For more background on SLOs, click here.
This walkthrough will cover the basics of creating SLOs in NOBL9. In future articles, we’ll walk you through some of the more complex features of the product.
NOBL9 is unique in that it can work with the metrics and logs you already have to build and report on SLOs. Numerous sources for indicators (the data that we use to calculate your SLOs) are supported; you can find the complete list at nobl9.com/integrations. Before getting started, you’ll need to know where your existing metrics data resides, and you should be able to answer yes to the following questions:
[ ] Is the data source supported by NOBL9? If not, let us know at firstname.lastname@example.org.
[ ] Do you know which metric you want to use for an indicator?
[ ] Do you know how to query your data source for that metric?
[ ] Do you have the credentials to authenticate to your data source?
Create a Project
Projects are the primary logical grouping of resources in the NOBL9 platform. All NOBL9 resources, such as data sources, SLOs, and alerts, are created within a project. Access controls at the project level enable you to control who can see and change these resources. For example, you might want to allow all of your users to view the SLOs in a given project, but only a few users to make changes.
You can think of your projects like folders. Before you can start creating SLOs, you have to create a project to put them in. To create a project, select Catalog from the navigation menu. Then, on the Projects tab, click the “+” button.
Enter a display name. The Name field will automatically be populated with a Kubernetes-style name, which you can modify if you like. We use this in our YAML configurations to ensure the uniqueness of object names. Next, add a description and click “Create Project.”
Create a Service
A service can represent a logical service endpoint like an API, a database, an application, or anything else you care about setting a service level objective for, such as a user journey. Every SLO in NOBL9 is tied to a service, and a service can have one or more SLOs.
To create a service, navigate to the Catalog page and select the Services tab.
Next, click the “+” to add a new service. The service creation form has four fields. If left blank, the service will automatically use a project called “default”. You can select the project you created in the first step.
Next, you can add a display name. This is a friendly name for the service, from which a unique Kubernetes-style name will automatically be generated in the Name field. You can use the name that is suggested, or edit it. You can also add an optional description for the service (this is recommended).
When you’re done, click “Add Service.”
Add a Data Source
NOBL9 uses your existing metrics or events data to calculate SLOs, and it needs access to those metrics before it can query for SLI data. To enable that, you need to add a data source.
To set up a data source, navigate to Integrations and click the “+” button on the Sources tab.
Next, select your data source.
We’ll use Prometheus for this example, but you should pick the data source that has your metrics. Setting up a data source is straightforward, and the process is very similar across all the supported sources.
Next, you’ll need to specify how to connect to the data source.
NOBL9 can connect to a data source either directly, or via an agent. Agent connections require you to run an agent either in a Kubernetes cluster or in a Docker container. Direct connections do not need an agent but do require you to enter your credentials, which NOBL9 will store safely. Most integrations support both direct and agent connections.
This example continues with an agent connection.
Configuring the Prometheus data source requires you to enter a URL, select a project, and specify a name. As usual, you can provide a display name that will be used to autogenerate the name, and adding a description is recommended.
Most data sources will require similar information for agent deployments. For direct connections, you will be asked for your credentials as well. Once you’ve entered the required data, click “Add Data Source.”
Since this is an agent deployment, NOBL9 will generate a Kubernetes configuration and a Docker command line for you to use to deploy the agent.
If you have a running Kubernetes cluster, you can copy and paste the YAML into your Kubernetes configuration. If you use Docker, you can run the Docker command to deploy the agent.
If you don’t have a Kubernetes cluster or Docker running anywhere, you can run Docker locally on your laptop, and then execute the Docker command to deploy the agent on your machine. We don’t recommend doing this for production usage, of course, but it’s a good way to test out the system. Note if you do run the agent locally on your machine, anytime your machine goes to sleep will lead to missing data and gaps in your metrics.
If you instead create a direct connection to a data source, the connection will be made through NOBL9, and no agent deployment is necessary.
Create an SLO
Now that you have a service and data source ready, it’s time to add your first service level objective. For a real-world scenario, you’ll want to think about what you want to measure that has real customer impact (for an introduction to SLOs with clear explanations and great examples, see the book Implementing Service Level Objectives).
In this example, we’re going to add a latency objective. To get started, navigate to the SLOs page, and click the “+” to launch the SLO wizard.
In step 1, select the service you created earlier. In step 2, select the data source you added previously, and add the query for the metric you want to build an SLO on top of. For our example, we will query Prometheus for a latency metric by adding the following query in the Threshold Metric box:
As well as threshold metrics, where a single time series is evaluated against a threshold, Prometheus and most other data sources support ratio metrics, where two time series metrics are compared (e.g., good requests vs. total requests). This requires providing two queries.
If you are using a different data source, such as AppDynamics or Splunk, you can leverage the query builders in those tools to create your SLI queries and copy and paste them into NOBL9.
In step 3, you define a time window for the SLO. This is the period over which NOBL9 calculates the error budget. For example, in a 30-day rolling window, a 99% target would provide 432 minutes of error budget.
NOBL9 supports a rolling time window that resets at the selected interval as well as a calendar-aligned time window. The right choice will depend on your SLO and business needs. For this example, we’ll use a rolling 7-day window.
In step 4, you define your objective. NOBL9 provides two ways to calculate the error budget:
- Occurrences count good attempts against the count of all attempts.
- Time Slices measures how many good minutes (minutes in which the system was operating within defined boundaries) were achieved compared to the total minutes in the window.
We’ll use Occurrences. Each SLO allows up to 6 objectives. SLO objectives are also known as SLO units. The free trial provides up to 25 SLO units for your account.
In this example, we’ll set the target at 99%. This means 99% of occurrences for latency will be with the tolerance defined below. We set the experience name to “Good” and the desired value to less than 2000.
This means that for 99% of the events in the time window, we would expect the latency to be under 2 seconds in order to meet our objective. Any event with a latency over 2 seconds causes error budget to burn. If more than 1% of the events during the time period have a latency over 2 seconds, we will run out of error budget.
Finally, in step 5, you provide a name, description, and other details about your objective.
Entering a display name automatically generates a name for the SLO. Since we haven’t set up alert policies yet, we’ll skip that for now.
If you want to add a label to the SLO, you can add it here. Labels are key/value pairs that you can use to filter across SLOs and services in the SLO grid view and in reports. For example, you might want to add a label with a key of “Team” and a value of “My Team.”
You can also optionally add a link that appears in the SLO details view which can be used to link directly to the metrics source, a runbook, or anything else you want. And, as usual, you can add a description of the SLO. Feel free to leave some of these fields empty for now; you can always edit the SLO and add them later.
When you’re done, click “Create SLO.”
You will see a new chart appear in the SLO grid view. It takes a few minutes for the data to be populated, but once it is you’ll see something like this:
Note that if you selected a longer time window earlier, it may take longer for the chart to appear as the data is downsampled into larger slices for the chart view.
Clicking on the name of the SLO objective will take you into the SLO details view. You can also click “View Query” to see your query or click the pencil icon to edit your SLO.
The SLO details view provides you with the raw SLI data, the burn-down chart, and the error budget burn rate.
These charts help you view what’s happening with your SLO and aid in debugging when things go wrong.
Congratulations on creating your first SLO!
Check out these other tutorials to learn more about NOBL9: