https://sre.google/sre-book/service-level-objectives/
You want the application's SLO to more closely reflect it's observed reliability. The key here is error budget never goes over 5%. This means they can have additional downtime and still stay within their budget.

The best option for implementing a scalable solution to support global Prometheus querying and minimize management overhead is to use Google Cloud Managed Service for Prometheus. Google Cloud Managed Service for Prometheus is a fully managed service that allows you to collect, query, and visualize metrics from your GKE clusters using Prometheus-based tooling. You can use Google Cloud Managed Service for Prometheus to query metrics across multiple clusters and regions using a global view. You can also use Google Cloud Managed Service for Prometheus to integrate with other Google Cloud services, such as Cloud Monitoring, Cloud Logging, and BigQuery. By using Google Cloud ManagedService for Prometheus, you can avoid managing and scaling your own Prometheus servers and focus on your application performance.

The best option for storing and using the API key in your application by following Google-recommended practices is to save the API key in Secret Manager as a secret and reference the secret as an environment variable in the Cloud Run application. Secret Manager is a service that allows you to store and manage sensitive data, such as API keys, passwords, and certificates, in Google Cloud. A secret is a resource that represents a logical secret, such as an API key. You can save the API key in Secret Manager as a secret and use IAM policies to control who can access it. You can also reference the secret as an environment variable in the Cloud Run application by using the ${SECRET_NAME} syntax. This way, you can securely store and use the API key in your application without exposing it in your code or configuration files.