This service innovation guide explains the innovation journey from SAP Application Logging Service towards SAP Cloud Logging. It shall help customers to understand the benefits and differences to make best possible use of SAP Cloud Logging and to prepare for the migration from SAP Application Logging. It is complemented by a service migration guide describing a procedure to switch from the Application Logging to the Cloud Logging.

Motivation

SAP Application Logging Service successfully served Cloud Foundry application developers for almost 10 years to store, visualize, and analyze their log messages. To improve its usability even further we repeatedly received some key requirements from customers, incl. alerting, configurable retention periods, custom dashboards, and many more. Technical, security, privacy, and compliance restrictions did not allow us to implement these requirements on the underlying multi-tenant stack.

That’s why we created a completely new solution, SAP Cloud Logging. It provides each customer with a dedicated and fully isolated observability stack. This allowed us to not only implement the above-listed requirements but also to innovate by supporting metrics & traces (via OpenTelemetry) and serving major BTP runtimes (Cloud Foundry, Kyma, and others).

SAP Cloud Logging has been proven to be a successful and reliable service, which is why we will deprecate SAP Application Logging Service. This service innovation guide is designed to help you to understand the full potential of SAP Cloud Logging, including its additional benefits, while highlighting the areas where it behaves differently or requires different setup and usage considerations. 

Capabilities & Differences

SAP Cloud Logging comes with many advanced capabilities beyond those of SAP Application Logging Service. The most relevant ones are listed below:

Regarding SAP Cloud ALM check the setup guides for SAP Cloud ALM and SAP Focused Run.

While almost all feature improvements can be simply consumed on top of the existing experience there are few features worth mentioned explicitly:

The authentication system for Cloud Logging has to cover a much more general scenario as it is not only bound to Cloud Foundry. The resulting setup via SAML/IDP requires a one-time setup effort on the customer side compared to the Cloud Foundry-centric setup in Application Logging. We provide detailed documentation on how this can be done. This makes the service future proof to support all SAP runtimes.

As of today, there is one minor feature gap which is the exposure of Cloud Foundry container metrics for Cloud Foundry applications. This gap shall be closed by soon.

Regarding dashboards, the “Statistics” dashboard about quota-based logs dropping is not provided for Cloud Logging Service because it does not drop logs based on quota. The “Metrics” dashboard will be part of the CF container metrics rollout

Service Deployment

Regional Hub

Application Logging is deployed in every Cloud Foundry landscape.

Cloud Logging is a generalized offering that can be jointly used across runtimes, environments, and even data centers. Therefore, it’s deployed once per (compliance) region, from where it’s offered for consumption in all related BTP landscapes and can be commonly used.

Cross tenant (CF Space) analytics

Application Logging is a shared stack where authorization is restricted based on tenant (CF Space) access. For a user, who has CF Space Developer role in multiple spaces, this allows “joint log analysis” across all logging service instances from those multiple spaces within that BTP landscape.

As Cloud Logging is built on dedicated stacks for each service instance, it does not offer joint analysis across those service instances. However, as Cloud Logging enables sharing beyond a single space, you could setup single instance to serve apps from multiple spaces, thereby achieving joint analytics. Also note, CLS also offers fine-grained user authorization – so within a single instance you could define different access rule for your users (restrict space-based access to logs, read only, edit dashboard, hide sensitive logs etc.).

Landscape availability

The availability of Cloud Logging in certain BTP landscapes may shortly follow the initial launch of the landscape. Currently, this is the case for BTP in Switzerland (CH20).

Improved Ingestion Security

Securing Ingestion and rotation

Since ingestion to Cloud Logging is flexible and in customers’ control, it is necessary to offer industry standard ingestion authentication. OTel and ingest endpoints support mTLS ingestion which help to comply with most organization’s security standards.

With certificate-based security mechanism users are responsible for timely rotation of certificate to avoid expiration. With just regular re-binding, Cloud logging allows renewing client certificates. Please read the document for best practice on rotation without disruption.

User provided services

We also recommend leveraging User provided service which offers much faster and effective binding in following cases:

• If your deployment does frequent bind/unbind but you do not intend to rotate certificates, then you could create a User provided service based on the Cloud Logging instance’s service key. As the actual bind/unbind is an expensive and time-consuming process involving new certificate generation, this approach w/ UPS makes the binding faster, fail safe and uninterrupted. Also minimizes the number of certificates that needs to be monitored and rotated. When it is actually needed to rotate the certificates, you could create an additional service key, update the UPS and rebind workloads.
• When you are using central CLS instance across different sub-accounts / BTP landscapes, then UPS is way to reuse the existing instance in one location across many.
• Of course, this requires keeping the UPS manually up-to date with the service keys.

Service Plans & Pricing

Application Logging comes with fixed-size service plans charged monthly. In contrast to this, the flexible configuration and auto-scaling of Cloud Logging comes with a more fine-grained approach.

Service Plans

Application Logging service plans are designed based on ingestion quota. Cloud Logging offers service plans along the number of provisioned resources.

Cloud Logging offers

• 2 production-ready service plans (standard & large) which offer net storage capacity from 75 GB to 3.75 TB.
• 1 development plan (dev) which offers all functional features of production plans but lacks cloud qualities, i.e. no data replication, no auto-scaling, and very limited ingestion throughput. è It’s for evaluation purposes only and must not be used in production use cases.
• its development plan also in the context of SAP Build Code

While Application Logging offers a specific lite plan for evaluation purposes at no charge cost there is no comparable offering in Cloud Logging. This is due to the dedicated provisioning of resources per Cloud Logging instance. Furthermore, customers may reduce cost for development scenarios by sharing Cloud Logging service instances across stakeholders and scenarios.

Pricing

In contrast to the fixed-size and full-month approach of Application Logging the pricing approach of Cloud Logging works as follows:

• Metering & pricing is based on provisioned components (foundation, storage, ingestion).
• Autoscaling of underlying components is explicitly captured.
• Metering is done ”hourly”, i.e. is based on what is physically deployed at the metered hour.
• Usage is aggregated into a single metric: “capacity unit”.

The benefits of this approach are metering is highly accurate to actual consumption per hour and selection of service plans incl. specification of upper bounds for auto-scaling components ensures cost planning safety while still allowing for flexible adaption for changes in observability load.

Example: If customers have configured to use autoscaling for ingest (say 10 blocks) and has been using all 10. At a certain hour when there is no ingest, they will scale-in to 2 and be billed for 2 blocks for that hour. (same applies to disk/storage). Note: There is no scale to 0.
So, if users create an instance with min configuration of standard plan (without auto-scaling) – they would’ve consumed 538 Capacity Units after “30 days” (lower for 28 day months and slightly higher for 31 day months).

Activity shorter than a month (mostly in evaluation cases or cases where instances are created & destroyed often): If the instance is created on 5th morning at 9:00 AM à evaluated until 6th evening 16:00 PM and deprovisioned – then the billing is only for the 31 hours instead of the complete month. On the other hand, this is different from Application logging, where it is billed for complete month for each short incarnation of the service’s instances. i.e., each service “recreate” will be fully billed for a month.

Use SAP Cloud Logging cost estimator to determine cost based on your needs.

Eventual billing numbers are reported via a specific service plan, called “overall”, which is used to aggregate billing numbers across all actual service plans (standard, large, dev). Metrics captured on detailed metrics (e.g. number of ingest blocks) can be inspected via Cloud Cockpit.

Sizing

This section discusses how many SAP Cloud Logging instances you need, and how many apps to bind to one instance. We recommend revisiting which apps are bound to which logging service plan instance and not having a one-to-one mapping of service plans. Compared to Application Logging, Cloud Logging instances can handle significantly more log volume. For most use cases, peak ingestion performance should not be a problem for either of the services.

What previously has been sent to multiple service instances can now be sent to one. This solution allows for additional analyses and can be the more cost-efficient option.

For Application Logging, log ingestion is limited by the service plan quota limit, which defines the theoretical maximum of log storage. Cloud Logging instances have auto-scaling capabilities and a pay-per-resource usage model in which it checks usage hourly.

To determine the storage demand, check the statistics dashboard in Application Logging on how many logs (and bytes) you are shipping and how many have been dropped. When calculating the storage demand, consider the data retention. While data retention is 7 days for Application Logging, it is configurable in Cloud Logging Service. An increase in data retention increases your storage demand. For example, increasing retention from 7 to 30 days increases storage demand by the factor of 4.28.

If you have multiple applications forming a single solution and the combined log volume fits a single Cloud Logging instance, we recommend binding them all to that instance. This allows you to analyze your application data together and investigate cross-application problems.

Find more detailed information on sizing, performance, and Cloud Logging Service plans at the Discovery Center.

Instance Planning

While the setup and usage of Application Logging is always confined by a Cloud Foundry landscape Cloud Logging offers way more flexibility for re-using existing instances in multiple contexts. This comes with technical, cost, and legal implications thus requires proper planning.

Reusability / Shareability

Cloud Logging comes with the ability to “share” single instance across various application deployment locations.

• This increases convenience of centralizing observability in a single or fewer instances.
• This drastically reduces cost in contrast to Application Logging where customers need to create different instances per space.

Sharing across spaces:

Sharing across runtimes:

Sharing across DCs in a region:

Sharing across DCs across a region:

• Technically it is also possible to extend the above scenario to share across regions (US <-> EU) depending on the ingest load and latency.

Tradeoffs

Planning actual instantiation and deployment of Cloud Logging service instances should consider the following aspects:

• Cost: in most cases shared instances should be more cost effective than dedicated ones.
• Legal/Operational aspects might enforce customers to run dedicated instances. Still Cloud Logging provides many security-level features (e.g. index-level security) which might be sufficient to satisfy those.
• Analytics: putting observability data into a shared instances allows for combined analysis. Note: in contrast to Application Logging, data cannot be analyzed across service instances.
• Ingestion throughput: Remote ingestion (from Cloud Foundry) may come with bit lower throughput.

Outlook

Cloud Logging service is constantly innovating to bring OpenSearch features after passing them through SAP Product standards. Few of the highlighting features which are under investigation are:

• Migration to OpenSearch 2.x
• OIDC based Auth in addition to SAML
• Extra Large plan with possible tiering/snapshot
• Multiple data sources for visualization across multiple Cloud logging instances
• Index compression for efficient storage and performance
• Simplified default user managements and API access
• And many more…