Automagically scale your apps on Cloud Foundry

This is great news – the App-Autoscaler was released and now you're able to scale your apps automatically and for free within the Swisscom Application Cloud!

The Swisscom Application Cloud uses the open-source distribution of Cloud Foundry as it’s basis to provide you with a platform-as-a-service. Ever since our platforms official release we’ve continually been adding and implementing new features, and one of these is the App-Autoscaler, which is an incubation project for Cloud Foundry.

https://github.com/cloudfoundry-incubator/app-autoscaler

 

The App-Autoscaler provides the capability to adjust the computation resources for Cloud Foundry applications through:

  • Dynamic scaling based on application performance metrics
    Dynamic scaling adjusts the application instance number based on the rules defined on performance metrics. It is used to scale application in response to dynamically changing workload
  • Time-based scheduling
    Scheduled scaling adjusts the application instance number at a predefined time slot. It is mainly used when application resource demand is predictable.

The following metrics are currently supported for dynamic scaling:

  • memoryused
    This is the absolute value of used memory of an application instance.
    The unit of „memoryused“ is „MB“.
  • memoryutil
    This is a short name for „memory utilization“ and it’s the percentage of used memory for total memory allocated to an application instance. If the memory usage of the application is „100 MB of 200 MB“, the value of „memoryutil“ is 50%.
    The unit of „memoryutil“ is percentage.
  • responsetime
    This is metric measures the average value of the aggregated total elapsed time of all processesd requests occurred in a specific time window (collection interval) for an application instance.
    The unit of „responsetime“ is „ms“ (milliseconds).
  • throughput
    This is metric measures the total number of processed requests occurred in a specific time window for an application instance.
    The unit of „throughput“ is „rps“ (requests per second)

Now let’s see how the App-Autoscaler works!

 

 

If you list the marketplace on our Application Cloud Platform you’ll see, that there is a service called „autoscaler“ with a free plan available.
The first step you need to do is to create such an Autoscaler service:

$ cf create-service autoscaler my-autoscaler

The next step then is to bind the newly created Autoscaler service to the app you want to scale automatically.

$ cf bind-service my-app my-autoscaler -c <auto-scaler-policy>

But wait, what does the Autoscaler policy refer to?

The policy is a JSON code file which configures the behaviour of this specific Autoscaler instance. You can find a detailed example here: https://github.com/cloudfoundry-incubator/app-autoscaler/blob/develop/src/integration/fakePolicyWithSchedule.json

Here is a compact example of a scheduled policy:

{
  "instance_min_count": 1,
  "instance_max_count": 3,
  "scaling_rules": [
    {
      "metric_type": "memoryutil",
      "breach_duration_secs": 600,
      "threshold": 30,
      "operator": "<",
      "cool_down_secs": 300,
      "adjustment": "-1"
    },
    {
      "metric_type": "memoryutil",
      "breach_duration_secs": 600,
      "threshold": 90,
      "operator": ">=",
      "cool_down_secs": 300,
      "adjustment": "+1"
    }
  ],
  "schedules": {
    "timezone": "Europe/Zurich",
    "recurring_schedule": [
      {
        "start_time": "10:00",
        "end_time": "18:00",
        "days_of_week": [
          1,
          2,
          3
        ],
        "instance_min_count": 1,
        "instance_max_count": 10,
        "initial_min_instance_count": 5
      },
      {
        "start_date": "2099-06-27",
        "end_date": "2099-07-23",
        "start_time": "11:00",
        "end_time": "19:30",
        "days_of_month": [
          5,
          15,
          25
        ],
        "instance_min_count": 3,
        "instance_max_count": 10,
        "initial_min_instance_count": 5
      }
    ],
    "specific_date": [
      {
        "start_date_time": "2099-06-02T10:00",
        "end_date_time": "2099-06-15T13:59",
        "instance_min_count": 1,
        "instance_max_count": 4,
        "initial_min_instance_count": 2
      }
    ]
  }
}

 

 

If you’ve already prepared a policy for the Autoscaler, then you’re able to attach this directly with using the „-c“-flag during the service binding.  After the service binding is done, you need to restage your app to let the policy take effect.

If you haven’t prepared a policy yet – don’t worry about that. It’s also possible to set this afterwards with the public API as described here https://github.com/cloudfoundry-incubator/app-autoscaler/blob/master/docs/Public_API.rst . Another possibility to easily manage policies and retrieve metrics and scaling history is to use the App-Autoscaler CLI Plugin https://github.com/cloudfoundry-incubator/app-autoscaler-cli-plugin

An advantage of using the CLI Plugin is, that changes on policies will take effect immediately without restaging an app.

Now that we have an idea of how the Autoscaler works and which metrics it supports, let’s have a look at some more advanced topics.

First of all, it’s possible to define multiple scale-out and scale-in rules. But how does the App-Autoscaler handle conflict situations? Let’s assume you’ve defined an overlapping policy. The App-Autoscaler  does not detect conflicts among them.  It will guarantee the one which starts first, while the later one is completely ignored.

Example:

  • Schedule #1: ——–sssssssssss—————————-
  • Schedule #2: —————ssssssssssssss—————–
  • Schedule #3: ————————–sssssssss————

With the above definition, Schedule #1 and Schedule #3 will be applied, Schedule #2 will be ignored.

If a schedule’s start time is earlier than the policy creation time, the schedule will not be executed immediately.

Example:

  • Schedule #1: 10:00 – 15:00, Everyday

If the above schedule is created at 11:00 a.m., it won’t take effect immediately once its created but it will be triggered the next day.

For more details about what can be defined in a policy, you can take a closer look here: https://github.com/cloudfoundry-incubator/app-autoscaler/blob/master/docs/Policy_definition.rst

Things you should know before auto-scaling an application

It’s important to know and understand the performance and load characteristics of your application to be able to setup scaling policies properly. You might consider taking the following steps before creating a scaling policy:

  • Benchmark the application to understand the performance
  • Scale the application manually to understand how the application behaves when scaling out and in
  • Load test the policy to see how it works

There are also some good general guidelines:

  • Do not use App-AutoScaler service to handle sudden burst. If the burst is predictable, use scheduled scaling to prepare enough application instances for it.
  • Prevent excessive scale in/out by setting min/max instance number
  • Carefully set the threshold, don’t push too hard. Make sure it will not exceed memory limit
  • For CPU intensive app, be aware that CPU is actually shared in a weighted manner with other apps on the same host VM. Use other metrics like throughput/responsetime for scaling in such cases.
  • Design dynamic scaling rules carefully when using multiple metrics

Try it out now! https://developer.swisscom.com/