Cloud computing offers a wide range of services through a pool of heterogeneous Physical Machines (PMs) hosted on cloud data centers, where each PM can host several Virtual Machines (VMs). Resource sharing among VMs comes with major benefits, but it can create technical challenges that have a detrimental effect on the performance. To ensure a specific service level requested by the cloud-based applications, there is a need for an approach to assign adequate resources to each VM. To this end, we present our novel Multi-Loop Control approach, called MultiScaler, to allocate resources to VMs based on the Service Level Agreement (SLA) requirements and the run-time conditions. MultiScaler is mainly composed of three different levels working closely with each other to achieve an optimal resource allocation. We propose a set of tailor-made controllers to monitor VMs and take actions accordingly to regulate contention among collocated VMs, to reallocate resources if required, and to migrate VMs from one PM to another. The evaluation in a VMware cluster have shown that the MultiScaler approach can meet applications performance goals and guarantee the SLA by assigning the exact resources that the applications require. Compared with sophisticated baselines, MultiScaler produces significantly better reaction to changes in workloads even under the presence of noisy neighbors.