SmallTail: Scaling cores and probabilistic cloning requests for web systems
Users quality of experience on web systems are largely determined by the tail latency, e.g., 95th percentile. Scaling resources along, e.g., the number of virtual cores per VM, is shown to be effective to meet the average latency but falls short in taming the latency tail in the cloud where the perf...
- Autores:
- Tipo de recurso:
- Fecha de publicación:
- 2018
- Institución:
- Universidad del Rosario
- Repositorio:
- Repositorio EdocUR - U. Rosario
- Idioma:
- eng
- OAI Identifier:
- oai:repository.urosario.edu.co:10336/22856
- Acceso en línea:
- https://doi.org/10.1109/ICAC.2018.00013
https://repository.urosario.edu.co/handle/10336/22856
- Palabra clave:
- Controllers
Level control
Quality of service
Virtual machine
Websites
Autoscaling
Inner loop controls
Latency control
Level controllers
Performance variability
Probabilistic cloning
Probablistic
Quality of experience (qoe)
Cloning
Probablistic cloning
Query cloning
Tail latency control
Two level controller
Vertical autoscaling
- Rights
- License
- http://purl.org/coar/access_right/c_abf2
id |
EDOCUR2_4acaa80447f430d10708626a1fa7e693 |
---|---|
oai_identifier_str |
oai:repository.urosario.edu.co:10336/22856 |
network_acronym_str |
EDOCUR2 |
network_name_str |
Repositorio EdocUR - U. Rosario |
repository_id_str |
|
spelling |
SmallTail: Scaling cores and probabilistic cloning requests for web systemsControllersLevel controlQuality of serviceVirtual machineWebsitesAutoscalingInner loop controlsLatency controlLevel controllersPerformance variabilityProbabilistic cloningProbablisticQuality of experience (qoe)CloningProbablistic cloningQuery cloningTail latency controlTwo level controllerVertical autoscalingUsers quality of experience on web systems are largely determined by the tail latency, e.g., 95th percentile. Scaling resources along, e.g., the number of virtual cores per VM, is shown to be effective to meet the average latency but falls short in taming the latency tail in the cloud where the performance variability is higher. The prior art shows the prominence of increasing the request redundancy to curtail the latency either in the off-line setting or without scaling-in cores of virtual machines. In this paper, we propose an opportunistic scaler, termed SmallTail, which aims to achieve stringent targets of tail latency while provisioning a minimum amount of resources and keeping them well utilized. Against dynamic workloads, SmallTail simultaneously adjusts the core provisioning per VM and probabilistically replicates requests so as to achieve the tail latency target. The core of SmallTail is a two level controller, where the outer loops controls the core provision per distributed VMs and the inner loop controls the clones in a finer granularity. We also provide theoretical analysis on the steady-state latency for a given probabilistic replication that clones one out of N arriving requests. We extensively evaluate SmallTail on three different web systems, namely web commerce, web searching, and web bulletin board. Our testbed results show that SmallTail can ensure the 95th latency below 1000 ms using up to 53% less cores compared to the strategy of constant cloning, whereas scaling-core only solution exceeds the latency target by up to 70%. © 2018 IEEE.Institute of Electrical and Electronics Engineers Inc.20182020-05-25T23:58:24Zinfo:eu-repo/semantics/conferenceObjecthttp://purl.org/coar/version/c_970fb48d4fbd8a85http://purl.org/coar/resource_type/c_c94fapplication/pdfhttps://doi.org/10.1109/ICAC.2018.00013https://repository.urosario.edu.co/handle/10336/22856instname:Universidad del Rosarioreponame:Repositorio Institucional EdocURenghttps://www.scopus.com/inward/record.uri?eid=2-s2.0-85061316455&doi=10.1109%2fICAC.2018.00013&partnerID=40&md5=87c34c798122078edea62f43188634cdhttp://purl.org/coar/access_right/c_abf2Lakew E.B.Birke R.Pérez, Juan F.Elmroth E.Chen L.Y.oai:repository.urosario.edu.co:10336/228562022-05-02T07:37:17Z |
dc.title.none.fl_str_mv |
SmallTail: Scaling cores and probabilistic cloning requests for web systems |
title |
SmallTail: Scaling cores and probabilistic cloning requests for web systems |
spellingShingle |
SmallTail: Scaling cores and probabilistic cloning requests for web systems Controllers Level control Quality of service Virtual machine Websites Autoscaling Inner loop controls Latency control Level controllers Performance variability Probabilistic cloning Probablistic Quality of experience (qoe) Cloning Probablistic cloning Query cloning Tail latency control Two level controller Vertical autoscaling |
title_short |
SmallTail: Scaling cores and probabilistic cloning requests for web systems |
title_full |
SmallTail: Scaling cores and probabilistic cloning requests for web systems |
title_fullStr |
SmallTail: Scaling cores and probabilistic cloning requests for web systems |
title_full_unstemmed |
SmallTail: Scaling cores and probabilistic cloning requests for web systems |
title_sort |
SmallTail: Scaling cores and probabilistic cloning requests for web systems |
dc.subject.none.fl_str_mv |
Controllers Level control Quality of service Virtual machine Websites Autoscaling Inner loop controls Latency control Level controllers Performance variability Probabilistic cloning Probablistic Quality of experience (qoe) Cloning Probablistic cloning Query cloning Tail latency control Two level controller Vertical autoscaling |
topic |
Controllers Level control Quality of service Virtual machine Websites Autoscaling Inner loop controls Latency control Level controllers Performance variability Probabilistic cloning Probablistic Quality of experience (qoe) Cloning Probablistic cloning Query cloning Tail latency control Two level controller Vertical autoscaling |
description |
Users quality of experience on web systems are largely determined by the tail latency, e.g., 95th percentile. Scaling resources along, e.g., the number of virtual cores per VM, is shown to be effective to meet the average latency but falls short in taming the latency tail in the cloud where the performance variability is higher. The prior art shows the prominence of increasing the request redundancy to curtail the latency either in the off-line setting or without scaling-in cores of virtual machines. In this paper, we propose an opportunistic scaler, termed SmallTail, which aims to achieve stringent targets of tail latency while provisioning a minimum amount of resources and keeping them well utilized. Against dynamic workloads, SmallTail simultaneously adjusts the core provisioning per VM and probabilistically replicates requests so as to achieve the tail latency target. The core of SmallTail is a two level controller, where the outer loops controls the core provision per distributed VMs and the inner loop controls the clones in a finer granularity. We also provide theoretical analysis on the steady-state latency for a given probabilistic replication that clones one out of N arriving requests. We extensively evaluate SmallTail on three different web systems, namely web commerce, web searching, and web bulletin board. Our testbed results show that SmallTail can ensure the 95th latency below 1000 ms using up to 53% less cores compared to the strategy of constant cloning, whereas scaling-core only solution exceeds the latency target by up to 70%. © 2018 IEEE. |
publishDate |
2018 |
dc.date.none.fl_str_mv |
2018 2020-05-25T23:58:24Z |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/conferenceObject |
dc.type.coarversion.fl_str_mv |
http://purl.org/coar/version/c_970fb48d4fbd8a85 |
dc.type.coar.fl_str_mv |
http://purl.org/coar/resource_type/c_c94f |
dc.identifier.none.fl_str_mv |
https://doi.org/10.1109/ICAC.2018.00013 https://repository.urosario.edu.co/handle/10336/22856 |
url |
https://doi.org/10.1109/ICAC.2018.00013 https://repository.urosario.edu.co/handle/10336/22856 |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
https://www.scopus.com/inward/record.uri?eid=2-s2.0-85061316455&doi=10.1109%2fICAC.2018.00013&partnerID=40&md5=87c34c798122078edea62f43188634cd |
dc.rights.coar.fl_str_mv |
http://purl.org/coar/access_right/c_abf2 |
rights_invalid_str_mv |
http://purl.org/coar/access_right/c_abf2 |
dc.format.none.fl_str_mv |
application/pdf |
dc.publisher.none.fl_str_mv |
Institute of Electrical and Electronics Engineers Inc. |
publisher.none.fl_str_mv |
Institute of Electrical and Electronics Engineers Inc. |
dc.source.none.fl_str_mv |
instname:Universidad del Rosario reponame:Repositorio Institucional EdocUR |
instname_str |
Universidad del Rosario |
institution |
Universidad del Rosario |
reponame_str |
Repositorio Institucional EdocUR |
collection |
Repositorio Institucional EdocUR |
repository.name.fl_str_mv |
|
repository.mail.fl_str_mv |
|
_version_ |
1803710415617654784 |