Friday, March 1, 2019

VPS Bench Comparison for AWS, Azure and Google Cloud (Free Tier & Regular Tier)


In this post, I am put all of following topics together
  • Benchmark free services for AWS, Azure, GCP
  • Another bench script
  • Benchmark regular size VM on AWS, Azure, GCP
  • How to find out what kind of hypervisor your cloud virtual machine is using?

Benchmark Free Services for AWS, Azure, GCP
All of AWS, Azure and Google Cloud are providing some kind of free tier to the users. I am always wondering how those free tier VPS performance looks like. I did some bench work to test them out.

Here is the method and result.

I am going to use some bench scripts found from github and running them through the same size VM created on AWS, Azure and GCP. It will benchmark their CPU, Disk IO, network throughput




First Bench script:
https://github.com/n-st/nench provides a nice bench.sh and a single command for you to run this script on your VPS.


Commands to run this script:

(curl -s wget.racing/nench.sh | bash; curl -s wget.racing/nench.sh | bash) 2>&1 | tee nench.log
(wget -qO- wget.racing/nench.sh | bash; wget -qO- wget.racing/nench.sh | bash) 2>&1 | tee nench.log


Basically this benchmark script will do following tasks:
  • get system info
  • benchmark the processor and memory (both single and multi core)
  • benchmark the disk
  • benchmark networking

AWS Free Tier 
Ubuntu 18.04 - Type t2.micro, vCPU 1, Memory 1GiB, Storage EBS only

-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-03-01 16:02:20 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
CPU cores:    1
Frequency:    2399.859 MHz
RAM:          983M
Swap:         -
Kernel:       Linux 4.15.0-1021-aws x86_64

Disks:
loop0   87.9M  HDD
loop1   12.7M  HDD
xvda      8G  SSD

CPU: SHA256-hashing 500 MB
    3.282 seconds
CPU: bzip2-compressing 500 MB
    5.570 seconds
CPU: AES-encrypting 500 MB
    1.663 seconds

ioping: seek rate
    min/avg/max/mdev = 249.4 us / 353.0 us / 5.39 ms / 151.4 us
ioping: sequential read speed
    generated 1.47 k requests in 5.00 s, 366.2 MiB, 292 iops, 73.2 MiB/s

dd: sequential write speed
    1st run:    64.66 MiB/s
    2nd run:    61.32 MiB/s
    3rd run:    61.13 MiB/s
    average:    62.37 MiB/s

IPv4 speedtests
    your IPv4:    54.163.88.xxxx

    Cachefly CDN:         60.98 MiB/s
    Leaseweb (NL):        6.55 MiB/s
    Softlayer DAL (US):   23.52 MiB/s
    Online.net (FR):      4.98 MiB/s
    OVH BHS (CA):         22.21 MiB/s

No IPv6 connectivity detected
-------------------------------------------------

Azure Free Services 
Size B1S , vCPU 1, Memory 1GiB, Max IOPS 400.
-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-03-01 16:04:44 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
CPU cores:    1
Frequency:    2294.687 MHz
RAM:          922M
Swap:         -
Kernel:       Linux 4.15.0-1037-azure x86_64

Disks:
sda     30G  HDD
sdb      4G  HDD

CPU: SHA256-hashing 500 MB
    3.527 seconds
CPU: bzip2-compressing 500 MB
    5.900 seconds
CPU: AES-encrypting 500 MB
    1.759 seconds

ioping: seek rate
    min/avg/max/mdev = 40.5 us / 1.64 ms / 48.8 ms / 8.03 ms
ioping: sequential read speed
    generated 476 requests in 5.03 s, 119 MiB, 94 iops, 23.6 MiB/s

dd: sequential write speed
    1st run:    17.07 MiB/s
    2nd run:    17.07 MiB/s
    3rd run:    17.07 MiB/s
    average:    17.07 MiB/s

IPv4 speedtests
    your IPv4:    52.168.16.xxxx

    Cachefly CDN:         74.85 MiB/s
    Leaseweb (NL):        19.27 MiB/s
    Softlayer DAL (US):   42.60 MiB/s
    Online.net (FR):      17.30 MiB/s
    OVH BHS (CA):         50.53 MiB/s

No IPv6 connectivity detected
-------------------------------------------------


GCP Always Free Tier  
Ubuntu 18.04 , type f1-micro, 1vCPU, 0.6 GB Memory
-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-03-01 16:02:49 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU @ 2.30GHz
CPU cores:    1
Frequency:    2300.000 MHz
RAM:          581M
Swap:         -
Kernel:       Linux 4.15.0-1027-gcp x86_64

Disks:
loop0     91M  HDD
loop1   55.7M  HDD
sda     10G  HDD

CPU: SHA256-hashing 500 MB
    3.259 seconds
CPU: bzip2-compressing 500 MB
    5.633 seconds
CPU: AES-encrypting 500 MB
    1.280 seconds

ioping: seek rate
    min/avg/max/mdev = 363.8 us / 7.42 ms / 63.7 ms / 10.9 ms
ioping: sequential read speed
    generated 2.47 k requests in 5.00 s, 617.8 MiB, 493 iops, 123.5 MiB/s

dd: sequential write speed
    1st run:    35.76 MiB/s
    2nd run:    35.86 MiB/s
    3rd run:    35.86 MiB/s
    average:    35.83 MiB/s

IPv4 speedtests
    your IPv4:    35.196.31.xxxx

    Cachefly CDN:         94.34 MiB/s
    Leaseweb (NL):        18.63 MiB/s
    Softlayer DAL (US):   30.92 MiB/s
    Online.net (FR):      13.99 MiB/s
    OVH BHS (CA):         9.03 MiB/s

No IPv6 connectivity detected
-------------------------------------------------



I put all results in a table to do a comparison. Green color is the winner on that item.
AWS
Azure
GCP
-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-03-01 16:02:20 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
CPU cores:    1
Frequency:    2399.859 MHz
RAM:          983M
Swap:         -
Kernel:       Linux 4.15.0-1021-
aws x86_64

Disks:
loop0   87.9M  HDD
loop1   12.7M  HDD
xvda      8G  SSD
CPU: SHA256-hashing 500 MB
    3.282 seconds
CPU: bzip2-compressing 500 MB
   
5.570 seconds
CPU: AES-encrypting 500 MB
    1.663 seconds

ioping: seek rate
    min/avg/max/mdev = 249.4 us / 353.0 us / 5.39 ms / 151.4 us
ioping: sequential read speed
    generated 1.47 k requests in 5.00 s, 366.2 MiB, 292 iops, 73.2 MiB/s

dd: sequential write speed
    1st run:    64.66 MiB/s
    2nd run:    61.32 MiB/s
    3rd run:    61.13 MiB/s
    average:    62.37 MiB/s


IPv4 speedtests
    your IPv4:    54.163.88.xxxx

    Cachefly CDN:         60.98 MiB/s
    Leaseweb (NL):        6.55 MiB/s
    Softlayer DAL (US):   23.52 MiB/s
    Online.net (FR):      4.98 MiB/s
    OVH BHS (CA):         22.21 MiB/s


No IPv6 connectivity detected
-------------------------------------------------

-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-03-01 16:04:44 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
CPU cores:    1
Frequency:    2294.687 MHz
RAM:          922M
Swap:         -
Kernel:       Linux 4.15.0-1037-
azure x86_64

Disks:
sda     30G  HDD
sdb      4G  HDD

CPU: SHA256-hashing 500 MB
    3.527 seconds
CPU: bzip2-compressing 500 MB
    5.900 seconds
CPU: AES-encrypting 500 MB
    1.759 seconds


ioping: seek rate
    min/avg/max/mdev = 40.5 us / 1.64 ms / 48.8 ms / 8.03 ms
ioping: sequential read speed
    generated 476 requests in 5.03 s, 119 MiB, 94 iops, 23.6 MiB/s

dd: sequential write speed
    1st run:    17.07 MiB/s
    2nd run:    17.07 MiB/s
    3rd run:    17.07 MiB/s
    average:    17.07 MiB/s


IPv4 speedtests
    your IPv4:    52.168.16.xxxx

    Cachefly CDN:         74.85 MiB/s
    Leaseweb (NL):        19.27 MiB/s
    Softlayer DAL (US):   42.60 MiB/s
    Online.net (FR):      17.30 MiB/s
    OVH BHS (CA):         50.53 MiB/s


No IPv6 connectivity detected
-------------------------------------------------

-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-03-01 16:02:49 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU @ 2.30GHz
CPU cores:    1
Frequency:    2300.000 MHz
RAM:          581M
Swap:         -
Kernel:       Linux 4.15.0-1027-
gcp x86_64

Disks:
loop0     91M  HDD
loop1   55.7M  HDD
sda     10G  HDD

CPU: SHA256-hashing 500 MB
    3.259 seconds
CPU: bzip2-compressing 500 MB
    5.633 seconds
CPU: AES-encrypting 500 MB
    1.280 seconds

ioping: seek rate
    min/avg/max/mdev = 363.8 us / 7.42 ms / 63.7 ms / 10.9 ms
ioping: sequential read speed
    generated 2.47 k requests in 5.00 s, 617.8 MiB, 493 iops, 123.5 MiB/s

dd: sequential write speed
    1st run:    35.76 MiB/s
    2nd run:    35.86 MiB/s
    3rd run:    35.86 MiB/s
    average:    35.83 MiB/s


IPv4 speedtests
    your IPv4:    35.196.31.xxxx

    Cachefly CDN:         94.34 MiB/s
    Leaseweb (NL):        18.63 MiB/s
    Softlayer DAL (US):   30.92 MiB/s
    Online.net (FR):      13.99 MiB/s
    OVH BHS (CA):        
9.03 MiB/s
No IPv6 connectivity detected
-------------------------------------------------



Azure seems slowest one from all results. I just did another bench for a bigger size vm on Azure. I choose Standard D1 (1 vcpus, 3.5 GB memory, Max IOPS 4x500. ).
Lets see the bench result:
[[email protected] ~]$ (curl -s wget.racing/nench.sh | bash; curl -s wget.racing/ne                                                                                                                                                             nch.sh | bash) 2>&1 | tee nench.log
-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-03-01 14:51:01 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
CPU cores:    1
Frequency:    2194.503 MHz
RAM:          3.3G
Swap:         -
Kernel:       Linux 3.10.0-862.11.6.el7.x86_64 x86_64

Disks:
sda     30G  HDD
sdb     50G  HDD

CPU: SHA256-hashing 500 MB
    3.300 seconds
CPU: bzip2-compressing 500 MB
    9.939 seconds
CPU: AES-encrypting 500 MB
    2.646 seconds

ioping: seek rate
    min/avg/max/mdev = 87.8 us / 177.6 us / 196.0 ms / 1.35 ms
ioping: sequential read speed
    generated 14.9 k requests in 5.00 s, 3.65 GiB, 2.99 k iops, 747.1 MiB/s

dd: sequential write speed
    1st run:    11.35 MiB/s
    2nd run:    14.88 MiB/s
    3rd run:    18.31 MiB/s
    average:    14.85 MiB/s

IPv4 speedtests
    your IPv4:    13.68.179.xxxx

    Cachefly CDN:         130.72 MiB/s
    Leaseweb (NL):        19.16 MiB/s
    Softlayer DAL (US):   37.52 MiB/s
    Online.net (FR):      15.01 MiB/s
    OVH BHS (CA):         43.25 MiB/s

No IPv6 connectivity detected
-------------------------------------------------

[[email protected] ~]$

YouTube Video:


======================================================================
Another bench scrip

This was running on Azure Free Service VM:

[email protected]:~$ sudo wget -qO- bench.sh | bash
----------------------------------------------------------------------
CPU model            : Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Number of cores      : 1
CPU frequency        : 2294.681 MHz
Total size of Disk   : 33.0 GB (4.7 GB Used)
Total amount of Mem  : 896 MB (595 MB Used)
Total amount of Swap : 0 MB (0 MB Used)
System uptime        : 0 days, 4 hour 55 min
Load average         : 3.74, 1.14, 0.45
OS                   : Ubuntu 18.04.2 LTS
Arch                 : x86_64 (64 Bit)
Kernel               : 4.18.0-1019-azure
----------------------------------------------------------------------
I/O speed(1st run)   : 16.3 MB/s
I/O speed(2nd run)   : 17.5 MB/s
I/O speed(3rd run)   : 17.0 MB/s
Average I/O speed    : 16.9 MB/s
----------------------------------------------------------------------
Node Name                       IPv4 address            Download Speed
CacheFly                        205.234.175.175         140MB/s
Linode, Tokyo, JP               106.187.96.148          9.54MB/s
Linode, Singapore, SG           139.162.23.4            12.2MB/s
Linode, London, UK              176.58.107.39           127MB/s
Linode, Frankfurt, DE           139.162.130.8           123MB/s
Linode, Fremont, CA             50.116.14.9             16.0MB/s
Softlayer, Dallas, TX           173.192.68.18           11.1MB/s
Softlayer, Seattle, WA          67.228.112.250          9.16MB/s
Softlayer, Frankfurt, DE        159.122.69.4            71.2MB/s
Softlayer, Singapore, SG        119.81.28.170           7.19MB/s
Softlayer, HongKong, CN         119.81.130.170          5.75MB/s
----------------------------------------------------------------------
[email protected]:~$






======================================================================

Benchmark regular size VM on AWS, Azure, GCP
Recently, I did another bench comparison with much higher instance type , such as  4vCPUs 16GB memory.
Here is the result for this comparison.

AWS
t2.xlarge 4vCPUs 16GB memory

8GB gp2
Azure
b4ms 4vCPUs 16GB memory

8GB gp2 32GB Premium SSD
GCP
n1-standard-4 4vCPUs 15GB memory
Standard 10G Disk
-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-06-08 21:53:05 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
CPU cores:    4
Frequency:    2394.678 MHz
RAM:          15G
Swap:         -
Kernel:       Linux 4.15.0-1032-aws x86_64

Disks:
loop0     91M  HDD
loop1     18M  HDD
xvda      8G  SSD

CPU: SHA256-hashing 500 MB
    3.220 seconds
CPU: bzip2-compressing 500 MB
    5.539 seconds
CPU: AES-encrypting 500 MB
    1.615 seconds

ioping: seek rate
    min/avg/max/mdev = 161.4 us / 271.0 us / 9.23 ms / 157.3 us
ioping: sequential read speed
    generated 2.93 k requests in 5.00 s, 731.5 MiB, 585 iops, 146.3 MiB/s

dd: sequential write speed
    1st run:    125.89 MiB/s
    2nd run:    123.98 MiB/s
    3rd run:    123.98 MiB/s
    average:    124.61 MiB/s

IPv4 speedtests
    your IPv4:    35.182.143.xxxx

    Cachefly CDN:         65.88 MiB/s
    Leaseweb (NL):        7.56 MiB/s
    Softlayer DAL (US):   26.89 MiB/s
    Online.net (FR):      7.81 MiB/s
    OVH BHS (CA):         49.95 MiB/s

No IPv6 connectivity detected
-------------------------------------------------

-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-06-08 21:55:42 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
CPU cores:    4
Frequency:    2294.687 MHz
RAM:          15G
Swap:         -
Kernel:       Linux 4.18.0-1018-azure x86_64

Disks:
sda     30G  HDD
sdb     32G  HDD

CPU: SHA256-hashing 500 MB
    3.811 seconds
CPU: bzip2-compressing 500 MB
    6.656 seconds
CPU: AES-encrypting 500 MB
    2.275 seconds



ioping: seek rate
    min/avg/max/mdev = 40.1 us / 254.5 us / 29.7 ms / 1.69 ms
ioping: sequential read speed
    generated 714 requests in 5.03 s, 178.5 MiB, 141 iops, 35.5 MiB/s

dd: sequential write speed

    1st run:    17.36 MiB/s
    2nd run:    17.36 MiB/s
    3rd run:    17.26 MiB/s
    average:    17.33 MiB/s

IPv4 speedtests
    your IPv4:    40.83.217.xxxx

    Cachefly CDN:         113.25 MiB/s
    Leaseweb (NL):        16.42 MiB/s
    Softlayer DAL (US):   36.95 MiB/s
    Online.net (FR):      17.06 MiB/s
    OVH BHS (CA):         12.93 MiB/s

No IPv6 connectivity detected
-------------------------------------------------
-------------------------------------------------
 nench.sh v2019.03.01 -- https://git.io/nench.sh
 benchmark timestamp:    2019-06-08 21:52:59 UTC
-------------------------------------------------

Processor:    Intel(R) Xeon(R) CPU @ 2.30GHz
CPU cores:    4
Frequency:    2300 MHz
RAM:          14G
Swap:         -
Kernel:       Linux 4.15.0-1032-gcp x86_64

Disks:
loop0 88.4M HDD
loop1 58.9M HDD
sda     10G  HDD

CPU: SHA256-hashing 500 MB
    3.256 seconds
CPU: bzip2-compressing 500 MB
    5.515 seconds
CPU: AES-encrypting 500 MB
    1.230 seconds



ioping: seek rate
    min/avg/max/mdev = 314.9 us / 1.05 ms / 35.2 ms / 1.98 ms
ioping: sequential read speed
    generated 2.47 k requests in 5.00 s, 617.8 MiB, 493 iops, 123.5 MiB/s

dd: sequential write speed

    1st run:    111.58 MiB/s
    2nd run:    110.63 MiB/s
    3rd run:    110.63 MiB/s
    average:    110.94 MiB/s

IPv4 speedtests
    your IPv4:    35.224.86.xxxx

    Cachefly CDN:         141.04 MiB/s
    Leaseweb (NL):        22.15 MiB/s
    Softlayer DAL (US):   53.89 MiB/s
    Online.net (FR):      23.23 MiB/s
    OVH BHS (CA):         11.23 MiB/s

No IPv6 connectivity detected
-------------------------------------------------


======================================================================

How to find out what kind of hypervisor your cloud virtual machine is using?

virt-what is a shell script which can be used to detect if the program is running in a virtual machine.
virt-what supports a very large number of different hypervisor types, including common open source hypervisors (KVM, Xen, QEMU, VirtualBox), mainframe systems like IBM Systemz, LPAR, z/VM, hardware partitioning schemes like Hitachi Virtage, proprietary hypervisors like VMWare, Microsoft Hyper-V and much more.
yum -y install virt-what
sudo apt install virt-what
Run:
virt-what
Check the output
[email protected]:~$ sudo apt install virt-what
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  virt-what
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 14.7 kB of archives.
After this operation, 39.9 kB of additional disk space will be used.
Get:1 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 virt-what amd64 1.18-2 [14.7 kB]
Fetched 14.7 kB in 0s (213 kB/s)
Selecting previously unselected package virt-what.
(Reading database ... 88964 files and directories currently installed.)
Preparing to unpack .../virt-what_1.18-2_amd64.deb ...
Unpacking virt-what (1.18-2) ...
Setting up virt-what (1.18-2) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
[email protected]:~$ virt-what
virt-what: this script must be run as root
[email protected]:~$ sudo virt-what
hyperv
[email protected]:~$

References:





No comments:

Post a Comment