Using Rclone To Add Free Cloud Storage to VPS & Execute Read/Write Performance Test

This guide walks you through the steps to mount all kinds cloud drives locally using Rclone. It use Google Drive as an example, and has been tested on Ubuntu 20.04 Linux, but the steps are same for other cloud drives, also fits for all other Linux and Unix distributions. 

For Windows, please check following post:

• Use Rclone To Mount Azure Blob Storage into Local Windows File System

Linux Rclone related posts:

• Build Cloud Download Site Using One Docker (FileBrowser+Aria2+AriaNg+Rclone+Caddy)
• Use Aria2 docker+ariang to download and Rclone to Mount Google Drive and Sync
• Mount Google Drive into NextCloud Using Rclone

Similar post to add cloud storage for VPS:

• Add FREE 75GB Storage to Your Own VPS (Scaleway S3 Free Storage Bucket)

Diagram

Installation

Ubuntu: 

apt update -y && apt install rclone -y

CentOS

yum -y install epel-release
yum -y install wget unzip screen fuse fuse-devel
curl https://rclone.org/install.sh | sudo bash

Rclone Config

 

Please visit https://rclone.org/drive/ to get deep and detailed explanation for all configuration. 

root@ubuntu-20-04-rclone:~# rclone config
2022/10/01 23:50:26 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> gd
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / 1Fichier
   \ "fichier"
 2 / Alias for an existing remote
   \ "alias"
 3 / Amazon Drive
   \ "amazon cloud drive"
 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
   \ "s3"
 5 / Backblaze B2
   \ "b2"
 6 / Box
   \ "box"
 7 / Cache a remote
   \ "cache"
 8 / Citrix Sharefile
   \ "sharefile"
 9 / Dropbox
   \ "dropbox"
10 / Encrypt/Decrypt a remote
   \ "crypt"
11 / FTP Connection
   \ "ftp"
12 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
13 / Google Drive
   \ "drive"
14 / Google Photos
   \ "google photos"
15 / Hubic
   \ "hubic"
16 / JottaCloud
   \ "jottacloud"
17 / Koofr
   \ "koofr"
18 / Local Disk
   \ "local"
19 / Mail.ru Cloud
   \ "mailru"
20 / Microsoft Azure Blob Storage
   \ "azureblob"
21 / Microsoft OneDrive
   \ "onedrive"
22 / OpenDrive
   \ "opendrive"
23 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
24 / Pcloud
   \ "pcloud"
25 / Put.io
   \ "putio"
26 / SSH/SFTP Connection
   \ "sftp"
27 / Transparently chunk/split large files
   \ "chunker"
28 / Union merges the contents of several remotes
   \ "union"
29 / Webdav
   \ "webdav"
30 / Yandex Disk
   \ "yandex"
31 / http Connection
   \ "http"
32 / premiumize.me
   \ "premiumizeme"
Storage> 13
** See help for drive backend at: https://rclone.org/drive/ **
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id>
Google Application Client Secret
Setting your own is recommended.
Enter a string value. Press Enter for the default ("").
client_secret>
Scope that rclone should use when requesting access from drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Full access all files, excluding Application Data Folder.
   \ "drive"
 2 / Read-only access to file metadata and file contents.
   \ "drive.readonly"
   / Access to files created by rclone only.
 3 | These are visible in the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ "drive.file"
   / Allows read and write access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ "drive.appfolder"
   / Allows read-only access to file metadata but
 5 | does not allow any access to read or download file content.
   \ "drive.metadata.readonly"
scope> 1
ID of the root folder
Leave blank normally.
Fill in to access "Computers" folders (see docs), or for rclone to use
a non root folder as its starting point.
Note that if this is blank, the first time rclone runs it will fill it
in with the ID of the root folder.
Enter a string value. Press Enter for the default ("").
root_folder_id>
Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Enter a string value. Press Enter for the default ("").
service_account_file>
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> n
If your browser doesn't open automatically go to the following link: https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=202264815644.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&state=NuNX1w_yc8CSxTkS1RQ
Log in and authorize rclone for access
Enter verification code> 4/1ARtbsJrzLdsfasfsfasdfasdfasdfsadfswerqwerJ-bwmWyDdMo
Configure this as a team drive?
y) Yes
n) No
y/n> n
--------------------
[gd]
scope = drive
token = {"access_token":"ya29.a0Aa4xrXMF-PorKtxgrBByOVDzDaMcnQCiJ2PK3TLitkBccl3Ct3LJJV-Wl96ubVjPAhIhI9jwerqwerzQc_xj9t_b0PeSw6B3K8W5-iVcwnL5vhAjUyucfpadsfasdfbaicvaCgYKATASARMSFQEjDvL9y6KpqM9oegjJfiGv1wkqJw0163","token_type":"Bearer","refresh_token":"1//04nUNdP-4fdJSCgYIARAAGAQSNwF-L9Ircm74TeQh_6g0clYrlAf8MVgQChIpyFr0Rpbhjv-lAe7aVxc2f9gEnadsfwerwrLRglw","expiry":"2022-10-02T00:53:06.384873373Z"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name                 Type
====                 ====
gd                   drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
root@ubuntu-20-04-rclone:~# rclone lsd gd:

Mount to local

 

mkdir -p /home/gdrive
rclone mount gd: /home/gdrive --allow-other --allow-non-empty --vfs-cache-mode writes

Tip:gdis the Rclone configuration name.

Tip: ctrl+c to unmount in the ssh session

fusermount -qzu /home/gdrive

Tip: You also can mount one Google Drive's subfolder (for example, subfolder : backup) to local.

rclone mount gd:backup /home/gdrive --allow-other --allow-non-empty --vfs-cache-mode writes

You also can only mount to Google Drive certain folder by changing root_folder_id at the file /root/.config/rclone/rclone.conf

[root@centos7-test1 rclone]# cat rclone.conf
[google-drive]
type = drive
scope = drive
token = {"access_token":"ya29.a0Ae4lwerwerqertR-TaIh2_I3et3TqsXxbr73XnS0Gj19zalB4xke2FXosB8kThTG8akkjqc94ZjOoY8sLOokoxZf26YrUf04F14abUGRaGZTWznSGB","token_type":"Bearer","refresh_token":"1//0fqzXgAuJUWU2CgYIARAGA8SNwF-L9IrsitYDMEfAeJgOutDZV7OyqaWLjeoaj9mDw-e-4beZlwucBY8Br32tQM5cXFl7BnyLo","expiry":"2020-05-08T20:00:53.118819799Z"}
root_folder_id = 1lMF61yEsdfwerqwBwCH

For subfolder's root_folder_id, you can find it from the drive folder's url as shown below:

Verify using command "df -k" to check local drivers, or using following command to check cloud drives:

root@ubuntu-20-04-rclone:~# rclone lsd gd:
          -1 2022-09-30 10:47:33        -1 Colab Notebooks
          -1 2022-05-02 20:34:48        -1 Temp

root@ubuntu-20-04-rclone:~# df -k
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              455492       0    455492   0% /dev
tmpfs              98732    1052     97680   2% /run
/dev/sda1       47143192 2217276  44909532   5% /
tmpfs             493656       0    493656   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             493656       0    493656   0% /sys/fs/cgroup
/dev/loop0         56960   56960         0 100% /snap/core18/2538
/dev/loop1         63488   63488         0 100% /snap/core20/1593
/dev/loop2         69504   69504         0 100% /snap/lxd/22753
/dev/loop4         60928   60928         0 100% /snap/oracle-cloud-agent/42
/dev/loop3         48128   48128         0 100% /snap/snapd/16292
/dev/sda15        106858    5321    101537   5% /boot/efi
tmpfs              98728       0     98728   0% /run/user/1001
gd:             15728640 2863200   6689528  30% /home/gdrive

If you mounted as Team drive, you will get about 1PB for your gd: folder:

root@ubuntu-20-04-rclone:~# df -k
Filesystem         1K-blocks    Used     Available Use% Mounted on
udev                  455492       0        455492   0% /dev
tmpfs                  98732    1044         97688   2% /run
/dev/sda1           47143192 2217740      44909068   5% /
tmpfs                 493656       0        493656   0% /dev/shm
tmpfs                   5120       0          5120   0% /run/lock
tmpfs                 493656       0        493656   0% /sys/fs/cgroup
/dev/loop0             56960   56960             0 100% /snap/core18/2538
/dev/loop1             63488   63488             0 100% /snap/core20/1593
/dev/loop2             69504   69504             0 100% /snap/lxd/22753
/dev/loop3             60928   60928             0 100% /snap/oracle-cloud-agent/42
/dev/loop4             48128   48128             0 100% /snap/snapd/16292
/dev/sda15            106858    5321        101537   5% /boot/efi
gd:            1099511627776       0 1099511627776   0% /home/gdrive
tmpfs                  98728       0         98728   0% /run/user/1001
root@ubuntu-20-04-rclone:~#

Make it become service to start with system 

To make rclone mount the google drive even after rebooted the vps, create /usr/lib/systemd/system/rclone.service with following information:

vi /usr/lib/systemd/system/rclone.service

[Unit]
Description=rclone 

[Service]
User=root
ExecStart=/usr/bin/rclone mount gd: /home/gdrive --allow-other --allow-non-empty --vfs-cache-mode writes
Restart=on-abort

 
[Install]
WantedBy=multi-user.target

Start the service and enable service to start with system:

systemctl start rclone
systemctl enable rclone

Use DD Command to Execute Disk Performance Test

Using internal client ID :

Local HardDrive Performance Test

root@ubuntu-20-04-rclone:~# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 0.124241 s, 675 MB/s

Google Drive Performance Test:

root@ubuntu-20-04-rclone:~# dd if=/dev/zero of=/home/gdrive/output bs=8k count=10k; rm -f /home/gdrive/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 10.9557 s, 7.7 MB/s

root@ubuntu-20-04-rclone:~# dd if=/dev/zero of=/home/gdrive/output bs=384k count=1k; rm -f /home/gdrive/output
1024+0 records in
1024+0 records out
402653184 bytes (403 MB, 384 MiB) copied, 25.1151 s, 16.0 MB/s
root@ubuntu-20-04-rclone:~#

Make Your Own Client ID and Test Performance

 Document: https://rclone.org/drive/#making-your-own-client-id

Using our own Client ID:

root@ubuntu-20-04-rclone:~# dd if=/dev/zero of=/home/gdrive/output bs=8k count=10k; rm -f /home/gdrive/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 10.3352 s, 8.1 MB/s
root@ubuntu-20-04-rclone:~# dd if=/dev/zero of=/home/gdrive/output bs=384k count=1k; rm -f /home/gdrive/output
1024+0 records in
1024+0 records out
402653184 bytes (403 MB, 384 MiB) copied, 27.9803 s, 14.4 MB/s
root@ubuntu-20-04-rclone:~# dd if=/dev/zero of=/home/gdrive/output bs=384k count=1k; rm -f /home/gdrive/output
1024+0 records in
1024+0 records out
402653184 bytes (403 MB, 384 MiB) copied, 23.4585 s, 17.2 MB/s
root@ubuntu-20-04-rclone:~#

Videos

 

References

• Build Cloud Download Site Using One Docker (FileBrowser+Aria2+AriaNg+Rclone+Caddy)
• Use Aria2 docker+ariang to download and Rclone to Mount Google Drive and Sync
• Mount Google Drive into NextCloud Using Rclone