Build Cloud Download Site Using One Docker (FileBrowser+Aria2+AriaNg+Rclone+Caddy)

This post is regarding how to use wahyd4/aria2-ui's docker image to achieve a downloading and file managing / sharing center. In just one docker image, you can do downloading (Aria2), Web GUI management for your downloading task (AriaNG) , File Managing (FileBrowser), supporting multi-type file view, and downloaded  files will be saved to Google Drive or other cloud drives.

Topology

Pre-requirement:

Install Docker, Portainer, Nginx:

• Install Docker, Docker-Compose, Portainer & Nginx on CentOS8

Open Firewall port at cloud firewall and local firewall if enabled:

• for tcp 8000
  
  

#CentOS 6
iptables -A INPUT -p tcp --dport 8000 -j ACCEPT
service iptables save
service iptables restart

#CentOS 7
firewall-cmd --zone=public --add-port=8000/tcp --permanent
firewall-cmd --reload

Github Project and Docker Image Information

Github project : 

• wahyd4/aria2-ariang-docker

Docker images:

• wahyd4/aria2-ui:latest

Other related GitHub Sites:

• https://github.com/wahyd4/aria2-ariang-x-docker-compose
• https://github.com/51sec/aria2-ariang-x-docker-compose

Features:

• Aria2 (SSL support)
• AriaNg
• Rclone
• File Browser: Files mangement and videos playing
• Auto HTTPS （Let's Encrypt）
• Bind non root user into container, so non root user can also manage downloaded files.
• Basic Auth
• Support ARM CPUs as well, all supported CPU platforms can be found here
• Cloud Storage platforms synchronization

Quick Run for Testing

docker run -d --name aria2-ui -p 80:80 wahyd4/aria2-ui

Here are default URL and user name / password. 

• Aria2: http://yourip
• FileManger: http://yourip/files
• Rclone: http://yourip/rclone
• Please use admin/admin as username and password to login Filebrowser for the first time. And if rclone enabled, please use user/password to login Rclone if you didn't update ARIA2_USER and ARIA2_PWD
• ENABLE_RCLONE set to false to disable Rclone in the docker

Docker Run Command Based Your Requirements

Quick run is not exactly what we want. For my configuration, I would like to put this docker into different port, 8000, also I will need to map my host folders into docker. No need rclone to run in the docker, I will get it running on my host to mount my Google Drive. Also I will need some kind of authentication to be enabled for AriaNG web gui. 

1. Change Docker Port to 8000 since Nginx using port 80 & 443
2. Enable Basic Authentication
3. After completed downloading, file will be moved to Cloud Drive, such as Google Drive
4. Nginx will be used as reverse porxy for domain access downloading site

docker run -d --name aria2-ui \
  --restart=unless-stopped \
  -p 8000:80 \
  -e ENABLE_AUTH=true \
  -e ARIA2_SSL=false \
  -e ARIA2_USER=user \
  -e ARIA2_PWD=password \
  -e ARIA2_EXTERNAL_PORT=8000 \
  -e ENABLE_RCLONE=False \
  -v ~/data:/data \
  -v ~/gdrive:/gdrive \
  wahyd4/aria2-ui

Notes:

• Both data and gdrive folders will be automatically created under host's /root folder
• /data folder is aria2 docker's downloading folder. After finished downloading, file will be transferred to /gdrive folder automatically with a script.
• /gdrive folder is host's Rclone mounted Google Drive folder， which is mapped to ~/gdrive folder (/root/gdrive). 

Configure FileBrowser to Manage Gdrive (Optional)

This is an option to do if you would like FileBrowser web gui to manage your mounted google drive. Default it is managing download folder /data. 

Change File Browser's folder. vi /app/Procfile
Change default /data folder to /gdrive. So File Browser will open /gdrive when you log in. You can mange your Google Drive from File Browser directly. It is not necessary to do this step if you does not want to manage your Google Drive folder from FileBrowser page. 

[root@centos-nextcloud-aria2 ~]docker exec -it aria2-ui /bin/bash
bash-5.0# ls
Procfile        aria2c.sh       caddy.sh        conf            filebrowser     filebrowser.db  forego          init.sh         start.sh
bash-5.0# vi Procfile
filebrowser: /app/filebrowser -p 8080 -d /app/filebrowser.db -r /gdrive
caddy: /app/caddy.sh
aria2c: /app/aria2c.sh
[root@centos-nextcloud-aria2 ~]# docker restart aria2-ui
bash-5.0# cd /app

Configure Aria2 to Move Completed File to /Gdrive

[root@centos-nextcloud-aria2 ~]docker exec -it aria2-ui /bin/bash

Inside the docker, create a shell script , vi /app/conf/rcloneupload.sh,with following code：

bash-4.3#vi /app/conf/rcloneupload.sh

#!/bin/bash

GID="$1";
FileNum="$2";
File="$3";
MinSize="5"  #限制最低上传大小，默认5k
MaxSize="157286400"  #限制最高文件大小(单位k)，默认15G
RemoteDIR="/gdrive/";  #rclone挂载的本地文件夹，最后面保留/
LocalDIR="/data/";  #Aria2下载目录，最后面保留/

if [[ -z $(echo "$FileNum" |grep -o '[0-9]*' |head -n1) ]]; then FileNum='0'; fi
if [[ "$FileNum" -le '0' ]]; then exit 0; fi
if [[ "$#" != '3' ]]; then exit 0; fi

function LoadFile(){
  IFS_BAK=$IFS
  IFS=$'\n'
  if [[ ! -d "$LocalDIR" ]]; then return; fi
  if [[ -e "$File" ]]; then
    FileLoad="${File/#$LocalDIR}"
    while true
      do
        if [[ "$FileLoad" == '/' ]]; then return; fi
        echo "$FileLoad" |grep -q '/';
        if [[ "$?" == "0" ]]; then
          FileLoad=$(dirname "$FileLoad");
        else
          break;
        fi;
      done;
    if [[ "$FileLoad" == "$LocalDIR" ]]; then return; fi
    EXEC="$(command -v mv)"
    if [[ -z "$EXEC" ]]; then return; fi
    Option=" -f";
    cd "$LocalDIR";
    if [[ -e "$FileLoad" ]]; then
      ItemSize=$(du -s "$FileLoad" |cut -f1 |grep -o '[0-9]*' |head -n1)
      if [[ -z "$ItemSize" ]]; then return; fi
      if [[ "$ItemSize" -le "$MinSize" ]]; then
        echo -ne "\033[33m$FileLoad \033[0mtoo small to spik.\n";
        return;
      fi
      if [[ "$ItemSize" -ge "$MaxSize" ]]; then
        echo -ne "\033[33m$FileLoad \033[0mtoo large to spik.\n";
        return;
      fi
      eval "${EXEC}${Option}" \'"${FileLoad}"\' "${RemoteDIR}";
    fi
  fi
  IFS=$IFS_BAK
}
LoadFile;

make file become executable: chmod +x /app/conf/rcloneupload.sh

Edit Aria2 configuration file (vi /app/conf/aria2.conf) to add one following line at the file end: ($ in vi is to move to the end of line)

on-download-complete=/app/conf/rcloneupload.sh

Restart aria2-ui docker to take change into effect

[root@centos-nextcloud-aria2 ~]# docker restart aria2-ui
bash-4.3# cd /app/conf/
bash-4.3# ls
aria2.conf      aria2.session      aria2c.sh      key

bash-4.3# vi /app/conf/aria2.conf
# Bit Torrent: The amount of time and the upload-to-download ratio you wish to
# seed to. If either the time limit ( seconds ) or the seed ratio is reached,
# torrent seeding will stop. You can set seed-time to zero(0) to disable
# seeding completely.
seed-ratio=0.01
seed-time=1
on-download-complete=/app/conf/rcloneupload.sh

Configure Host Rclone to mount Google Drive

1   First to install epel source

yum -y install epel-release

Note:In Ubuntu, just use this command to install rclone: apt install rclone

2  Install some components
yum -y install wget unzip screen fuse fuse-devel


3  Install rclone
[root@centos7-test1 data]# curl https://rclone.org/install.sh | sudo bash

In Ubuntu, apt install rclone

4  configure rclone
rclone config

create a new remote configuration and name it as google-drive

Other steps for rclone configuration, you can check following posts: 

• https://blog.51sec.org/2020/05/use-rclone-to-mount-google-drive-and.html
• Mount Google Drive into NextCloud Using Rclone

5  mount Google Drive using rclone

• create a new folder at /root/gdrive
• note: in our previous settings,  /root/gdrive has been created. 

mkdir -p /root/gdrive

• mount system

rclone mount google-drive: /root/gdrive --allow-other --allow-non-empty --vfs-cache-mode writes

Note: google-drive is the Rclone configuration name.


6   create rclone.service

To make rclone mount the google drive even after rebooted the vps, create /usr/lib/systemd/system/rclone.service with following information:
vi /usr/lib/systemd/system/rclone.service

[Unit]
Description=rclone

[Service]
User=root
ExecStart=/usr/bin/rclone mount google-drive: /root/gdrive --allow-other --allow-non-empty --vfs-cache-mode writes
Restart=on-abort

7  systemctl enable rclone.service

Strongly suggest to reboot system to confirm it is working after a reboot. 

8  Using browser to access system

FileBrowser: http://ip/files
aria2ng: http://ip

Default FileBrowser username/password will be admin/admin.

Aria will download files to docker's local /data folder . Once completed downloading, a script will be trigged to move the file to /gdrive folder.

Gdrive folder has been mounted with Google Drive. You can configure filebrowser to manage either temp download folder /data, or Google Drive folder /gdrive.

FileBrowse Page

http://download.51sec.eu.org/files/

It will take a while to show you all files since they are in your Rclone's connection to Google Drive. 

Edit users or add users into the Filebrowser. Default usernmae and password is admin/admin. 

Cloudflare Configuration

DNS A record should set to no proxy, else you will have a problem to get your AriaNG to connect to Aria2's rpc port 8000.

You will need to add a new configuration file into your Nginx. It can be copied from portainer.conf file. 

AriaNG Configuration

You will not need to make any change here as long as you disabled proxy status from Cloudflare. But if there is a problem with AriaNG connectingto Aria2 RPC port, you might need to make some changes here, such changing from domain name to ip address.

Remove Installed Docker

#Delete Docker ContainerID=`docker ps|grep wahyd4/aria2-ui|awk '{print $1}'` docker kill ${ContainerID} docker rm ${ContainerID} docker rmi `docker images|grep wahyd4/aria2-ui|awk '{print $3}'` #Remove Download Folder rm -rf ~/data
rm -rf /home/gdrive

YouTube Videos

References