Comments

Latest Posts

Useful Linux Network Analysis/Monitoring/Backup Shell Scripts Collection

Some Scripts to help linux operation:
  • Speedtest script
  • Network Traffic Analysis Script
  • VPS Backup Script and ftp Upload
  • mysql/php/nginx monitoring script

Speedtest Script



wget https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz -O speedtest-cli.tgz && tar xfvz speedtest-cli.tgz && echo yes | ./speedtest


[[email protected] ~]# wget https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz -O speedtest-cli.tgz && tar xfvz speedtest-cli.tgz && echo yes | ./speedtest
--2020-04-16 17:21:41--  https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz
Resolving bintray.com (bintray.com)... 108.168.194.93
Connecting to bintray.com (bintray.com)|108.168.194.93|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://dl.bintray.com/ookla/download/ookla-speedtest-1.0.0-x86_64-linux.tgz?expiry=1587057731446&signature=N%2F%2FEyyWnLJRqFhHwYJ08IM0%2B0OU66hX1%2BgGCWG43CaY3dmuJOyA0M8gy36G2RwtgfT8Elro6jQpIhBd8yTOKNQ%3D%3D [following]
--2020-04-16 17:21:41--  https://dl.bintray.com/ookla/download/ookla-speedtest-1.0.0-x86_64-linux.tgz?expiry=1587057731446&signature=N%2F%2FEyyWnLJRqFhHwYJ08IM0%2B0OU66hX1%2BgGCWG43CaY3dmuJOyA0M8gy36G2RwtgfT8Elro6jQpIhBd8yTOKNQ%3D%3D
Resolving dl.bintray.com (dl.bintray.com)... 52.26.64.218, 52.11.170.179
Connecting to dl.bintray.com (dl.bintray.com)|52.26.64.218|:443... connected.
HTTP request sent, awaiting response... 302 
Location: https://akamai.bintray.com/5f/5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168?__gda__=exp=1587058421~hmac=bcc7e0e4e8f71f5d0af7ebf6178ae0534027fb63a80234c4870051da23c2fbfa&response-content-disposition=attachment%3Bfilename%3D%22ookla-speedtest-1.0.0-x86_64-linux.tgz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX19FmhEAfVfGnWNhHLMH9_FIedcu869F-5_L6eYlhAQ-vBUL-KjMmlOg3_Pt0gfPKOS-M8PpIXM7iVCKOdekGMaDStQwm92EfjfQDX_lGbiCXiYR9ao_wwmHjKOiB6RTgnyrDECxGx8spA&response-X-Checksum-Sha1=41ca19b8bea7614c27370453be3c6ef7ea7fa76a&response-X-Checksum-Sha2=5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168 [following]
--2020-04-16 17:21:41--  https://akamai.bintray.com/5f/5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168?__gda__=exp=1587058421~hmac=bcc7e0e4e8f71f5d0af7ebf6178ae0534027fb63a80234c4870051da23c2fbfa&response-content-disposition=attachment%3Bfilename%3D%22ookla-speedtest-1.0.0-x86_64-linux.tgz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX19FmhEAfVfGnWNhHLMH9_FIedcu869F-5_L6eYlhAQ-vBUL-KjMmlOg3_Pt0gfPKOS-M8PpIXM7iVCKOdekGMaDStQwm92EfjfQDX_lGbiCXiYR9ao_wwmHjKOiB6RTgnyrDECxGx8spA&response-X-Checksum-Sha1=41ca19b8bea7614c27370453be3c6ef7ea7fa76a&response-X-Checksum-Sha2=5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168
Resolving akamai.bintray.com (akamai.bintray.com)... 23.66.53.169
Connecting to akamai.bintray.com (akamai.bintray.com)|23.66.53.169|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 930614 (909K) [application/gzip]
Saving to: ‘speedtest-cli.tgz’100%[=========================================================================>] 930,614     --.-K/s   in 0.08s   

2020-04-16 17:21:41 (10.5 MB/s) - ‘speedtest-cli.tgz’ saved [930614/930614]

speedtest
speedtest.md
speedtest.5
==============================================================================

You may only use this Speedtest software and information generated
from it for personal, non-commercial use, through a command line
interface on a personal computer. Your use of this software is subject
to the End User License Agreement, Terms of Use and Privacy Policy at
these URLs:

        https://www.speedtest.net/about/eula
        https://www.speedtest.net/about/terms
        https://www.speedtest.net/about/privacy

==============================================================================

Do you accept the license? [type YES to accept]: License acceptance recorded. Continuing.


   Speedtest by Ookla

     Server: ZeptoVM - Ashburn, VA (id = 30561)
        ISP: Google Cloud
    Latency:    25.69 ms   (4.63 ms jitter)
   Download:  3977.15 Mbps (data used: 6.2 GB)                               
     Upload:   918.83 Mbps (data used: 1.5 GB)                               
Packet Loss:     0.0%
 Result URL: 
[[email protected] ~]# 




Network Traffic Analysis Script


In fact, I mainly use this script to view the port occupancy, and which IP is desperately running traffic. 
The functions included in this script are:

1. Monitor the traffic of any network card in real time
2. Count the average traffic within 10 seconds
3. Count the average traffic of each port within 10 seconds, based on the client and server port statistics. It can be seen which ports account for more traffic. For web servers, port 80 is generally used. When other ports are attacked, there may be other ports with relatively large traffic. So this function can help us to check whether the port traffic is normal.
4. Count the top 10 IPs with the largest bandwidth in 10s. This function can help us to find out if there are malicious IPs occupying bandwidth.
5. Statistics connection status. This feature allows us to see which connection status is relatively large. If there are more SYN-RECV states, there may be a semi-connection attack. If ESTABLISED is very large, but it is found that there are not so many requests through the log, or if a large number of IPs are found through tcpdump and only the connection is established without requesting data, it may be a full connection attack. Add listen 80 deferred to prevent.
6. Count the connection status of each port. When it is possible to be attacked, this function can help us discover which port was attacked.
7. The statistics port is 80 and the top 10 IPs with the largest number of ESTAB connections. This feature can help us to find out too many connections to create Ip, and then shield.
8. Count the top 10 IPs with port 80 and status SYN-RECV with the most connections. This feature can help us find malicious ips when subjected to semi-connection attacks.
Run in your linux command line:

wget https://raw.githubusercontent.com/91yun/91yuncode/master/network-analysis.sh && bash network-analysis.sh


$wget https://raw.githubusercontent.com/91yun/91yuncode/master/network-analysis.sh && bash network-analysis.sh


$ bash network-analysis.sh
1) real time traffic.
2) traffic and connection overview.

please input your select(ie 1): 2
tcpdump not found,going to install it.
network-analysis.sh: line 125: apt-get: command not found

#################### nic setting ####################

1) docker0
2) eth0
3) eth1
4) veth49c9398

which nic you'd select: 3
your selection: eth1
please wait for 10s to generate network data...


network device ens3 average traffic in 10s:
ens3 Receive: 4.9Kb/s
ens3 Transmit: 8.7Kb/s                            average traffic in 10s base on client port:
                                                  10.0.0.2:34421 > server 8.1Kb/s
average traffic in 10s base on server port:       140.204.0.165:443 > server 4.2Kb/s
clients > 140.204.0.165:443 8.1Kb/s               169.254.169.254:53 > server 396b/s
clients > 10.0.0.2:34421 4.2Kb/s                  10.0.0.2:36428 > server 150b/s
clients > 10.0.0.2:36428 396b/s                   10.0.0.2:22 > server 83b/s
clients > 169.254.169.254:53 150b/s               169.254.169.254:123 > server 60b/s
clients > 160.32.192.89:7520 83b/s                10.0.0.2:57613 > server 60b/s
clients > 169.254.169.254:123 60b/s               top 10 ip average traffic in 10s base on client:
clients > 10.0.0.2:57613 60b/s                    10.0.0.2:34421 > 140.204.0.165 8.1Kb/s
top 10 ip average traffic in 10s base on server:  140.204.0.165:443 > 10.0.0.2 4.2Kb/s
10.0.0.2 > 140.204.0.165:443 8.1Kb/s              169.254.169.254:53 > 10.0.0.2 396b/s
140.204.0.165 > 10.0.0.2:34421 4.2Kb/s            10.0.0.2:36428 > 169.254.169.254 150b/s
69.254.169.254 > 10.0.0.2:36428 396b/s           10.0.0.2:22 > 160.32.192.89 83b/s
10.0.0.2 > 169.254.169.254:53 150b/s              169.254.169.254:123 > 10.0.0.2 60b/s
10.0.0.2 > 160.32.192.89:7520 83b/s               10.0.0.2:57613 > 169.254.169.254 60b/s
169.254.169.254 > 10.0.0.2:57613 60b/s            160.32.192.89:7520 > 10.0.0.2 32b/s
10.0.0.2 > 169.254.169.254:123 60b/s
connection state count: :22 32b/s
0 102
TIME-WAIT 6
CLOSE-WAIT 6
ESTAB 1


connection state count by port base on server:    connection state count by port base on client:
0 * 102                                           TIME-WAIT 140.204.0.165:443 5
TIME-WAIT 10.0.0.2:34421 1                        CLOSE-WAIT 169.254.169.254:80 4
TIME-WAIT 10.0.0.2:34420 1                        CLOSE-WAIT 140.204.0.151:443 2
TIME-WAIT 10.0.0.2:34419 1                        TIME-WAIT 169.254.169.254:80 1
TIME-WAIT 10.0.0.2:34417 1                        ESTAB 160.32.192.89:7520 1
TIME-WAIT 10.0.0.2:34416 1                        0 23041 1
TIME-WAIT 10.0.0.2:34061 1                        0 23040 1
ESTAB 10.0.0.2:22 1                               0 22575 1
CLOSE-WAIT 10.0.0.2:47916 1                       0 22574 1
CLOSE-WAIT 10.0.0.2:47910 1                       0 22111 1

top 10 ip ESTAB state count at port 80:
* 102
160.32.192.89 1

top 10 ip SYN-RECV state count at port 80:
[[email protected] ~]#



VPS Backup Script and ftp Upload

脚本代码

  1. #!/bin/bash
  2. MYSQL_USER=root
  3. MYSQL_PASS=MySQLrootPassWord
  4. MYSQL_DB_NAME=mywordpressdb
  5. FTP_HOST=ftp.1fichier.com
  6. FTP_PORT=21
  7. FTP_USER=myftpuser
  8. FTP_PASS=myftppassword
  9. FTP_PATH=/
  10. WEB_FILES_PATH=/srv/www/example.com
  11. LOCAL_BACKUP_PATH=~/backup
  12. WEBSITE_NAME=example.com
  13.  
  14.  
  15.  
  16. DB_BACKUP_FILE_NAME=$WEBSITE_NAME.$(date +"%Y%m%d").db
  17. WEBSITE_FILES_BACKUP_FILE_NAME=$WEBSITE_NAME.$(date +"%Y%m%d").webfiles
  18. mysqldump -u $MYSQL_USER -p$MYSQL_PASS $MYSQL_DB_NAME > $DB_BACKUP_FILE_NAME.sql
  19. tar zcf $LOCAL_BACKUP_PATH/$DB_BACKUP_FILE_NAME.tar.gz *.sql
  20. rm -f $DB_BACKUP_FILE_NAME.sql
  21. tar zcf $LOCAL_BACKUP_PATH/$WEBSITE_FILES_BACKUP_FILE_NAME.tar.gz $WEB_FILES_PATH
  22. ftp -v -n $FTP_HOST $FTP_PORT<< END
  23. user $FTP_USER $FTP_PASS
  24. type binary
  25. passive
  26. cd $FTP_PATH
  27. put $LOCAL_BACKUP_PATH/$DB_BACKUP_FILE_NAME.tar.gz
  28. put $LOCAL_BACKUP_PATH/$WEBSITE_FILES_BACKUP_FILE_NAME.tar.gz
  29. bye
  30. END

预置参数

  1. MYSQL_USER=root #mysql数据库用户名
  2. MYSQL_PASS=MySQLrootPassWord #mysql数据库密码
  3. MYSQL_DB_NAME=mywordpressdb #mysql数据库名
  4. FTP_HOST=ftp.1fichier.com #远程ftp服务器地址
  5. FTP_PORT=21 #远程ftp端口
  6. FTP_USER=myftpuser #远程ftp用户名
  7. FTP_PASS=myftppassword ¥远程ftp密码
  8. FTP_PATH=/ #远程ftp备份文件存储路径
  9. WEB_FILES_PATH=/srv/www/example.com #需要备份的网站目录
  10. LOCAL_BACKUP_PATH=~/backup #本地备份文件存储地址
  11. WEBSITE_NAME=example.com #备份的网站名

定时任务

把脚本代码另存为auto_backup.sh,并执行chmod +x auto_backup.sh赋予执行权限。
执行crontab -e ,在最后追加下面这段内容
  1. #每天3:30分执行自动备份
  2. 30 3 * * * * /root/auto_backup.sh



mysql/php/nginx monitoring script


脚本内容

这个脚本必须由root来执行
  1. #!/bin/bash
  2. #mysql进程监控
  3. pgrep -x mysqld &> /dev/null
  4. if [ $? -ne 0 ];then
  5. echo At time: `date` :MySQL is stop .”>> /root/public_log
  6. /usr/sbin/service mysqld start
  7. else
  8. echo MySQL server is running .”
  9. fi
  10.  #php进程监控
  11. pgrep -x php-fpm &> /dev/null
  12. if [ $? -ne 0 ];then
  13. echo At time: `date` :php-fpm is stop .”>> /root/public_log
  14. /usr/sbin/service php-fpm start
  15. else
  16. echo php-fpm server is running .”
  17. fi
  18. #nginx进程监控
  19. pgrep -x nginx &> /dev/null
  20. if [ $? -ne 0 ];then
  21. echo At time: `date` :Nginx is stop .”>> /root/public_log
  22. /usr/sbin/service nginx start
  23. else
  24. echo Nginx server is running .”
  25. fi
添加定时监控。每隔5分钟执行一次
  1. */5 * * * * /root/public.sh


There are some dependencies need to be installed to get pgrep and crontab working.

[email protected]:/# cat monitor.sh 
#!/bin/bash
#mysqlmonitoring
pgrep -x mysqld &> /dev/null 
if [ $? -ne 0 ];then 
        echo “At time: `date` :MySQL is stopped.”>> /public_log 
        /usr/sbin/service mysql start 
else 
        echo “MySQL server is running .” >> /public_log

fi
[email protected]:/# 

apt-get update 
apt-get install procps
apt-get install nano
apt-get install cron

chmod u+x monitor.sh

crontab -e
*/5 * * * * /monitor.sh
service cron restart








No comments