Useful Linux Network Analysis/Monitoring/Backup Shell Scripts Collection - NETSEC

Latest

Learning, Sharing, Creating

Cybersecurity Memo

Thursday, April 16, 2020

Useful Linux Network Analysis/Monitoring/Backup Shell Scripts Collection

Some Scripts to help linux operation:
  • Speedtest script
  • Network Traffic Analysis Script
  • VPS Backup Script and ftp Upload
  • mysql/php/nginx monitoring script
  • BlueSkyXN 综合工具箱


Speedtest Script





wget https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz -O speedtest-cli.tgz && tar xfvz speedtest-cli.tgz && echo yes | ./speedtest


[root@centos7-zabbix-grafana-1 ~]# wget https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz -O speedtest-cli.tgz && tar xfvz speedtest-cli.tgz && echo yes | ./speedtest
--2020-04-16 17:21:41--  https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz
Resolving bintray.com (bintray.com)... 108.168.194.93
Connecting to bintray.com (bintray.com)|108.168.194.93|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://dl.bintray.com/ookla/download/ookla-speedtest-1.0.0-x86_64-linux.tgz?expiry=1587057731446&signature=N%2F%2FEyyWnLJRqFhHwYJ08IM0%2B0OU66hX1%2BgGCWG43CaY3dmuJOyA0M8gy36G2RwtgfT8Elro6jQpIhBd8yTOKNQ%3D%3D [following]
--2020-04-16 17:21:41--  https://dl.bintray.com/ookla/download/ookla-speedtest-1.0.0-x86_64-linux.tgz?expiry=1587057731446&signature=N%2F%2FEyyWnLJRqFhHwYJ08IM0%2B0OU66hX1%2BgGCWG43CaY3dmuJOyA0M8gy36G2RwtgfT8Elro6jQpIhBd8yTOKNQ%3D%3D
Resolving dl.bintray.com (dl.bintray.com)... 52.26.64.218, 52.11.170.179
Connecting to dl.bintray.com (dl.bintray.com)|52.26.64.218|:443... connected.
HTTP request sent, awaiting response... 302 
Location: https://akamai.bintray.com/5f/5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168?__gda__=exp=1587058421~hmac=bcc7e0e4e8f71f5d0af7ebf6178ae0534027fb63a80234c4870051da23c2fbfa&response-content-disposition=attachment%3Bfilename%3D%22ookla-speedtest-1.0.0-x86_64-linux.tgz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX19FmhEAfVfGnWNhHLMH9_FIedcu869F-5_L6eYlhAQ-vBUL-KjMmlOg3_Pt0gfPKOS-M8PpIXM7iVCKOdekGMaDStQwm92EfjfQDX_lGbiCXiYR9ao_wwmHjKOiB6RTgnyrDECxGx8spA&response-X-Checksum-Sha1=41ca19b8bea7614c27370453be3c6ef7ea7fa76a&response-X-Checksum-Sha2=5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168 [following]
--2020-04-16 17:21:41--  https://akamai.bintray.com/5f/5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168?__gda__=exp=1587058421~hmac=bcc7e0e4e8f71f5d0af7ebf6178ae0534027fb63a80234c4870051da23c2fbfa&response-content-disposition=attachment%3Bfilename%3D%22ookla-speedtest-1.0.0-x86_64-linux.tgz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX19FmhEAfVfGnWNhHLMH9_FIedcu869F-5_L6eYlhAQ-vBUL-KjMmlOg3_Pt0gfPKOS-M8PpIXM7iVCKOdekGMaDStQwm92EfjfQDX_lGbiCXiYR9ao_wwmHjKOiB6RTgnyrDECxGx8spA&response-X-Checksum-Sha1=41ca19b8bea7614c27370453be3c6ef7ea7fa76a&response-X-Checksum-Sha2=5fe2028f0d4427e4f4231d9f9cf70e6691bb890a70636d75232fe4d970633168
Resolving akamai.bintray.com (akamai.bintray.com)... 23.66.53.169
Connecting to akamai.bintray.com (akamai.bintray.com)|23.66.53.169|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 930614 (909K) [application/gzip]
Saving to: ‘speedtest-cli.tgz’100%[=========================================================================>] 930,614     --.-K/s   in 0.08s   

2020-04-16 17:21:41 (10.5 MB/s) - ‘speedtest-cli.tgz’ saved [930614/930614]

speedtest
speedtest.md
speedtest.5
==============================================================================

You may only use this Speedtest software and information generated
from it for personal, non-commercial use, through a command line
interface on a personal computer. Your use of this software is subject
to the End User License Agreement, Terms of Use and Privacy Policy at
these URLs:

        https://www.speedtest.net/about/eula
        https://www.speedtest.net/about/terms
        https://www.speedtest.net/about/privacy

==============================================================================

Do you accept the license? [type YES to accept]: License acceptance recorded. Continuing.


   Speedtest by Ookla

     Server: ZeptoVM - Ashburn, VA (id = 30561)
        ISP: Google Cloud
    Latency:    25.69 ms   (4.63 ms jitter)
   Download:  3977.15 Mbps (data used: 6.2 GB)                               
     Upload:   918.83 Mbps (data used: 1.5 GB)                               
Packet Loss:     0.0%
 Result URL: 
[root@centos7-zabbix-grafana-1 ~]# 




Network Traffic Analysis Script


In fact, I mainly use this script to view the port occupancy, and which IP is desperately running traffic. 
The functions included in this script are:

1. Monitor the traffic of any network card in real time
2. Count the average traffic within 10 seconds
3. Count the average traffic of each port within 10 seconds, based on the client and server port statistics. It can be seen which ports account for more traffic. For web servers, port 80 is generally used. When other ports are attacked, there may be other ports with relatively large traffic. So this function can help us to check whether the port traffic is normal.
4. Count the top 10 IPs with the largest bandwidth in 10s. This function can help us to find out if there are malicious IPs occupying bandwidth.
5. Statistics connection status. This feature allows us to see which connection status is relatively large. If there are more SYN-RECV states, there may be a semi-connection attack. If ESTABLISED is very large, but it is found that there are not so many requests through the log, or if a large number of IPs are found through tcpdump and only the connection is established without requesting data, it may be a full connection attack. Add listen 80 deferred to prevent.
6. Count the connection status of each port. When it is possible to be attacked, this function can help us discover which port was attacked.
7. The statistics port is 80 and the top 10 IPs with the largest number of ESTAB connections. This feature can help us to find out too many connections to create Ip, and then shield.
8. Count the top 10 IPs with port 80 and status SYN-RECV with the most connections. This feature can help us find malicious ips when subjected to semi-connection attacks.
Run in your linux command line:
wget https://raw.githubusercontent.com/91yun/91yuncode/master/network-analysis.sh && bash network-analysis.sh


$wget https://raw.githubusercontent.com/91yun/91yuncode/master/network-analysis.sh && bash network-analysis.sh


$ bash network-analysis.sh
1) real time traffic.
2) traffic and connection overview.

please input your select(ie 1): 2
tcpdump not found,going to install it.
network-analysis.sh: line 125: apt-get: command not found

#################### nic setting ####################

1) docker0
2) eth0
3) eth1
4) veth49c9398

which nic you'd select: 3
your selection: eth1
please wait for 10s to generate network data...


network device ens3 average traffic in 10s:
ens3 Receive: 4.9Kb/s
ens3 Transmit: 8.7Kb/s                            average traffic in 10s base on client port:
                                                  10.0.0.2:34421 > server 8.1Kb/s
average traffic in 10s base on server port:       140.204.0.165:443 > server 4.2Kb/s
clients > 140.204.0.165:443 8.1Kb/s               169.254.169.254:53 > server 396b/s
clients > 10.0.0.2:34421 4.2Kb/s                  10.0.0.2:36428 > server 150b/s
clients > 10.0.0.2:36428 396b/s                   10.0.0.2:22 > server 83b/s
clients > 169.254.169.254:53 150b/s               169.254.169.254:123 > server 60b/s
clients > 160.32.192.89:7520 83b/s                10.0.0.2:57613 > server 60b/s
clients > 169.254.169.254:123 60b/s               top 10 ip average traffic in 10s base on client:
clients > 10.0.0.2:57613 60b/s                    10.0.0.2:34421 > 140.204.0.165 8.1Kb/s
top 10 ip average traffic in 10s base on server:  140.204.0.165:443 > 10.0.0.2 4.2Kb/s
10.0.0.2 > 140.204.0.165:443 8.1Kb/s              169.254.169.254:53 > 10.0.0.2 396b/s
140.204.0.165 > 10.0.0.2:34421 4.2Kb/s            10.0.0.2:36428 > 169.254.169.254 150b/s
69.254.169.254 > 10.0.0.2:36428 396b/s           10.0.0.2:22 > 160.32.192.89 83b/s
10.0.0.2 > 169.254.169.254:53 150b/s              169.254.169.254:123 > 10.0.0.2 60b/s
10.0.0.2 > 160.32.192.89:7520 83b/s               10.0.0.2:57613 > 169.254.169.254 60b/s
169.254.169.254 > 10.0.0.2:57613 60b/s            160.32.192.89:7520 > 10.0.0.2 32b/s
10.0.0.2 > 169.254.169.254:123 60b/s
connection state count: :22 32b/s
0 102
TIME-WAIT 6
CLOSE-WAIT 6
ESTAB 1


connection state count by port base on server:    connection state count by port base on client:
0 * 102                                           TIME-WAIT 140.204.0.165:443 5
TIME-WAIT 10.0.0.2:34421 1                        CLOSE-WAIT 169.254.169.254:80 4
TIME-WAIT 10.0.0.2:34420 1                        CLOSE-WAIT 140.204.0.151:443 2
TIME-WAIT 10.0.0.2:34419 1                        TIME-WAIT 169.254.169.254:80 1
TIME-WAIT 10.0.0.2:34417 1                        ESTAB 160.32.192.89:7520 1
TIME-WAIT 10.0.0.2:34416 1                        0 23041 1
TIME-WAIT 10.0.0.2:34061 1                        0 23040 1
ESTAB 10.0.0.2:22 1                               0 22575 1
CLOSE-WAIT 10.0.0.2:47916 1                       0 22574 1
CLOSE-WAIT 10.0.0.2:47910 1                       0 22111 1

top 10 ip ESTAB state count at port 80:
* 102
160.32.192.89 1

top 10 ip SYN-RECV state count at port 80:
[root@centos7-test1 ~]#



VPS Backup Script and ftp Upload

脚本代码

  1. #!/bin/bash
  2. MYSQL_USER=root
  3. MYSQL_PASS=MySQLrootPassWord
  4. MYSQL_DB_NAME=mywordpressdb
  5. FTP_HOST=ftp.1fichier.com
  6. FTP_PORT=21
  7. FTP_USER=myftpuser
  8. FTP_PASS=myftppassword
  9. FTP_PATH=/
  10. WEB_FILES_PATH=/srv/www/example.com
  11. LOCAL_BACKUP_PATH=~/backup
  12. WEBSITE_NAME=example.com
  13.  
  14.  
  15.  
  16. DB_BACKUP_FILE_NAME=$WEBSITE_NAME.$(date +"%Y%m%d").db
  17. WEBSITE_FILES_BACKUP_FILE_NAME=$WEBSITE_NAME.$(date +"%Y%m%d").webfiles
  18. mysqldump -u $MYSQL_USER -p$MYSQL_PASS $MYSQL_DB_NAME > $DB_BACKUP_FILE_NAME.sql
  19. tar zcf $LOCAL_BACKUP_PATH/$DB_BACKUP_FILE_NAME.tar.gz *.sql
  20. rm -f $DB_BACKUP_FILE_NAME.sql
  21. tar zcf $LOCAL_BACKUP_PATH/$WEBSITE_FILES_BACKUP_FILE_NAME.tar.gz $WEB_FILES_PATH
  22. ftp -v -n $FTP_HOST $FTP_PORT<< END
  23. user $FTP_USER $FTP_PASS
  24. type binary
  25. passive
  26. cd $FTP_PATH
  27. put $LOCAL_BACKUP_PATH/$DB_BACKUP_FILE_NAME.tar.gz
  28. put $LOCAL_BACKUP_PATH/$WEBSITE_FILES_BACKUP_FILE_NAME.tar.gz
  29. bye
  30. END

预置参数

  1. MYSQL_USER=root #mysql数据库用户名
  2. MYSQL_PASS=MySQLrootPassWord #mysql数据库密码
  3. MYSQL_DB_NAME=mywordpressdb #mysql数据库名
  4. FTP_HOST=ftp.1fichier.com #远程ftp服务器地址
  5. FTP_PORT=21 #远程ftp端口
  6. FTP_USER=myftpuser #远程ftp用户名
  7. FTP_PASS=myftppassword ¥远程ftp密码
  8. FTP_PATH=/ #远程ftp备份文件存储路径
  9. WEB_FILES_PATH=/srv/www/example.com #需要备份的网站目录
  10. LOCAL_BACKUP_PATH=~/backup #本地备份文件存储地址
  11. WEBSITE_NAME=example.com #备份的网站名

定时任务

把脚本代码另存为auto_backup.sh,并执行chmod +x auto_backup.sh赋予执行权限。
执行crontab -e ,在最后追加下面这段内容
  1. #每天3:30分执行自动备份
  2. 30 3 * * * * /root/auto_backup.sh



mysql/php/nginx monitoring script


脚本内容

这个脚本必须由root来执行
  1. #!/bin/bash
  2. #mysql进程监控
  3. pgrep -x mysqld &> /dev/null
  4. if [ $? -ne 0 ];then
  5. echo At time: `date` :MySQL is stop .”>> /root/public_log
  6. /usr/sbin/service mysqld start
  7. else
  8. echo MySQL server is running .”
  9. fi
  10.  #php进程监控
  11. pgrep -x php-fpm &> /dev/null
  12. if [ $? -ne 0 ];then
  13. echo At time: `date` :php-fpm is stop .”>> /root/public_log
  14. /usr/sbin/service php-fpm start
  15. else
  16. echo php-fpm server is running .”
  17. fi
  18. #nginx进程监控
  19. pgrep -x nginx &> /dev/null
  20. if [ $? -ne 0 ];then
  21. echo At time: `date` :Nginx is stop .”>> /root/public_log
  22. /usr/sbin/service nginx start
  23. else
  24. echo Nginx server is running .”
  25. fi
添加定时监控。每隔5分钟执行一次
  1. */5 * * * * /root/public.sh


There are some dependencies need to be installed to get pgrep and crontab working.

root@4743ba7cdb0e:/# cat monitor.sh 
#!/bin/bash
#mysqlmonitoring
pgrep -x mysqld &> /dev/null 
if [ $? -ne 0 ];then 
        echo “At time: `date` :MySQL is stopped.”>> /public_log 
        /usr/sbin/service mysql start 
else 
        echo “MySQL server is running .” >> /public_log

fi
root@4743ba7cdb0e:/# 

apt-get update 
apt-get install procps
apt-get install nano
apt-get install cron

chmod u+x monitor.sh

crontab -e
*/5 * * * * /monitor.sh
service cron restart


BlueSkyXN 综合工具箱

Github : https://github.com/BlueSkyXN/SKY-BOX

使用方法

wget -O box.sh https://raw.githubusercontent.com/BlueSkyXN/SKY-BOX/main/box.sh && chmod +x box.sh && clear && ./box.sh

wget -O box.sh https://raw.githubusercontent.com/BlueSkyXN/SKY-BOX/main/box.sh && chmod +x box.sh && clear && ./box.sh

ARM beta使用方法

ARM beta使用方法wget -O box.sh https://raw.githubusercontent.com/BlueSkyXN/SKY-BOX/main/armbox.sh && chmod +x box.sh && clear && ./box.sh
wget -O box.sh https://raw.githubusercontent.com/BlueSkyXN/SKY-BOX/main/armbox.sh && chmod +x box.sh && clear && ./box.sh

[root@ocp3arm1oracle ~]# ./box.sh
 BlueSkyXN 综合工具箱 Linux Supported ONLY
 FROM: https://github.com/BlueSkyXN/SKY-BOX
 HELP: https://www.blueskyxn.com/202104/4465.html
 USE:  wget -O box.sh https://raw.githubusercontent.com/BlueSkyXN/SKY-BOX/main/box.sh && chmod +x box.sh && clear && ./box.sh
 ==================================================
 1. IPV.SH ipv4/6优先级调整一键脚本·下载
 2. IPT.SH iptable一键脚本
 3. SpeedTest-Linux 下载
 4. Rclone&Fclone·下载
 5. ChangeSource Linux换源脚本·下载
 6. Besttrace 路由追踪·下载
 7. NEZHA.SH哪吒面板/探针
 --------------------------------------------------
 11. 获取本机IP
 12. 安装最新BBR内核·使用YUM·仅支持CentOS
 13. 启动BBR FQ算法
 14. 系统网络配置优化
 15. Git 新版 安装·仅支持CentOS
 16. 宝塔面板 自动磁盘挂载工具
 17. BBR一键管理脚本
 18. SWAP一键安装/卸载脚本
 19. F2B一键安装脚本
 --------------------------------------------------
 21. Superbench 综合测试
 22. MT.SH 流媒体解锁测试
 23. Lemonbench 综合测试
 24. UNIXbench 综合测试
 25. 三网Speedtest测速
 26. Memorytest 内存压力测试
 27. Route-trace 路由追踪测试
 28. YABS LINUX综合测试
 29. Disk Test 硬盘&系统综合测试
 210.TubeCheck Google/Youtube CDN分配节点测试
 211.RegionRestrictionCheck 流媒体解锁测试
 --------------------------------------------------
 31. MTP&TLS 一键脚本
 32. Rclone官方一键安装脚本
 33. Aria2 最强安装与管理脚本
 --------------------------------------------------
 00. 宝塔面板综合安装脚本
 ==================================================
 0. 退出脚本

请输入数字:






No comments:

Post a Comment