Showing posts with label Checkpoint. Show all posts
Showing posts with label Checkpoint. Show all posts

Wednesday, August 26, 2015

Checkpoint Monitord Process Consumes Excess Memory

During a regular review firewall mem and cpu usage, I found some of Checkpoint UTM272 R77.10 gateways are using lots memory and ssh / snmp access seems slow sometimes. With the TOP command , I am able to sort the mem / cpu usage and see who is hogging the resources.

The result of finding is monitord service. Monitord server is used by device sensors to monitor hardware and saves data into DB file stored on local. Before R76, it will keep one year data in DB. After R76, it only keeps 3 months history to save devices resources during process the data. In my case, the DB file is more than 350M which cause monitord service consumes lots memory to process DB file. Although we are using R77.10, it seems upgrading to R771.10, not fresh installation,  wont reset your DB file structure.

There is workaround provided at SK93587. Here are all steps I recorded to fix this.


1. Before applied the workaround, monitord is using 42.5% MEM.


top - 10:56:37 up 10 days,  1:08,  1 user,  load average: 0.00, 0.06, 0.43
Tasks:  83 total,   3 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.2%us,  1.1%sy,  0.0%ni, 97.3%id,  0.2%wa,  0.1%hi,  0.1%si,  0.0%st
Mem:    957272k total,   947392k used,     9880k free,     2772k buffers
Swap:  2096472k total,    43292k used,  2053180k free,   209280k cached
%MEM   PID USER      PR  NI  VIRT  RES  SHR S %CPU    TIME+  COMMAND             
 5.0  4226 admin     15   0  263m  47m  11m S  0.4  59:12.98 cpd                 
 0.1  2782 admin     15   0  2172 1084  836 R  0.2   0:00.05 top                 
 0.8  3988 admin     15   0 24344 7956 5780 S  0.2  22:38.83 snmpd               
 1.4  3947 admin     16   0 33796  13m 7964 S  0.1   2947:10 confd               
42.5  3952 admin     15   0  400m 397m 2332 S  0.1 119:05.53 monitord            
 0.1  3545 admin     18   0  1708  688  584 S  0.1   2:38.13 syslogd             
 0.1     1 admin     15   0  2040  580  548 S  0.0   0:01.47 init                
 0.0     2 admin     RT  -5     0    0    0 S  0.0   0:00.00 migration/0         
 0.0     3 admin     15   0     0    0    0 S  0.0   0:00.67 ksoftirqd/0         
 0.0     4 admin     RT  -5     0    0    0 S  0.0   0:00.00 watchdog/0          
 0.0     5 admin     10  -5     0    0    0 S  0.0   0:01.56 events/0                                                                                             

Sunday, August 16, 2015

Checkpoint Gateway Lost SIC After Jumbo Hotfix Installed

Our Checkpoint Products are stilling sitting at R77.10. Checkpoint has release Jumbo Hotfix Accumulator for R77.10 (gypsy_hf_base_021).

The installation procedures from Command Line is quite simple:
  1. Transfer the Jumbo Hotfix Accumulator to the machine /var/tmp folder
  2. Unpack the Jumbo Hotfix Accumulator:

    [[email protected]]# cd /var/tmp
    [[email protected]]# tar zxvf Check_Point_R77.10.linux.tgz
  3. Install the Jumbo Hotfix Accumulator:
    [[email protected]]# ./UnixInstallScript

    Note: The script will stop all of Check Point services (cpstop) - read the output on the screen.
  4. Reboot the machine.
  5. Verify Installation with Command "cpinfo -y all"

Symptoms: 


I followed those steps and installed this Jumbo Hotfix on both cluster members at the same time also rebooted them at the same time. But after waited a couple of minutes, one of cluster members shows disconnected from Smartview Monitor.


When I ssh-ed into device and checked cluster status it shows ok. Also I were able to reach management server interface from problem cluster member. From the output of "cpinfo -y all " also shows the hotfix has been installed correctly. 

[[email protected]:0]# cpinfo -y all
------------------------
Hotfix versions
------------------------
[FW1] 
  HOTFIX_R77_10 
  HOTFIX_R77_HF_HA10_005 
  HOTFIX_GYPSY_HF_BASE_021 

[SecurePlatform] 
  HOTFIX_R77_10_GAIA_GHOST_833 
  HOTFIX_GYPSY_HF_BASE_021 

[SPSHARED] 
  No hotfixes..

[CVPN] 
  HOTFIX_R77_10 
  HOTFIX_GYPSY_HF_BASE_021 

[PPACK] 
  HOTFIX_R77_10 
  HOTFIX_GYPSY_HF_BASE_021 

[CPinfo] 
  No hotfixes..

[SmartLog] 
  HOTFIX_R77_10 

[rtm] 
  No hotfixes..

Troubleshooting:


I went back to SmartDashboard and checked SIC status and found it was out of SIC. I was confusing what could cause the SIC lost from this cluster member. Should I reset SIC?

SmartView Tracker saved me this time. There is one log shows firewall policy inconsistencies existing between cluster members.


Number:             7250420
Date:                 16Aug2015
Time:                 10:09:07
Origin:               CP-DMZ-1
Type:                 Log
Action:              
Information:       sync: Inconsistencies exist between policies installed on the cluster members. Please reinstall the policy on the cluster.
Product:             Security Gateway/Management
Product Family: Network
Policy Info:         Policy Name: defaultfilter
                          Created at: Sun Aug 16 07:12:25 2015
                          Installed from: CP-Management

Solutions:

I quickly pushed policy to cluster and it was failed because SIC error as shown below.


Amazing thing is this firewall policy push resolved SIC issue. Both firewall cluster members show green and OK status in Smartview Monitor. 

Thursday, July 16, 2015

Checkpoint NAT Concepts and Server Side NAT Explanation

Checkpoint have four main NAT concepts we will need to know their difference when implement a NAT rule:
a. Automatically NAT vs Manual NAT
b. Static NAT vs Dynamic NAT (Hide)
c. Source NAT vs Destination NAT
d. Client Side Destination NAT vs Server Side Destination NAT

Checkpoint Firewall NAT is quite different than any other firewall vendors, especially on destination NAT. For first three NAT concepts, Automatically NAT / Manual NAT, Static NAT / Dynamic NAT, Source NAT / Destination NAT, are easy to understand, and almost all vendors are using same way to handle packets. But on the part regarding source NAT and destination NAT, for all other vendors, they are acting same, which is doing destination NAT first at inbound traffic to firewalls, then do source NAT at outbound traffic before packets leave firewalls. Checkpoint is having two different place to handle destination NAT, which is client side (inbound side) or server side (outbound side).



Saturday, July 11, 2015

Checkpoint Standby Cluster Member Interface Not Reachable

It was a curious test that I tried to ping other interfaces on Checkpoint 4200 Cluster's active and passive firewalls. The result was interesting, I were able to ping both active (10.9.30.43) and standby (10.9.30.44) interfaces which are at the same zone as test PC (10.9.30.14), but not all of other interfaces on both cluster members. Only those active cluster member interfaces (such as 172.17.30.43) are reachable. Standby cluster member's interface (172.17.30.44) is unreachable at all. Not only icmp traffic, but also all other traffic such as https, ssh, sync traffic does not work on standby member's interface.

I did my search and found from Checkpoint Support Site, Checkpoint's explanation is "this is expected behavior. Connections to the Standby cluster members are not supported in HA clusters, by default."

To find out more details about this firewall behavior, I did some basic troubleshooting to see packets flow.

1. While I am pinging from  pc 10.9.30.14 to standby member 172.17.30.44, I got echo timed out. But 172.17.30.43 replied back

2. Check the drop packets from Active member 172.17.30.43, it seems the packets dropped by active firewall.It did not pass the traffic to standby member.

[[email protected]:0]# fw ctl zdebug drop | grep 10.9.30.14
;[cpu_0];[fw4_1];fw_log_drop_ex: Packet proto=1 10.9.30.14:2048 -> 172.17.30.43:19538 dropped by fwchain_reject_mtu Reason: rejected;
;[cpu_0];[fw4_1];fw_log_drop_ex: Packet proto=1 10.9.30.14:2048 -> 172.17.30.43:19537 dropped by fwchain_reject_mtu Reason: rejected;
;[cpu_0];[fw4_1];fw_log_drop_ex: Packet proto=1 10.99.30.14:2048 -> 172.17.30.43:19536 dropped by fwchain_reject_mtu Reason: rejected;

Note: during the research, also SK97587 mentioned "in some cases when the traffic originates from the standby member, return traffic is forwarded from the VIP to the active member, which drops that traffic."

My old post "Check Point Cluster Member Gateway Drops Ping Packets Without Log in Smartview Tracker" has a similar symptoms as this case, but cause is different. The solution is enable simultaneous ping parameter in the kernel by this command:  fw ctl set int fw_allow_simultaneous_ping

Resolution:

The solution is pretty simple, there is a magic parameter in firewall kernel for this kind of situation:

[[email protected]:0]# fw ctl get int fwha_forw_packet_to_not_active
fwha_forw_packet_to_not_active = 0

Basically, when this parameter is set to "0", packet forwarding will NOT be done to a non-active member. Instead, a reset packet will be sent to the client.

Set following command on both Cluster Members:

# fw ctl set int fwha_forw_packet_to_not_active 1

With following command you can verify the setting:
# fw ctl get int fwha_forw_packet_to_not_active


To set it permanently to survive reboot, add this line to the file $FWDIR/boot/modules/fwkern.conf :
fwha_forw_packet_to_not_active=1

Then reboot. Perform this on both cluster members.



Reference:

a. Troubleshooting "Clear text packet should be encrypted" error in ClusterXL




Wednesday, June 3, 2015

2015 Check Point’s Annual Security Report




Check Point's released annual security report recently to disclose 2014 security trends and issues that are on the rise or in decline by analyzed their collected event data from three different sources around the world.

Here are some highlights I would like to show in this post:

The Evolution of Malware:

Highlights

  • New malware increased 71% in 2014 cpmares to 2013
  • 106 unknown malware hit an organization per hour
  • 96% of organizations used at least one high-risk application.
  • 86% of organizations accessed a malicious site.
  • 83% of organizations had existing bot infections.
  • 81% of organizations suffered a data loss incident.
  • 71% increased  in loss of proprietary information over the past three years.
  • 52% of files infected with unknown malware are PDFs
  • 42% of businesses suffered mobile security incidents costing more than $250,000 to remediate.
  • 41% of organizations downloaded at least one infected file with unknown malware

Enterprise Endpoint Vulnerabilities and Misconfigurations:

What Happened in an average day at an enterprise organization: 


  • a host accesses a malicious website per 24s
  • an unknown malware is downloaded per 34s
  • a bot communicates with its command and control center per 1m
  • a high-risk application is used per 5m
  • a known malware is downloaded per 6m
  • sensitive data are sent outside of the organization per 36m




Tuesday, May 5, 2015

Using Command Line to Do First Time Wizard on Checkpoint Appliance without WebUI

Three years ago, I got a problem to do first time configuration wizard at SPLAT platform using WebUI remotely because CheckPoint by default set mgmt interface ip to 192.168.1.1. I have no way to change this mgmt interface ip address  and has to use a special trick to touch some files to bypass the first time wizard requirement before I can type some other CheckPoint commands such as Sysconfig or CPCONFIG.

Basicall you will have to touch wizard_accepted file from expert mode based on Checkpoint KB 71000 First Time Configuration Wizard on Check Point appliances
  • SecurePlatform OS:
    touch /opt/spwm/conf/wizard_accepted
  • Gaia OS:
    touch /etc/.wizard_accepted
This year, I had a Gaia R76 4200 appliance installed at remote site. Console access is ready, and mgmt interface is connected to network. Unfortunately, it is pre-configed to 192.168.1.1 as well. I managed to configed it using my way to bring it up.

1. Steps to run first time wizard at Gaia using command line

1.1 Confirm CPCONFIG is not availabe without run First Time Wizard first. 

gw-379eb9> cpconfig
In order to configure your system, please access the Web UI and finish the First Time Wizard.

1.2 Set Up Expert Password

gw-379eb9> set expert-password 
gw-379eb9> expert
Enter expert password:

1.3 Touch the magic file

[[email protected]:0]# touch /etc/.wizard_accepted
[[email protected]:0]# exit
exit

1.4 Change Mgmt Interface IP Address for Remote WebUI Access

gw-379eb9> set interface Mgmt ipv4-address 10.9.2.15 mask-length 24
gw-379eb9> set static-route default nexthop gateway address 10.9.2.1 on
gw-379eb9> set static-route default nexthop gateway address 192.168.1.254 off
gw-379eb9> save config

1.5 CPCONFIG for CheckPoint Product Configuration

gw-379eb9> cpconfig


Welcome to Check Point Configuration Program
=================================================
Please read the following license agreement.
Hit 'ENTER' to continue...



Software License Agreement & Limited Hardware Warranty
Check Point Software Technologies Ltd.

PART I - SOFTWARE LICENSE AGREEMENT

This License Agreement (the "Agreement") is an agreement between you (both the i
ndividual installing the Product and any legal entity on whose behalf such indiv
idual is acting) (hereinafter "You" or "Your") and Check Point Software Technolo
gies Ltd. (hereinafter "Check Point").

TAKING ANY STEP TO SET-UP, USE OR INSTALL THE PRODUCT CONSTITUTES YOUR ASSENT TO
 AND ACCEPTANCE OF THIS AGREEMENT. WRITTEN APPROVAL IS NOT A PREREQUISITE TO THE
 VALIDITY OR ENFORCEABILITY OF THIS AGREEMENT AND NO SOLICITATION OF ANY SUCH WR
ITTEN APPROVAL BY OR ON BEHALF OF YOU SHALL BE CONSTRUED AS AN INFERENCE TO THE
.......


Select installation type:
-------------------------

(1) Stand Alone - install Check Point Security Gateway and Security Management.
(2) Distributed - install Check Point Security Gateway, Security Management and/or Log Server.

Enter your selection  (1-2/a-abort) [1]: 2


Select installation type:
-------------------------

(1) Check Point Security Gateway.
(2) Security Management.
(3) Security Management and Check Point Security Gateway.
(4) Enterprise Log Server.
(5) Check Point Security Gateway and Enterprise Log Server.

Enter your selection  (1-5/a-abort) [1]: 1
Is this a Dynamically Assigned IP Address gateway installation ? (y/n) [n] ? n
Would you like to install a Check Point clustering product (CPHA, CPLS or State Synchronization)? (y/n) [n] ? y
IP forwarding disabled
Hardening OS Security: IP forwarding will be disabled during boot.
Generating default filter
Default Filter installed
Hardening OS Security: Default Filter will be applied during boot.
This program will guide you through several steps where you
will define your Check Point products configuration.
At any later time, you can reconfigure these parameters by
running cpconfig



Configuring Licenses and contracts...
=====================================
Host             Expiration  Signature                             Features          

Contract Coverage:

There is no contract coverage for the above licenses.
Note: The recommended way of managing licenses is using SmartUpdate.
cpconfig can be used to manage local licenses only on this machine.

Do you want to add licenses (y/n) [y] ? n


Configuring Administrator...
============================
No Check Point products Administrator is currently
defined for this Security Management Server.

Do you want to add an administrator (y/n) [y] ? n


No administrator is currently defined.
Are you sure you want to continue? (y/n) [n] ? n

Do you want to add an administrator (y/n) [y] ? y
Administrator name: admin
Password:
Verify Password:

Administrator admin was added successfully and has
Read/Write Permission for all products with Permission to Manage Administrators


Configuring GUI Clients...
==========================
GUI Clients are trusted hosts from which
Administrators are allowed to log on to this Security Management Server.

No GUI Clients defined
Do you want to add a GUI Client (y/n) [y] ? n


Configuring Random Pool...
==========================
Automatically collecting random data to be used in
various cryptographic operations.
.....


After all basic configuration completed, the appliance will be rebooted then you will be able to access it through WebUI, SSH or connect it to Smart Dashboard.

2. Checkpoint KB 69701 :

Run First Time Wizard at Command line using config_system command:

Checkpoint offers another command config_system to do First Time Wizard using a template file. It seems more complicated way to do compare CPCONFIG's wizard way. 

           This will create an empty template file for system configuration.

  • Open the file you created with a text editor and fill the appropriate fields.

This will run the First Time Configuration Wizard with the information provided in the filename.



Reference:

1. First Time Configuration Wizard on Check Point appliances
2. How to run the First Time Configuration Wizard through CLI in Gaia

Sunday, March 15, 2015

How to uninstall a CheckPoint Hotfix after a failed installation

There are always bad days during your life. The only thing we could do is to face it and find the solution. Just like today, it supposed to be a good weekend after a quick patch. But things quickly turned to bad way. There was a Checkpoint firewall not coming back after a hotfix installed. System crashed and kept rebooting during loading the policy from local host. (Root cause for this issue is another story.)

Since system crashed and no way for administrator to log in, what we could do is to log in to maintenance mode , either restore from previous backup / image (hopefully you have one, usually I will have a snapsot monthly and remote backup weekly), or uninstall the hotfix.

Usually uninstallation script will save your huge amounts of time from this awkward situation, the worst case is to get into maintenance mode to restore image you took before. Let me list all steps I experienced today:

1. System crushed during rebooting after applied a hotfix from Check Point

INIT: Entering runlevel: 3
Applying Intel CPU microcode update: [  OK  ]
Starting sysstat:  Calling the system activity data collector (sadc):
[  OK  ]
Running UP accel driver check.
IP series driver not present
Starting background readahead: [  OK  ]
Checking for hardware changes [  OK  ]
Configuring ipv6 kernel support:  [  OK  ]
Starting kdump:[  OK  ]
Inserting ipsctlmod.2.6.18.cp.i686: [  OK  ]
CKP: Loading SecureXL:  [  OK  ]
CKP: Loading FW-1 IPv4 Instance 0:  [  OK  ]
CKP: Loading VPN-1     Instance 0:  [  OK  ]
CKP: Loading FW-1 IPv4 Instance 1:  [  OK  ]
CKP: Loading VPN-1     Instance 1:  [  OK  ]
FW1: Starting cpWatchDog
Starting wrp: 
[  OK  ]
Starting auditd: [  OK  ]
Starting system logger: [  OK  ]
Starting kernel logger: [  OK  ]
Fulcrum switch not installed
Update Interfaces in Database:  0 bindings were imported
[  OK  ]
Generating vrfs:  [  OK  ]
Configuring NetAccess:  [  OK  ]
Generating NTP configuration:  [  OK  ]
Generating Time Zone configuration:  [  OK  ]
Generating domain name configuration:  [  OK  ]
Generating keyboard mapping configuration:  [  OK  ]
Generating hostname configuration:  [  OK  ]
Configuring Interfaces:  [  OK  ]
Generating /etc/monitor_mode:  [  OK  ]
Generating /etc/fonic_pairs:  [  OK  ]
Configuring NDP:  [  OK  ]
Generating hosts.conf:  [  OK  ]
Generating resolv.conf:  [  OK  ]
Generating dhclient.conf:  [  OK  ]
Generating pwcontrol.conf [  OK  ]
Generating passwd + shadow [  OK  ]
Generating group + gshadow [  OK  ]
Generating routed.conf [  OK  ]
Generating routed0.conf [  OK  ]
Generating extended commands:  [  OK  ]
Generating MOTD:  [  OK  ]
Generating banner message:  [  OK  ]
Generating /etc/raddb/server:  [  OK  ]
Generating TACACS+ configuration:  [  OK  ]
Generating /etc/msmtp.conf:  [  OK  ]
Generating /etc/pam.d/system-auth:  [  OK  ]
Generating /etc/sysconfig/external.if:  [  OK  ]
Generating /etc/lldpd.conf:  [  OK  ]
Generating DHCP server configuration:  Write DSTATE called
ServerConfigured = 1
DdnsConfigured = 0
[  OK  ]
Generating /etc/adjust_radius:  [  OK  ]
Running /bin/arp_xlate:  [  OK  ]
Generating SNMP configuration:  [  OK  ]
Generating Job Scheduler configuration:  [  OK  ]
Updating general configuraion file:  [  OK  ]
Updating syslogd configuration:  Reloading syslogd...[  OK  ]
Reloading klogd...[  OK  ]
[  OK  ]
Updating httpd2 configuration:  [  OK  ]
 Updating httpd-ssl configuration:  [  OK  ]
Applying NetFlow configuration [  OK  ]
Configuring PPPoE:  [  OK  ]
CPshell initialization:  [  OK  ]
Initializing CP Process Manager..
Starting cp_pm_rl2:  [  OK  ]
Starting cp_pm_rl3:  [  OK  ]
Starting cp_pm_rl4:  [  OK  ]
Starting acpi daemon: [  OK  ]
Starting sshd: [  OK  ]
Starting arp: <not configured>
Starting xinetd: [  OK  ]
Starting bp_init:  [  OK  ]
Starting bypass_off:  [  OK  ]
Starting crond: [  OK  ]
Starting cpri_d:  cpridstart: Starting cprid
[1] 7382
[  OK  ]
Starting cpboot:  cpstart: Power-Up self tests passed successfully

cpstart: Starting product - SVN Foundation

SVN Foundation: cpWatchDog already running
SVN Foundation: Starting cpd
Multiportal daemon: starting mpdaemon
SVN Foundation started

cpstart: Starting product - VPN-1

FireWall-1: starting external VPN module -- OK
cpwd_admin:
Process CPHAMCSET started successfully (pid=8208)
FireWall-1: Starting fwd

SecureXL disabled, cannot use affinity commands
SecureXL will be started after a policy is loaded.
FireWall-1: Fetching policy

Installing Security Policy Internet-CP-Cluster on [email protected]
wdt stop function not defined

Oops: 0000 [#1]
SMP
last sysfs file: /devices/pci0000:00/0000:00:00.0/class
Modules linked in: w83627ehf(U) hwmon_vid(U) hwmon(U) button(U) xfrm_nalgo(U) crypto_api(U) 8021q(U) wrpmodmod(PU) vpn_1(PU) fw_1(PU) vpn_0(PU) fw_0(PU) simmod(PU) bridge(U) llc(U) ipsctlmod(PU) parport_pc(U) lp(U) parport(U) sg(U) pcspkr(U) bypass_sb_gpio(U) i2c_i801(U) bypass_class(U) igb(U) i2c_core(U) e1000e(U) serio_raw(U) ip_srs_apic(U) dm_snapshot(U) dm_zero(U) dm_mirror(U) dm_mod(U) ata_piix(U) libata(U) sd_mod(U) scsi_mod(U) ext3(U) jbd(U) ehci_hcd(U) ohci_hcd(U) uhci_hcd(U)
CPU:    1
EIP:    0060:[<f13bf15b>]    Tainted: P      VLI
EFLAGS: 00010202   (2.6.18-92cp #1)
EIP is at cphwd_api_init+0x82b/0xe90 [simmod]
eax: 5505b527   ebx: 00000005   ecx: 00000000   edx: 00000080
esi: 00000001   edi: f1685580   ebp: f1683120   esp: e2e5b984
ds: 007b   es: 007b   ss: 0068
Process fw_full (pid: 8553, ti=e2e58000 task=ef452c70 task.ti=e2e58000)
Stack: f1441ac0 00000002 00000000 80405d5a f40e3c74 00000000 f40e3e80 00000000
       f13be930 e2e5b9cc f40e3c74 00000000 f2d2eb97 e2e5b9cc f338ae30 00000060
       00000202 f40e3e80 00000000 00000000 00000000 00000001 00000002 00000000
Call Trace:
[<e2e5b990>] <0> [<80405d5a>] common_interrupt+0x1a/0x20
[<e2e5b9a4>] <0> [<f13be930>] cphwd_api_init+0x0/0xe90 [simmod]
[<e2e5b9b4>] <0> [<f2d2eb97>] cphwd_api_init_+0x97/0x100 [fw_0]
[<e2e5b9bc>] <0> [<f338ae30>] fwhamultik_validate_not_locked+0x0/0x90 [fw_0]
[<e2e5b9e8>] <0> [<f2d1b0c4>] cphwd_start+0x2174/0x2cc0 [fw_0]
[<e2e5ba64>] <0> [<804388a9>] update_process_times+0x59/0x90
[<e2e5ba74>] <0> [<f2eaa135>] hmem_global_receive_returned_blocks+0x65/0xd0 [fw_0]
[<e2e5ba78>] <0> [<8041e50a>] smp_apic_timer_interrupt+0x7a/0x80
[<e2e5ba84>] <0> [<80405deb>] apic_timer_interrupt+0x1f/0x24

2. Enter into Maintenance Mode

Following Steps will bring your CheckPoint appliance into maintenance mode:
  • Connect to the machine over console (serial).
  • Reboot the machine (power cycle). 
  • During the boot, press a key on the "Press any key to see the boot menu" screen. This should open the Check Point Boot Menu. By default, user has only 5 seconds to press any key. 
  • Choose the "Start in maintenance mode" and press Enter.
  • Enter the Admin credentials and press Enter.


3. Uninstall the hotfix from /opt/CPsuite-R77 folder

sh-3.1# fw ver
This is Check Point's software version R77.10 - Build 243
 List all installed hotfix. You will see that problem one marked with red color:
sh-3.1# cpinfo -y
Error: 'Couldn't connect to /tmp/xgets:  Connection refused
'.
------------------------
Hotfix versions
------------------------
[FW1]
  HOTFIX_R77_10
  HOTFIX_R77_HF_HA10_005
  HOTFIX_GYPSY_LTE_HF_001 
[PPACK]
  HOTFIX_R77_10
[SecurePlatform]
  HOTFIX_R77_10_GAIA_GHOST_833
[CVPN]
  HOTFIX_R77_10
[CPinfo]
  No hotfixes..
[SmartLog]
  HOTFIX_R77_10 


Go to /opt/CPsuite-R77 folder:
Note: Usually it is the parent folder $FWDIR. Based on the version you are having on your Checkpoint Device, the real folder directory is different. In this case, it is Gaia R77.10, and folder is /opt/CPsuite-R77.


sh-3.1# cd CPsuite-R77
sh-3.1# ls
CPinstall    fw1_wrapper_HOTFIX_GYPSY_LTE_HF_001_bcp.tgz
LICENSE.TXT  fw1_wrapper_HOTFIX_GYPSY_LTE_HF_001_bcp.tgz.new.txt
conf         fw1_wrapper_HOTFIX_R77_HF_HA10_005_bcp.tgz
fg1          fw1_wrapper_HOTFIX_R77_HF_HA10_005_bcp.tgz.new.txt
fw1          uninstall_fw1_wrapper_HOTFIX_GYPSY_LTE_HF_001
fw1_wrapper  uninstall_fw1_wrapper_HOTFIX_R77_HF_HA10_005


sh-3.1# ls -ali
total 122712
328062 drwxrwx--x  7 admin bin      4096 Mar 15 10:26 .
 65537 drwxr-xr-x 19 admin root     4096 Aug  6  2014 ..
328064 drwxrwx---  2 admin bin      4096 Aug  6  2014 CPinstall
328066 -rwxrwx---  1 admin bin     38604 Jan 16  2014 LICENSE.TXT
328067 drwxrwx---  2 admin bin      4096 Aug  6  2014 conf
328069 drwxrwx---  9 admin bin      4096 Nov  9 01:37 fg1
328095 drwxrwx--x 30 admin bin      4096 Mar 15 12:35 fw1
852062 drwxr-x---  3 admin bin      4096 Apr  7  2014 fw1_wrapper
327694 -rw-rw----  1 admin root 72317473 Mar 15 10:25 fw1_wrapper_HOTFIX_GYPSY_LTE_HF_001_bcp.tgz
327692 -rw-rw----  1 admin root      763 Mar 15 10:24 fw1_wrapper_HOTFIX_GYPSY_LTE_HF_001_bcp.tgz.new.txt
329068 -rw-rw----  1 admin root 53080782 Aug  6  2014 fw1_wrapper_HOTFIX_R77_HF_HA10_005_bcp.tgz
329067 -rw-rw----  1 admin root      187 Aug  6  2014 fw1_wrapper_HOTFIX_R77_HF_HA10_005_bcp.tgz.new.txt
327700 -rwxr-x---  1 admin bin     18224 Nov  9 01:37 uninstall_fw1_wrapper_HOTFIX_GYPSY_LTE_HF_001
329069 -rwxr-x---  1 admin bin     18218 Apr  7  2014 uninstall_fw1_wrapper_HOTFIX_R77_HF_HA10_005

sh-3.1# ./uninstall_fw1_wrapper_HOTFIX_GYPSY_LTE_HF_001 
Validating uninstall archive...
Do you want to proceed with uninstallation of
Security Gateway Power/UTM R77.10 GYPSY_LTE_HF_001 on this computer?
If you choose to proceed, uninstall will perform CPSTOP.
To proceed type y to cancel type n :
y
 cpwd_admin: Failed to submit request to cpWatchDog
cvpnd: no process killed
dbwriter: no process killed
cvpnproc: no process killed
MoveFileServer: no process killed
CvpnUMD: no process killed
Mobile Access: Stopping MoveFileDemuxer service (if needed)
 cpwd_admin: Failed to submit request to cpWatchDog
Mobile Access: MoveFileDemuxer is not running
Exception: connect() failed - Network is unreachable
Multiportal daemon is not running
Pinger: no process killed
Mobile Access: Successfully stopped Mobile Access services
 cpwd_admin: Failed to submit request to cpWatchDog
SmartView Monitor: Unable to find CpWatchDog - run cpstart
FloodGate-1 is already stopped.
 Unable to open '/dev/fw0': No such file or directory
 fw_syncn_set: failed to set off synchronization
 cpwd_admin: Failed to submit request to cpWatchDog
 Unable to open '/dev/fw0': No such file or directory
 Failed to notify kernel: No such file or directory
 HA not stopped.
VPN-1/FW-1 stopped
Multi portal stopped
fw: Unable to open '/dev/fw0': Unknown error 4294967295
fw: Set operation failed: failed to get parameter
fw: set: Operation failed: Unknown error 4294967295
SVN Foundation: cpd is not running
Multiportal daemon: mpdaemon is not running
 cpwd_admin: Failed to submit request to cpWatchDog
SVN Foundation: cpWatchDog is not running
SVN Foundation stopped
Launching pre-uninstall utility
Removing gx.lf file from registry...
****************
Security Gateway Power/UTM R77.10
Security Gateway Power/UTM R77.10 GYPSY_LTE_HF_001
Uninstall completed successfully.
****************

***********************************************************

Don't forget to reboot the machine!!

***********************************************************

sh-3.1# reboot
Preforming soft reboot
INIT: Sending processes the TERM signal
INIT: Starting killall:  [  OK  ]
Starting bypass_on:  [  OK  ]
Sending all processes the TERM signal...
Sending all processes the KILL signal...
Saving random seed:
Syncing hardware clock to system time
Turning off swap:
Unmounting file systems:
mount: /proc is busy
Please stand by while rebooting the system...
Restarting system.

4. Verify Hotfix uninstalled

You will find HOTFIX_GYPSY_LTE_HF_001 has gone from the list.
[[email protected]:0]# cpinfo -y
------------------------
Hotfix versions
------------------------
[FW1]
  HOTFIX_R77_10
  HOTFIX_R77_HF_HA10_005
[SecurePlatform]
  HOTFIX_R77_10_GAIA_GHOST_833
[PPACK]
  HOTFIX_R77_10
[CVPN]
  HOTFIX_R77_10
[CPinfo]
  No hotfixes..
[SmartLog]
  HOTFIX_R77_10
[rtm]
  No hotfixes..

Monday, February 23, 2015

A Quick Test to Check Point Capsule Cloud Service

Not sure how many Checkpoint customers are using this service, but it is quite attractive when I heard about it. It will help your remote users connect with your global offices and Internet seamlessly.  

Roaming users will use a vpn tunnel to connect to Check Point's cloud network, from there they could access their company's Internal network with another pre-built vpn tunnel. Also from Check Point Cloud, roaming users will be able to browse Internet safely with Check Point's cloud service:
  • URL Filtering
  • Anti-Virus
  • Anti-Bot
  • Threat Emulation
  • IPS
  • HTTPS Inspection

Lets start to experience it:

Step 1: Register an account at https://cloud.checkpoint.com/ with your email account

 After registration, you will receive a email with subject "Your Capsule Connect registration code". Inside the email, there are all links to download the client for Windows, Macintosh, Android and iOS versions.

Step 2: Download and Install the Client:

From "Your Capsule Connect registration code" email, download windows client on your pc. Double click and follow on screen instruction to complete software installation. You will find a cloud icon appears on your right bottom screen. Basically the client will install a new Local Area Connection network driver "Check Point Virtual Network Adapter For Cloud Connect" in your system.
Right click the cloud icon:
Select 'Show Client' menu to enable main window. Choose connect button to make a connection to Check Point's Cloud network.
After system connected to the cloud, you will find your system got a new ip address (172.16.9.28/22) from DHCP server 172.16.9.27 with DNS server 8.8.8.8.

Also from Speedtest.net, we will see the system got an US ip address 208.43.242.98. The download and upload speed is not that bad. 

My pc is having almost 100Mbps download / upload speed without connecting to Checkpoint Cloud.

Step 3: Log into https://cloud.checkpoint.com/ to review the configuration and policies

Check Point Capsule Cloud Policy Tab.
Under Security Policy, there are three features enabled:

  • URL Filtering
  • Threat Prevention
  • HTTPS Inspection


Step 4: Troubleshooting for https website issue

By default, there is a problem to browse https website, such as Gmail site.It shows a This Connection is Untrusted and the connection is blocked. 
From the "Logs & Reports" tab, it shows Check Point Cloud Service Application Cloud blade blocked those 
Double click the log entry you will get the log details for that record.

After turned off HTTPS Inspection from Policy tab, those https website become available again. 

Reference:






Monday, December 1, 2014

Check Point Cluster Member Gateway Drops Ping Packets Without Log in Smartview Tracker

When working on network device monitoring project, there is interesting thing happened on Check Point cluster gateways. I am not able to ping active cluster member. Also surprising thing is there is no log in Smartview Tracker.

Symptoms: 

Cluster member 1 is active and having ip address 172.17.3.35. Cluster member 2 is standby and having ip address 172.17.3.36. Cluster virtual ip is 172.17.3.34. Network Monitoring software uses ping to decide if both cluster members alive or not to decide to send out alarming email.

Interesting thing is I am able to ping 172.17.3.34 (VIP) and 172.17.3.36 (Secondary Cluster member), but not 172.17.3.35 (active cluster member)

I did a test from server 172.17.3.83 which is same zone as firewalls interfaces locating.

From active cluster member gateway expert mode, use fw ctl zdebug drop command, I got some dropped packets log:

;[fw4_0];fw_log_drop_ex: Packet proto=1 172.17.3.83:2048 -> 172.17.3.35:43221 dropped by fw_handle_first_packet Reason: fwconn_key_init_links (INBOUND) failed


Causes:

Check Point sk26874 (Cannot simultaneously ping Virtual IP address of the cluster and IP addresses of physical interfaces on cluster members from a remote host) has well explanation the cause regarding this kind of drop:

  1. When the Check Point Security Gateway / cluster member creates an ICMP connection in the Connections Table, a dummy port is allocated in order to make this connection unique (since ICMP packets do not have real port numbers). The dummy port is calculated based on protocol-level session IDs.

    Under certain conditions, the dummy port is calculated to be the same for multiple connections, which causes a conflict in the Connections Table, causing the drop.
  2. In ClusterXL configured in High Availability New Mode / VRRP cluster , the ICMP Request sent to Cluster VIP address and to the IP address of the physical interface on Active/Master member, are processed by Active/Master member ("NAT-folded" from physical IP address of Active member). Since these two ICMP Requests have the same parameters, Active/Master member is not able to distinguish between them. As a result, the first of these two ICMP Requests will be processed correctly, and the second of these two ICMP Requests will be dropped.

Solutions:

step 1. Change on the fly but not survive a reboot

fw ctl set int fw_allow_simultaneous_ping 1 

step 2. Permanent Changes on the gateways:

Edit the $FWDIR/boot/modules/fwkern.conf
add following line in:

fw_allow_simultaneous_ping=1 


3. Verify the value of fw_allow_simultaneous_ping
fw ctl get int fw_allow_simultaneous_ping 


Reference:


Updates:

There is new post for another scenery Checkpoint firewall dropped the packets in the kernel without logged into SmartTrack and I recorded into the following link:

Monday, November 10, 2014

Using Symantec Verisign SSL Certificate for Check Point SSL VPN Mobile Access Portal

Mobile Blade has been enabled on Checkpoint Gateway from my series posts "Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 1 (Local User Authentication)".  Mobile Access portal by default will use self signed CA certificate which will cause a warning message from remote user's browser because of untrusted ssl certificate.

This post is to show all basic steps to how to get a CA signed certificate for Mobile Access Portal to avoid this kind of warning message.

Friday, October 17, 2014

Enable Checkpoint SSL VPN Remote Access: Step by Step Part 4 - Two Factor Authentication (AD and SMS)

Part 1: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 1 (Local User Authentication)
Part 2: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 2 (AD Authentication)
Part 3: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 3 (Certs and Two Factor Authentication)
Part 4: Enable Checkpoint SSL VPN Remote Access: Step by Step Part 4 - Two Factor Authentication  (AD and SMS)


Check Point provide nice integration for Two-Factor Authentication with DynamicID, which is One Time Password.

In this lab, I choose SMS Provider HQSMS.com. It is free for signup and provide 0.30 credit for you to test SMS function, which is 10 SMS messages.



To enable two factor authentication with DynamicID for SMS is also quite straight forward.

Steps:

1. For first factor authentication, username and password has been picked, which is Active Directory account.



2. Second factor authentication is DynamicID. 

Either Global settings or Custom settings for this gateway is fine. You have to check the option to choose "Challenge users to provide the DynamicID one time password sent to their email account or mobile device via SMS."

Then You will have to fill in SMS provider or Email Settings as show in the following screenshot.

3. Add email address and Mobile Phone number into Test1 AD account



4. After the policy push to the gateway, test it with this Test1 AD account.

The first authentication is AD account username and password.

 After you sign in with your AD account, automatically gateway will send out One Time Password (verification code) request to SMS Provider.

The registered mobile phone number (+1xxxxxx9266) in Test1 AD account will receive a SMS sent from +44 7156066456:
"Mobile Access DynamicID one time password:611720"

Then verification code can be entered into next screen.
If verification code is correct, you will get into Check Point Mobile window to access allowed resources defined in the Mobile Access Blade.

Reference:


Tuesday, October 14, 2014

Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 3 (Certs and Two Factor Authentication)

This is the third part of Checkpoint SSL VPN Series LAB.
Part 1: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 1 (Local User Authentication)
Part 2: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 2 (AD Authentication)
Part 3: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 3 (Certs and Two Factor Authentication)
Part 4: Enable Checkpoint SSL VPN Remote Access: Step by Step Part 4 - Two Factor Authentication  (AD and SMS)

In this part, it will introduce how to use Checkpoint Internal CA issued Certs and how to enable two factor authentication using certs and AD accounts.

In Part 2, AD authentication has been tested and next step is to create certificate for each remote user.

Enable Certificate Authentication Steps:

1. Create Certificate for user test2.

2. Click OK after enter all information for Certificate File.

3. Once certificate created, it will show in the list. It also can be revoked.


4. Import Certificate into Client Machine

Double click the cert file from Client Machine. It will enable Certificate Import Wizard. Enter your password, click next and next then it will be completed this import.



5. Change authentication method to from Username and Password to Personal Certificate on the Gateway Properties:

6. After installed polices to gateway, test it from Client Machine with your imported Personal Certificate:



Enable two factor authentication (Certs and AD account) Steps

1. Certificate will be the first method and AD username/password will be second factor. 

Here is the list Supported Authentication Schemes Combinations:

2. GuiDBedit Tool 

It will be used to configure Multiple Authentication Schemes to allow the administrator to request multiple proofs of user's identity. In our case, for example, request the user to install his certificate and enter his AD password for authenticating.

GuiDBedit Tool will be found at C:\Program Files (x86)\CheckPoint\SmartConsole\RXX\PROGRAM\GuiDBedit.exe

3. Verify the first factor authentication is Certificates.

It will be found at Tables -> Network Objects ->network_objects.
Found the Gateway / Cluster object, in this case , for example, it is R77.
From lower pane, check the value of realms_for blades-> ssl_vpn ->authentication-> auth_schemes->Element Index 0 -> auth_scheme.
It is certificate which is configured before thorough Smart Dashboard.

4. right click on 'auth_schemes' ,-> 'Add...'

5. Enter a value for Index:  

1 for the second authentication scheme. 0 is already there for 1st factor, which is 'Element Index 0'

6. Go to 'Element Index 1', right click on 'auth_scheme' -> 'Edit...'. Choose 'user_pass' for AD username and password as 2nd factor.

7. Save the changes ('File' menu - 'Save All').  

Close the GuiDBedit Tool. Connect to Security Management Server with SmartDashboard. Install the policy onto Security Gateway / Cluster object.

8. Test










Reference:

1. Multiple Authentication Schemes for Mobile Access





Monday, October 13, 2014

Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 2 (AD Authentication)

Part 1: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 1 (Local User Authentication)
Part 2: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 2 (AD Authentication)
Part 3: Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 3 (Certs and Two Factor Authentication)
Part 4: Enable Checkpoint SSL VPN Remote Access: Step by Step Part 4 - Two Factor Authentication  (AD and SMS)

In previous lab Part 1 "Enable Checkpoint SSL VPN Remote Access: Step by Step Instruction Part 1 (Local User Authentication)", it shows the first part how to enable Checkpoint SSL VPN with local user authentication and how to add a native application. In this second part of lab, it will show the integration with Active Directory accounts for remote ssl vpn access.

Topology: 

Steps:

1. Enable Identity Awareness Blade in Check Point gateway properties to start Identity Awareness Configuration wizard:

 2. Choose AD Query at Methods for Acquiring Identity window.

 3. Add new Active Directory

 4. Finish the configuration wizard

 5. Add AD Group into Mobile Access Policy as show in the rule No. 2.


6. Use AD account 'test' to do test 

After log in, it only shows Native Application Section to connect. If account is not defined in the any mobile access rule, the login window will show "User is unauthorized" although there is account in the AD.