Problem when two Checkpoint Clusters Connected on same Cisco Switch - NETSEC

Latest

Learning, Sharing, Creating

Cybersecurity Memo

Thursday, December 22, 2011

Problem when two Checkpoint Clusters Connected on same Cisco Switch

Got mac address flapping messages on Cisco Switch log. 
Dec 22 17:27:16: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/12 and port Gi0/11
Dec 22 17:27:16: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/15 and port Gi0/16
Dec 22 17:27:31: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/12 and port Gi0/11
Dec 22 17:27:31: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/16 and port Gi0/15
Dec 22 17:27:46: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/16 and port Gi0/15
Dec 22 17:27:46: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/11 and port Gi0/12
Dec 22 17:28:01: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/16 and port Gi0/15
Dec 22 17:28:01: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/11 and port Gi0/12
Dec 22 17:28:16: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/16 and port Gi0/15
Dec 22 17:28:16: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/12 and port Gi0/11
Dec 22 17:28:31: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/16 and port Gi0/15
Dec 22 17:28:31: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/12 and port Gi0/11
Dec 22 17:28:46: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/16 and port Gi0/15
Dec 22 17:28:46: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/11 and port Gi0/12
Dec 22 17:29:01: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 20 is flapping between port Gi0/16 and port Gi0/15
Dec 22 17:29:01: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 20 is flapping between port Gi0/11 and port Gi0/12

Did remember changed some settings on Checkpoint UTM clusters and messages gone. Not sure why it comes back again. After research, found why.

















The environment is Gaia R75.40 now, previous it was SPLAT R70, R71 and R75. Issue was happenning for quite a while. Good thing is not affecting production at all. Maybe just we do not know. Resolved once but solution did not survive the reboot.

Here is the reason why this flapping happened based on the word from
 CP_R75.40_ClusterXL_AdminGuide.pdf:

For CCP traffic:
The first four bytes of the source MAC address are all zero: 00.00.00.00
The fifth byte of the source MAC address is a magic number. By default, the value is 0xfe.
The sixth byte is the ID of the sending cluster member

Thats why the switch has mac add flapping since duplicate CCP traffic mac address if there are two set default setting checkpoint clusters on same vlan.

The solution is quite simple and it take effect immediately with following command on one set of checkpoint cluster:

fw ctl set int fwha_mac_magic 0xee  (This command change the fifth byte MAC address from 0xfe to 0xee.)

Unfortunately this change doesn't survive a reboot. Here is final solution for permanent change:
1. Edit the file $FWDIR/boot/modules/fwkern.conf.
2. Add the line  fwha_mac_magic=0xee
3. Reboot.


2 comments:

  1. another option, to use seperate VLANs for each cluster ?

    ReplyDelete
    Replies
    1. vlan won't help at this situation. those are layer 2 traffic.

      Delete