Configure High End Juniper SRX 1400 as Chassis Cluster Steps

There is a new project to configure a new pair of Juniper SRX1400 as Chassis Cluster implementation for one of our customers. Juniper documentation does not have clear guide for 1400 this device although I did find some of configuration guide for high-end device. During set up, I was confusing those ports numbers and fab and control port connections. This post is used to record my steps how to configure them.

root> show chassis hardware 
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                BH1014AA0023      SRX 1400
Midplane         REV 03   711-031012   ACDA5607          SRX1k Backplane
PEM 0            rev 03   740-032015   J027MN004003P     AC Power Supply
PEM 1            rev 03   740-032015   J027SS004003P     AC Power Supply
CB 0             REV 12   750-032544   ACDK8984          SRX1K-RE-12-10
  Routing Engine          BUILTIN      BUILTIN           Routing Engine
  CPP                     BUILTIN      BUILTIN           Central PFE Processor
  Mezz           REV 09   710-021035   ACDL9054          SRX HD Mezzanine Card
FPC 0            REV 19   750-031019   ACDL8051          SRX1k 10GE SYSIO
  PIC 0                   BUILTIN      BUILTIN           6x 1GE RJ45 3x 1GE SFP 3x 10GE SFP+
FPC 1            REV 12   750-032543   ACDH6938          SRX1k Dual Wide NPC+SPC Support Card
  PIC 0                   BUILTIN      BUILTIN           SPU Cp-Flow
FPC 3            REV 19   710-017865   ACDT1404          BUILTIN NPC
  PIC 0                   BUILTIN      BUILTIN           NPC PIC
Fan Tray         -N/A-    -N/A-        -N/A-             SRX 1400 Fan Tray

root> show version 
Model: srx1400
JUNOS Software Release [12.1X44-D25.5]

Configure Steps:

Chassis Cluster Flow Diagram (SRX5800, SRX5600, SRX5400 Devices)

1. Factory Default Configuration:

root> show configuration 
## Last commit: 2014-05-13 02:53:12 UTC by root
version 12.1X44-D25.5;
system {
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
    license {
        autoupdate {
            url https://ae1.juniper.net/junos/key_retrieval;
        }
    }
    ## Warning: missing mandatory statement(s): 'root-authentication'
}
security {
    idp {                               
        ##
        ## Warning: requires 'idp-sig' license
        ##
        security-package {
            url https://services.netscreen.com/cgi-bin/index.cgi;
        }
    }
}

2. Physical Topology and Cabling

Understanding Control Link and Fabric Link

• The control link is used to send control traffic between both the Routing Engines (REs) and between the Primary RE (RG0 primary) and the remote Packet Forwarding Engine (PFE).
• The fabric link connects both PFE’s together. This link is utilized for two main functions. The first function is to synchronize the session states between the two nodes. This is done via RTO (real-time objects) packets going over the fabric link. There are many types of RTO messages, but some of the most important ones are the ‘session create’ and ‘session delete’ messages. The second function of the fabric link is to pass traffic that needs to cross both nodes. This can only occur in A/A scenarios where traffic might enter an interface on one cluster node and needs to exit out of an interface on the other cluster node. Such traffic is also called Z-mode traffic.

To understanding the interface numbering, I list all port number in the following image.

Notes:

• From ge-0/0/0 to ge-0/0/6, it is giga port
• ge-0/0/7, ge-0/0/8 and ge-0/0/9 are 10g ports.
• port 10 and port 11 are giga ports and only can be used as control ports. High end SRX (1400 and up) are always using dual control ports configuration.
• The left port beside port 0 and port 1 is RE (Route Engine) port. Which will be used to fxp0
• In this lab, port 2 (ge-0/0/2 and ge-4/0/2) on both devices are connected to used as fabric port. Juniper recommend to use two ports for fabric connections. Any two ports from 0 to 9 can be used as fabric connections.
• If you plan to use switch in the middle to connect Control link and fabric link, you will have to enable MTU Jumbo Frame support on the interface (from JUNIPER SRX 1400 LAYER 2 HA). 

root@fw-1400-1> show interfaces terse 
Interface               Admin Link Proto    Local                 Remote
ge-0/0/0                up    down
gr-0/0/0                up    up  
ip-0/0/0                up    up  
lt-0/0/0                up    up  
ge-0/0/1                up    down
ge-0/0/2                up    up  
ge-0/0/2.0              up    up   aenet    --> fab0.0
ge-0/0/3                up    down
ge-0/0/4                up    down
ge-0/0/5                up    down
ge-0/0/6                up    down
xe-0/0/7                up    down
xe-0/0/8                up    down
xe-0/0/9                up    down
mt-1/0/0                up    up  
ge-4/0/0                up    down
ge-4/0/1                up    down
ge-4/0/2                up    up  
ge-4/0/2.0              up    up   aenet    --> fab1.0
ge-4/0/3                up    down
ge-4/0/4                up    down
ge-4/0/5                up    down
ge-4/0/6                up    down
xe-4/0/7                up    down
xe-4/0/8                up    down
xe-4/0/9                up    down
mt-5/0/0                up    up  
avs0                    up    up  
avs1                    up    up  
avs1.0                  up    up   inet     254.0.0.254         --> 0/0
                                   inet6    fe80::199       
dsc                     up    up  
em0                     up    up  
em0.0                   up    up   inet     130.48.0.1/2    
                                   tnp      0x2300004       
em1                     up    up  
em1.0                   up    up   inet     130.48.0.1/2    
                                   tnp      0x2300004       
fab0                    up    up  
fab0.0                  up    up   inet     30.49.0.200/24  
fab1                    up    up  
fab1.0                  up    up   inet     30.50.0.200/24  
fxp0                    up    up  
fxp0.0                  up    up   inet     10.9.12.27/24  
gre                     up    up  
ipip                    up    up  
irb                     up    up  
lo0                     up    up  
lo0.16384               up    up   inet     127.0.0.1           --> 0/0
lo0.16385               up    up   inet    
lsi                     up    up  
mtun                    up    up  
pimd                    up    up  
pime                    up    up  
ppd0                    up    up  
ppe0                    up    up  
st0                     up    up  
tap                     up    up  

3. set root password

By default, there is no password for root user.
set system root-authentication plain-text-password

4. Enable Cluster on node 0 and node 1

set chassis cluster cluster-id 3 node 0 reboot 
Note: If you have multiple SRX Chassis Cluster implementation in same Ethernet environmetn, it is mandatory to use different Cluster-ID. Cluster-ID will be used to generate virtual Mac Address for your interfaces. Multiple same cluster-id will cause network collision.

after node 0 rebooted, enable Chassis Cluster on node 1 with same Cluster-ID
set chassis cluster cluster-id 3 node 1 reboot 

5. Configure Fab Link

set interfaces fab0 fabric-options member-interfaces ge-0/0/2
set interfaces fab1 fabric-options member-interfaces ge-4/0/2

root@fw-1400-1> show chassis cluster interfaces    
Control link status: Up

Control interfaces: 
    Index   Interface        Status
    0       em0              Up    
    1       em1              Up    

Fabric link status: Up

Fabric interfaces: 
    Name    Child-interface    Status
                               (Physical/Monitored)
    fab0    ge-0/0/2           Up   / Up  
    fab0   
    fab1    ge-4/0/2           Up   / Up  
    fab1   
   
Redundant-pseudo-interface Information:
    Name         Status      Redundancy-group
    lo0          Up          0  

root@fw-1400-1> show chassis cluster status 
Cluster ID: 3 
Node                  Priority          Status    Preempt  Manual failover

Redundancy group: 0 , Failover count: 1
    node0                   1           primary        no       no  
    node1                   1           secondary      no       no  

6. Basic Configuration 

set groups node0 system host-name fw-1400-1

set groups node0 system syslog file default-log-messages any any
set groups node0 system syslog file default-log-messages structured-data
set groups node0 interfaces fxp0 unit 0 family inet address 10.9.12.26/24
set groups node0 interfaces fxp0 unit 0 family inet address 10.9.12.28/24 master-only
set groups node1 system host-name fw-1400-2

set groups node1 system syslog file default-log-messages any any
set groups node1 system syslog file default-log-messages structured-data
set groups node1 interfaces fxp0 unit 0 family inet address 10.9.12.28/24 master-only
set groups node1 interfaces fxp0 unit 0 family inet address 10.9.12.27/24
set groups system
set groups global-logging security policies from-zone <*> to-zone <*> policy <*> then log session-init
set groups global-policy security policies from-zone <*> to-zone <*> policy default-logdrop match source-address any
set groups global-policy security policies from-zone <*> to-zone <*> policy default-logdrop match destination-address any
set groups global-policy security policies from-zone <*> to-zone <*> policy default-logdrop match application any
set groups global-policy security policies from-zone <*> to-zone <*> policy default-logdrop then deny
set groups global-policy security policies from-zone <*> to-zone <*> policy default-logdrop then log session-init
set apply-groups "${node}"
set system backup-router 10.9.12.1
set system backup-router destination 0.0.0.0/0

set system login user admin uid 2001
set system login user admin class super-user
set system login user admin authentication encrypted-password "$1$2NhvvqTf$qU6tD6tlGGPJ1FdRmQC4/"

set system services ssh protocol-version v2    
set system services netconf ssh
set system services web-management http interface fxp0.0
set system services web-management https system-generated-certificate
set system services web-management https interface fxp0.0

set routing-options static route 0.0.0.0/0 next-hop 10.9.12.1

Note:
After finished commit on Node 0, copy same configuration to Node 1 and do a commit.

7. JunOS Upgrade

root@fw-1400-1> request system software in-service-upgrade /cf/var/tmp/junos-srx1k3k-12.1X44-D40.2-domestic.tgz no-copy reboot 
Chassis ISSU Started
node1:
--------------------------------------------------------------------------
Chassis ISSU Started
ISSU: Validating Image
Initiating in-service-upgrade

node1:
--------------------------------------------------------------------------
Initiating in-service-upgrade
Checking compatibility with configuration
Initializing...
Verified manifest signed by PackageProduction_12_1_0
Using /cf/var/tmp/junos-srx1k3k-12.1X44-D40.2-domestic.tgz
tar: +CONTENTS: time stamp Aug 28 15:10 2014 is 7977882 s in the future
......(Omitted))
tar: junos-srx1k3k-12.1X44-D40.2-domestic.sha1: time stamp Aug 28 15:08 2014 is 7977777 s in the future
tar: junos-srx1k3k-12.1X44-D40.2-domestic.sig: time stamp Aug 28 15:09 2014 is 7977810 s in the future
tar: certs.pem: time stamp Aug 28 10:15 2014 is 7960201 s in the future
tar: +install.junos: time stamp Aug 28 15:09 2014 is 7977838 s in the future
Checking junos requirements on /
Available space: 297274 require: 278080
cp: /var/etc/master.passwd: No such file or directory
tar: etc/db/manifest/jboot: time stamp Aug 28 14:45 2014 is 7976348 s in the future
tar: etc/db/manifest/jboot.certs: time stamp Aug 28 10:15 2014 is 7960200 s in the future
......(Omitted)
tar: etc/namedb: time stamp Aug 28 14:45 2014 is 7976348 s in the future
Saving boot file package in /var/sw/pkg/junos-boot-srx1k3k-12.1X44-D40.2.tgz
Verified manifest signed by PackageProduction_12_1_0
Hardware Database regeneration succeeded
Validating against /config/juniper.conf.gz
mgd: commit complete
Validation succeeded
failover all RG 1+ groups to node 0 
Initiated manual failover for all redundancy-groups to node0
Redundancy-groups-0 will not failover and the primaryship remains unchanged.
ISSU: Preparing Backup RE
Pushing bundle to node1
Installing package '/cf/var/tmp/junos-srx1k3k-12.1X44-D40.2-domestic.tgz' ...
Verified junos-boot-srx1k3k-12.1X44-D40.2.tgz signed by PackageProduction_12_1_0
Verified junos-srx1k3k-12.1X44-D40.2-domestic signed by PackageProduction_12_1_0
Available space: 297274 require: 278080
Saving boot file package in /var/sw/pkg/junos-boot-srx1k3k-12.1X44-D40.2.tgz
JUNOS 12.1X44-D40.2 will become active at next reboot
WARNING: A reboot is required to load this software correctly
WARNING:     Use the 'request system reboot' command
WARNING:         when software installation is complete
Saving state for rollback ...
Finished upgrading secondary node node1
Rebooting Secondary Node

node1:
--------------------------------------------------------------------------
Shutdown NOW!
[pid 4395]
ISSU: Backup RE Prepare Done
Waiting for node1 to reboot.
node1 booted up.
Waiting for node1 to become secondary
node1 became secondary.
Waiting for node1 to be ready for failover
ISSU: Preparing Daemons
Secondary node1 ready for failover.
Installing package '/cf/var/tmp/junos-srx1k3k-12.1X44-D40.2-domestic.tgz' ...
tar: +CONTENTS: time stamp Aug 28 15:10 2014 is 7975937 s in the future
tar: +COMMENT: time stamp Aug 28 15:10 2014 is 7975937 s in the future
......(Omitted)
tar: boot/loader.conf: time stamp Aug 28 09:54 2014 is 7956926 s in the future
Saving boot file package in /var/sw/pkg/junos-boot-srx1k3k-12.1X44-D40.2.tgz
JUNOS 12.1X44-D40.2 will become active at next reboot
WARNING: A reboot is required to load this software correctly
WARNING:     Use the 'request system reboot' command
WARNING:         when software installation is complete
Saving state for rollback ...
Failing over all redundancy-groups to node1
ISSU: Preparing for Switchover
Initiated failover for all the redundancy groups to node1
Waiting for node1 take over all redundancy groups

Message from syslogd@fw-1400-1 at May 28 07:39:50  ...
fw-1400-1 node0.cpp0 RDP: Remote side closed connection: rdp.(19922946:16384).(serverRouter:chassis) 
Exiting in-service-upgrade window

node1:
--------------------------------------------------------------------------
Exiting in-service-upgrade window

error: the chassis-control subsystem is not running

8. Licensing Steps

• Click SUBSCRIPTION REGISTRATION at SUPPORT - SUBSCRIPTION REGISTRATION page to register your subscription with your device's serial number and authentication code
• Generate License for your subscription from Juniper Networks::Manage Product Licenses page. Choose right product category for next step. In my case, it is J-Series Service Routers and SRX Devices.
• Enter your Serial Number and JunOS version to generate key.
• 
• Download a key file to local then copy into device with following command:
• root@fw-1400-1> request system license add terminal 

[Type ^D at a new line to end input,

 enter blank line between each license key]

JUNOS572483 aeaqia qmijed cojrgr aumbq gi3qqb qcdw77

            zqlia4 ujcod2 hu4kvr aamia lwtbvc kzqvs

            zimck hu5k3 gd64cy hvhg66 hfbl2r 2nedh

            iqq

JUNOS572483: successfully added

add license complete (no errors)

9. Accessories

10Gb OM3 Duplex Multimode Fiber Patch Cabel

 Juniper 1GE SFP

 Cisco 10G SFP

 LC-LC Fiber Cable

Reference:

• Example: Configuring an SRX Series Services Gateway for the High-End as a Chassis Cluster
• Configure Chassis Cluster (High Availability) on the High-End SRX devices: SRX1400, SRX3400, SRX3600
• request system software in-service-upgrade (Maintenance)
• SRX Series Chassis Cluster Configuration Overview