Latest Posts

SRX Load Rescue Configuration After Reboot

It did not happen often, but when it happened, you will need to know how to fix it.


The rescue configuration is a previously committed, valid configuration. You must have previously set the rescue configuration through the J-Web interface or the CLI.

[email protected]> request system configuration rescue save 

[email protected]> show system configuration rescue 



During today's work, one of SRX firewalls got a problem to load regular configuration file, and it loaded rescue configuration. Here is what the console output told me:



*** FINAL System shutdown message from [email protected] ***           

System going down IMMEDIATELY                                                  

                                                                          


{secondary:node1}

[email protected]> Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `vnlru_mem' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 0 done

syncing disks... All buffers synced.

Uptime: 32m35s
Rebooting...
cpu_reset: Stopping other CPUs


U-Boot 1.1.6-JNPR-1.7 (Build time: May  4 2010 - 06:59:58)


SRX_240_HIGHMEM board revision major:1, minor:50, serial #: AAEK3334

OCTEON CN5230R-SCP pass 2.0, Core clock: 600 MHz, DDR clock: 333 MHz (666 Mhz data rate)
DRAM:  1024 MB
Starting Memory POST... 
Checking datalines... OK
Checking address lines... OK
Checking 512K memory for U-Boot... OK.
Running U-Boot CRC Test... OK.
Flash:  4 MB
USB:   scanning bus for devices... 
Root Hub 0: 3 USB Device(s) found
Root Hub 1: 1 USB Device(s) found
       scanning bus for storage devices... 1 Storage Device(s) found
Clearing DRAM........ done
BIST check passed.
1:00:00.0 Vendor/Device ID = 0x811210b5
1:01:07.0 Vendor/Device ID = 0xc72414e4
Boot Media: nand-flash usb 
Net:   octeth0
POST Passed
Press SPACE to abort autoboot in 1 seconds
ELF file is 32 bit
Loading .text @ 0x8f000078 (246092 bytes)
Loading .rodata @ 0x8f03c1c4 (13940 bytes)
Loading .rodata.str1.4 @ 0x8f03f838 (16580 bytes)
Loading set_Xcommand_set @ 0x8f0438fc (104 bytes)
Loading .rodata.cst4 @ 0x8f043964 (20 bytes)
Loading .data @ 0x8f044000 (5620 bytes)
Loading .data.rel.ro @ 0x8f0455f4 (120 bytes)
Loading .data.rel @ 0x8f04566c (136 bytes)
Clearing .bss @ 0x8f0456f8 (11912 bytes)
## Starting application at 0x8f000078 ...
Consoles: U-Boot console  
Found compatible API, ver. 1.7

FreeBSD/MIPS U-Boot bootstrap loader, Revision 1.7

([email protected], Tue May  4 07:15:51 UTC 2010)
Memory: 1024MB
[0]Booting from nand-flash slice 1
Un-Protected 1 sectors
writing to flash...
Protected 1 sectors
Loading /boot/defaults/loader.conf 
/kernel data=0xb0567c+0x134494 syms=[0x4+0x8aa50+0x4+0xc8fc6]


Hit [Enter] to boot immediately, or space bar for command prompt.

Booting [/kernel]...               
Kernel entry at 0x801000e0 ...
init regular console
Primary ICache: Sets 64 Size 128 Asso 4
Primary DCache: Sets 1 Size 128 Asso 64
Secondary DCache: Sets 512 Size 128 Asso 8
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
kld_map_v: 0x8ff80000, kld_map_p: 0x0
Copyright (c) 1996-2014, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
JUNOS 12.1X44-D40.2 #0: 2014-08-28 12:20:14 UTC
    [email protected]:/volume/build/junos/12.1/service/12.1X44-D40.2/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
JUNOS 12.1X44-D40.2 #0: 2014-08-28 12:20:14 UTC
    [email protected]:/volume/build/junos/12.1/service/12.1X44-D40.2/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
real memory  = 1073741824 (1024MB)
avail memory = 526438400 (502MB)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
netisr_init: !debug_mpsafenet, forcing maxthreads from 4 to 1
cpu0 on motherboard
: CAVIUM's OCTEON 52XX CPU Rev. 0.8 with no FPU implemented
        L1 Cache: I size 32kb(128 line), D size 8kb(128 line), sixty four way.
        L2 Cache: Size 512kb, 8 way
obio0 on motherboard
uart0: <Octeon-16550 channel 0> on obio0
uart0: console (9600,n,8,1)
twsi0 on obio0
dwc0: <Synopsis DWC OTG Controller Driver> on obio0
usb0: <USB Bus for DWC OTG Controller> on dwc0
usb0: USB revision 2.0
uhub0: vendor 0x0000 DWC OTG root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1: vendor 0x0409 product 0x005a, class 9/0, rev 2.00/1.00, addr 2
uhub1: single transaction translator
uhub1: 3 ports with 2 removable, self powered
umass0: STMicroelectronics ST72682  High Speed Mode, rev 2.00/2.10, addr 3
dwc1: <Synopsis DWC OTG Controller Driver> on obio0
usb1: <USB Bus for DWC OTG Controller> on dwc1
usb1: USB revision 2.0
uhub2: vendor 0x0000 DWC OTG root hub, class 9/0, rev 2.00/1.00, addr 1
uhub2: 1 port with 1 removable, self powered
cpld0 on obio0
pcib1: <Cavium on-chip PCIe HOST bridge> on obio0
Disabling Octeon big bar support
PCIe: Waiting for port 0 to finish reset
PCIe: Port 0 link active, 2 lanes
PCIe: Waiting for port 1 to finish reset
PCIe: Port 1 link active, 1 lanes
pcib1: Initialized controller
pci0: <PCI bus> on pcib1
pcib2: <PCI-PCI bridge> irq 0 at device 0.0 on pci0
pci1: <PCI bus> on pcib2
pci1: <serial bus, USB> at device 2.0 (no driver attached)
pci1: <network> at device 7.0 (no driver attached)
pcib0: <Cavium on-chip PCIe HOST bridge> on obio0
pci2: <PCI bus> on pcib0
pci2: <processor> at device 0.0 (no driver attached)
gblmem0 on obio0
octpkt0: <Octeon RGMII> on obio0
cfi0: <AMD/Fujitsu - 4MB> on obio0
Timecounter "mips" frequency 600000000 Hz quality 0
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <ST ST72682 2.10> Removable Direct Access SCSI-2 device 
da0: 40.000MB/s transfers
da0: 1000MB (2048000 512 byte sectors: 64H 32S/T 1000C)
Trying to mount root from ufs:/dev/da0s1a
Attaching /cf/packages/junos via /dev/mdctl...
Mounted junos package on /dev/md0...

Media check on da0

Zone 04 Block 0499 Addr 11f300 : Bad read
Recovering Block
Automatic reboot in progress...
** /dev/da0s1a
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
250 files, 75946 used, 73580 free (28 frags, 9194 blocks, 0.0% fragmentation)
Verified junos signed by PackageProduction_12_1_0
Verified jboot signed by PackageProduction_12_1_0
Verified junos-12.1X44-D40.2-domestic signed by PackageProduction_12_1_0
Checking integrity of BSD labels:
  s1: Passed
  s2: Passed
  s3: Passed
  s4: Passed
** /dev/bo0s3e
** Last Mounted on /config
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
28 files, 52 used, 12386 free (10 frags, 1547 blocks, 0.1% fragmentation)
** /dev/bo0s3f
** Last Mounted on /cf/var
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
616 files, 98342 used, 76976 free (544 frags, 9554 blocks, 0.3% fragmentation)
Checking integrity of licenses:
  JUNOS137657.lic: Passed
  JUNOS187398.lic: Passed
  JUNOS187665.lic: Passed
  JUNOS628672.lic: Passed
Checking integrity of configuration:
  rescue.conf.gz: Passed
Loading configuration ...
mgd: error: Cannot open configuration file: /config/juniper.conf
mgd: warning: loading configuration from /config/rescue.conf.gz
Time and ticks drifted too much,             resetting synchronization...
mgd: commit complete
Setting initial options: .
Starting optional daemons:  usbd.
Doing initial network setup:.
Initial interface configuration:
additional daemons: eventd.
Additional routing options:kern.module_path: /boot//kernel;/boot/modules -> /boot/modules;/modules/ifpfe_drv;/modules;
kld netpfe drv: ifpfed_dialer.
Doing additional network setup:.
Starting final network daemons:.
setting ldconfig path: /usr/lib /opt/lib
starting standard daemons: cron.
Initial rc.mips initialization:.
Local package initialization:.
starting local daemons:set cores for group access
.
kern.securelevel: -1 -> 1
Creating JAIL MFS partition...
JAIL MFS partition created
boot.upgrade.uboot="0xBFC00000"
boot.upgrade.loader="0xBFE00000"
Boot media /dev/da0 has dual root support
WARNING: JUNOS versions running on dual partitions are not same
** /dev/da0s2a
** Last Mounted on /mfs/tmp/snap-tmp.1334/mnt.1334
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
250 files, 75914 used, 74124 free (28 frags, 9262 blocks, 0.0% fragmentation)
Sun Sep 20 17:48:34 UTC 2015

fw-srx-2 (ttyu0)


login: 

For somehow, the regular configuration could not be loaded, and system used rescue configuration instead.

Fix is quite simple. Made a little change to configuration and commit it to generate a new configuration. Reboot and this time console showed the regular configuration loaded successfully:



Checking integrity of licenses:

  JUNOS137657.lic: Passed
  JUNOS187398.lic: Passed
  JUNOS187665.lic: Passed
  JUNOS628672.lic: Passed
Checking integrity of configuration:
  rescue.conf.gz: Passed
Loading configuration ...
mgd: commit complete
Setting initial options: .


No comments