O Errors During TFTP Cisco IOS Image Copying - NETSEC

Latest

Learning, Sharing, Creating

Cybersecurity Memo

Wednesday, January 20, 2016

O Errors During TFTP Cisco IOS Image Copying

I were using TFTPD32 to transfer a image to Cisco 3560 switch. It was fine at the beginning then when it was almost done I got lots of O errors.

O usually indicates that a packet was received out of order and the copy process may fail.


Symptoms:


Switch#copy tftp: flash:
Address or name of remote host []? 1.1.1.2
Source filename []? c3560-ipbasek9-mz.150-2.SE9.bin
Destination filename [c3560-ipbasek9-mz.150-2.SE9.bin]?
Accessing tftp://1.1.1.2/c3560-ipbasek9-mz.150-2.SE9.bin...
Loading c3560-ipbasek9-mz.150-2.SE9.bin from 1.1.1.2 (via Vlan1): !!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!O!OOOOOO!OOOOOO!OOOOOOOO!OOOOOOOO!OOOOOOOOOO... [timed out]

%Error reading tftp://1.1.1.2/c3560-ipbasek9-mz.150-2.SE9.bin (Timed out)
Switch#




I did many times of this kind of tasks before. I even have a post "Upgrade Cisco 3560X IOS Take 30+ Minutes" regarding the process how to upgrade 3560X-48T-S IOS to latest version. At that time, upgrading took me more than 30 minutes.

Lets review what TFTP used for and what O means for the file transferring.

TFTP uses UDP port 69, and block numbers within packets to order them. There was originally a 512 byte block per packet with a 32MB file limit but this has now increased to a variable block size and a 4GB limit with RFC 2348. TFTP is only used significantly today to transfer network device images & associated configurations across trusted networks, it has no methods of listing directories, packet encryption or passwords.

The characters which show the status of a transfer in IOS are
! – Indicates a successful transfer to 10 packets
O – Indicates out of sequence packets
. – Indicates a timeout
E – Uppercase E indicates an error
e – Lowercase e indicates flash is being erased
V – Indicates checksum verification
Solution:
I was thinking it may relates to my network cable or switch flash itself. During reviewing the folder on 3560 Switch, I found it is caused by not enough space. The IOS file size is more than 16M bytes.


Switch#sh flash:

Directory of flash:/

    2  -rwx        1940   Mar 1 1993 00:02:57 +00:00  private-config.text
    3  -rwx     8811199   Mar 1 1993 02:36:58 +00:00  c3560-ipbasek9-mz.122-46.SE.bin
    4  -rwx          24  Sep 21 2012 16:01:17 +00:00  private-config.text.renamed
    5  -rwx        8459  Sep 21 2012 16:01:17 +00:00  config.text.renamed
    6  -rwx         796   Mar 1 1993 00:02:34 +00:00  vlan.dat.renamed
    7  drwx         192   Mar 1 1993 00:08:16 +00:00  c3560-ipbase-mz.122-35.SE5
  465  -rwx        2072   Mar 1 1993 00:02:57 +00:00  multiple-fs
  467  -rwx        2496   Mar 1 1993 00:02:57 +00:00  config.text

32514048 bytes total (14631936 bytes free)
Switch#
Switch#delete /force /recursive flash:/c3560-ipbase-mz.122-35.SE5
Switch#sh flash:                                                 

Directory of flash:/

    2  -rwx        1940   Mar 1 1993 00:02:57 +00:00  private-config.text
    3  -rwx     8811199   Mar 1 1993 02:36:58 +00:00  c3560-ipbasek9-mz.122-46.SE.bin
    4  -rwx          24  Sep 21 2012 16:01:17 +00:00  private-config.text.renamed
    5  -rwx        8459  Sep 21 2012 16:01:17 +00:00  config.text.renamed
    6  -rwx         796   Mar 1 1993 00:02:34 +00:00  vlan.dat.renamed
  465  -rwx        2072   Mar 1 1993 00:02:57 +00:00  multiple-fs
  467  -rwx        2496   Mar 1 1993 00:02:57 +00:00  config.text

32514048 bytes total (23684096 bytes free)


After removed one folder from flash:/ directory, new IOS transferred again. This time, it is quite smooth.


nSwitch#copy tftp: flash:
Address or name of remote host []? 1.1.1.2
Source filename []? c3560-ipbasek9-mz.150-2.SE9.bin
Destination filename [c3560-ipbasek9-mz.150-2.SE9.bin]? 
Accessing tftp://1.1.1.2/c3560-ipbasek9-mz.150-2.SE9.bin...
Loading c3560-ipbasek9-mz.150-2.SE9.bin from 1.1.1.2 (via Vlan1): !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[OK - 16663424 bytes]

16663424 bytes copied in 183.676 secs (90722 bytes/sec)

witch#sh flash: 

Directory of flash:/

    2  -rwx        1940   Mar 1 1993 00:02:57 +00:00  private-config.text
    3  -rwx     8811199   Mar 1 1993 02:36:58 +00:00  c3560-ipbasek9-mz.122-46.SE.bin
    4  -rwx          24  Sep 21 2012 16:01:17 +00:00  private-config.text.renamed
    5  -rwx        8459  Sep 21 2012 16:01:17 +00:00  config.text.renamed
    6  -rwx         796   Mar 1 1993 00:02:34 +00:00  vlan.dat.renamed
    7  -rwx    16663424   Mar 1 1993 00:31:46 +00:00  c3560-ipbasek9-mz.150-2.SE9.bin
  465  -rwx        2072   Mar 1 1993 00:02:57 +00:00  multiple-fs
  467  -rwx        2496   Mar 1 1993 00:02:57 +00:00  config.text

32514048 bytes total (7020032 bytes free)





Rebooting 3560 after uploaded new



Switch#reload
Proceed with reload? [confirm]

*Mar  1 00:37:29.136: %SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload command.
Base ethernet MAC Address: 00:26:ca:45:dc:80
Xmodem file system is available.
The password-recovery mechanism is enabled.
Initializing Flash...
flashfs[0]: 8 files, 1 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 32514048
flashfs[0]: Bytes used: 25494016
flashfs[0]: Bytes available: 7020032
flashfs[0]: flashfs fsck took 10 seconds.
...done Initializing Flash.
Boot Sector Filesystem (bs) installed, fsid: 3
done.
Loading "flash:c3560-ipbasek9-mz.150-2.SE9.bin"...@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
File "flash:c3560-ipbasek9-mz.150-2.SE9.bin" uncompressed and installed, entry point: 0x1000000
executing...

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706



Cisco IOS Software, C3560 Software (C3560-IPBASEK9-M), Version 15.0(2)SE9, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Tue 01-Dec-15 06:56 by prod_rel_team
HULC:Collect Entropy thread starting on box...
Initializing flashfs...

flashfs[2]: 8 files, 1 directories
flashfs[2]: 0 orphaned files, 0 orphaned directories
flashfs[2]: Total bytes: 32514048
flashfs[2]: Bytes used: 25494016
flashfs[2]: Bytes available: 7020032
flashfs[2]: flashfs fsck took 2 seconds.
flashfs[2]: Initialization complete....done Initializing flashfs.


FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found, FIPS Mode Not Enabled

POST: CPU MIC register Tests : Begin
POST: CPU MIC register Tests : End, Status Passed

POST: PortASIC Memory Tests : Begin
POST: PortASIC Memory Tests : End, Status Passed

POST: CPU MIC interface Loopback Tests : Begin
POST: CPU MIC interface Loopback Tests : End, Status Passed

POST: PortASIC RingLoopback Tests : Begin
POST: PortASIC RingLoopback Tests : End, Status Passed

extracting front_end/front_end_ucode_info (172 bytes)
 Software version is different so extracting
 the whole bundle

front_end/ (directory)
extracting front_end/fe_type_1 (34760 bytes)
extracting front_end/fe_type_2 (78520 bytes)
extracting front_end/front_end_ucode_info (172 bytes)
extracting front_end/fe_type_3 (78520 bytes)
extracting front_end/fe_type_4 (78520 bytes)
extracting ucode_info (76 bytes)

Front-end Microcode IMG MGR: Installed 3 image(s) in cache:

Front-end Microcode IMG MGR: found microcode images for 3 devices.
Image for front-end 0: flash:/front_end_ucode_cache/ucode.1
Image for front-end 7: flash:/front_end_ucode_cache/ucode.1
Image for front-end 14: flash:/front_end_ucode_cache/ucode.1

Front-end Microcode IMG MGR: Preparing to program device microcode...
Front-end Microcode IMG MGR: Preparing to program device[0]...28550 bytes.
Front-end Microcode IMG MGR: Programming device 0...rwRrrrrrrwsssspsssspssssps

psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspwRr!
Front-end Microcode IMG MGR: Microcode programming complete for device 0.
Front-end Microcode IMG MGR: Preparing to program device[7]...28550 bytes.
Front-end Microcode IMG MGR: Programming device 7...rwRrrrrrrwsssspsssspss

psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspwRr!
Front-end Microcode IMG MGR: Microcode programming complete for device 7.
Front-end Microcode IMG MGR: Preparing to program device[14]...28550 bytes.
Front-end Microcode IMG MGR: Programming device 14...rwRrrrrrrwsssspsssspssss

psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspss
sspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssssp
sssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspssss
psssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsssspsss
psssspwRr!
Front-end Microcode IMG MGR: Microcode programming complete for device 14.
Front-end Microcode IMG MGR: Microcode programming complete.
POST: PortASIC CAM Subsystem Tests : Begin
POST: PortASIC CAM Subsystem Tests : End, Status Passed

POST: PortASIC Port Loopback Tests : Begin
POST: PortASIC Port Loopback Tests : End, Status Passed

Waiting for Port download...Complete


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
[email protected].

cisco WS-C3560G-48TS (PowerPC405) processor (revision D0) with 131072K bytes of memory.
Processor board ID FOC1332D23F
Last reset from power-on
1 Virtual Ethernet interface
52 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : 00:26:CA:45:DB:80
Motherboard assembly number     : 73-10214-04
Power supply part number        : 341-0107-01
Motherboard serial number       : FOC13322DSE
Power supply serial number      : AZS132709CX
Model revision number           : D0
Motherboard revision number     : D0
Model number                    : WS-C3560G-48TS-S
System serial number            : FOC1332W23F
Top Assembly Part Number        : 800-26849-01
Top Assembly Revision Number    : C0
Version ID                      : V03
CLEI Code Number                : CNMWY00DRC
Hardware Board Revision Number  : 0x09


Switch Ports Model              SW Version            SW Image               
------ ----- -----              ----------            ----------             
*    1 52    WS-C3560G-48TS     15.0(2)SE9            C3560-IPBASEK9-M         



Reference:
3560 tftp errors - what does O mean?

No comments:

Post a Comment