Latest

Cisco Catalyst SDWAN C8300 Flash disk full issue

Cisco Catalyst SDWAN C8300 Flash disk full issue

We faced a problem in which we saw a notification indicating that one of the C8300 SDWAN router's Flash discs was full. 

The router bootflash is being clogged with hundreds/thousands of sdavc packs that never appear to be removed.  So far, this has resulted in one of our routers failing owing to a shortage of disc space - the cedge services would not start, and the device was rendered inoperable.  Another router has now reached 100% use on the bootflash as a result of these sdavc packs.

C8300 @ www.thenetworkdna.com
Fig 1.1- Cisco Catalyst SDWAN routers

We attempted to solve the issue and discovered that it is caused by a bug that occurs repeatedly. IOS-XE image 17.6.3 is installed on the SDWAN router C8300.

Lets Verify first, you will see below that " num_of_errs_wrong_batch_id: 2486790" 

Bug Id:CSCwb20082 (cEdge running 17.3.4 SD-AVC protocol packs filling bootflash to 100%)

C8300_NDNA#sh platform resources
**State Acronym: H - Healthy, W - Warning, C - Critical
Resource                 Usage                 Max             Warning         Critical        State
----------------------------------------------------------------------------------------------------
RP0 (ok, active)                                                                               H
 Control Processor       10.10%                100%            80%             90%             H
  DRAM                   4703MB(66%)           7065MB          88%             93%             H
  bootflash              7338MB(100%)          7338MB          70%             90%             C
  harddisk               797MB(6%)             14534MB         90%             95%             H

C8300_NDNA#dir sdavc | count pp_update
Number of lines which match regexp = 11318

C8300_NDNA#show platform hardware qfp active feature nbar client function sci_cpr_non_batch_show_info
Non-graph batch sender info:
        is_initialized: TRUE
        is_batch_in_progress: TRUE
        num_of_batches_created: 16866
        num_of_batches_destroyed: 16865
        num_of_errs_not_initialized: 0
        num_of_errs_invalid_batch_pointer: 0
        num_of_errs_wrong_batch_id: 2486790
        num_of_errs_invalid_batch_state: 0
        num_of_errs_batch_in_progress: 0
        num_of_errs_msg_reset_failed: 0
        num_of_errs_failed_to_create: 0
        num_of_errs_invalid_batch: 0
        num_of_errs_failed_to_show_batch_info: 0
        num_of_errs_failed_to_destroy_batch: 0
        num_of_errs_invalid_msg_buffer: 0
        num_of_errs_failed_to_parse_header: 0
        num_of_errs_failed_to_process_msg: 2486790

Non-graph batch info:
        batch_id: 16866
        batch state: NON_GRAPH_BATCH_STATE_MSG_RECEIVE
        total_num_of_msgs_in_batch: 0
        total_num_of_records_in_batch: 0

This is a bug, which can be resolved by upgrading the router to the newer version. 
The newer version should be 17.9.3 in order to resolve this issue.


Continue Reading...