Есть пара коммутаторов Huawei CE6810-48S4Q-LI работающих в MLAG-паре.
Коммутаторы работают на софте версии V200R002C50SPC800 и установлен патч версии V200R002SPH016.
Текущая версия софта
Текущая версия софта
<sw21>dis version Huawei Versatile Routing Platform Software VRP (R) software, Version 8.150 (CE6810LI V200R002C50SPC800) Copyright (C) 2012-2017 Huawei Technologies Co., Ltd. HUAWEI CE6810-48S4Q-LI uptime is 7 days, 18 hours, 6 minutes Patch Version: V200R002SPH016 CE6810-48S4Q-LI(Master) 1 : uptime is 7 days, 18 hours, 5 minutes StartupTime 2019/09/11 17:15:26+03:00 Memory Size : 2048 M bytes Flash Size : 512 M bytes CE6810-48S4Q-LI version information 1. PCB Version : CEM48S4QP04 VER A 2. MAB Version : 1 3. Board Type : CE6810-48S4Q-LI 4. CPLD1 Version : 101 5. CPLD2 Version : 101 6. BIOS Version : 386 <sw21>dis patch-information Patch Package Name :flash:/CE6810LI-V200R002SPH016.PAT Patch Package Version :V200R002SPH016 Patch Package State :Running Patch Package Run Time:2018-11-03 17:57:22+03:00
Надо обновить коммутаторы до версии софта V200R005C10SPC800 и до патча версии V200R005SPH008.
В документации нет информации о том, как надо обновлять коммутаторы работающие в MLAG-паре.
Но т.к. по большому счету MLAG-пара это два независимо работающий (с оговорками) коммутатора, то обновлять их будем как standalone коммутаторы.
Перед обновлением софта, huawei рекомендует поставить на коммутатор самый свежий патч к текущей версии софта.
Процедура обновления:
Посмотреть последние версии софта, патчей и документации к ним можно по ссылке - https://support.huawei.com/enterprise/en/switches/ce6810-48s4q-li-pid-22460553/software
Софт дают бесплатно, но сначала надо зарегистрироваться в huawei и зарегистрировать сами коммутаторы.
Ничего сложного, регистрация коммутаторов идет по серийному номеру.
На портале поддержки находим нужный коммутатор и идем в раздел софта.
В выпадающем меню “Select Version” выбираем свежую версию, в нашем случае это “V200R005”.
Проваливаемся в нужную версию софта, читаем документацию и выбираем.
Перебираем версии патчей и в имени патча ищем упоминание модели нашего коммутатора - “CE6810LI”.
Читаем документацию и выбираем.
В итоге у нас получается, что сначала, на текущую версию софта, мы должны поставить патч - CE6810LI-V200R002SPH022.PAT.
Потом обновить софт до - CE6810LI-V200R005C10SPC800.cc.
И сразу же поставить патч - CE6810LI-V200R005SPH008.PAT.
Скачиваем с портала нужные файлы.
Обмениваться файлами между серверов и коммутатором будет по ssh\sftp.
Все нужные файлы будут лежать на сервере в директории “/opt/soft/huawei”.
А бэкапить будем в директории “/opt/soft/huawei/bup_sw21” и “/opt/soft/huawei/bup_sw22”.
Процедура коротко.
Выполняем на обоих коммутаторах.
<sw22>dir flash:/ <sw22>system-view [~sw22]sftp -force-receive-pubkey 10.x.x.x -vpn-instance VRF-DAD-1 sftp-client>cd /opt/soft/huawei/bup_sw22 sftp-client>put CE6810LI-V200R002C50SPC800.cc CE6810LI-V200R002C50SPC800.cc sftp-client>put vrpcfg.zip vrpcfg.zip sftp-client>bye [~sw22]quit <sw22>
Процедура подробнее.
Процедура подробнее.
<sw22>dir flash:/ Directory of flash:/ Idx Attr Size(Byte) Date Time FileName 0 dr-x - Sep 19 2019 13:11:35 $_checkpoint 1 dr-x - Aug 22 2017 12:12:27 $_install_mod 2 dr-x - Nov 03 2018 17:44:11 $_license 3 dr-x - Jun 19 2019 00:31:05 $_security_info 4 dr-x - Sep 19 2019 13:10:46 $_system 5 -rw- 128,772,300 Jan 04 2018 13:21:49 CE6810LI-V200R002C50SPC800.cc 6 -rw- 1,339,283 Nov 03 2018 17:45:40 CE6810LI-V200R002SPH016.PAT 7 -rw- 4,206,539 Sep 19 2019 20:36:55 CE6810LI-V200R002SPH022.PAT 8 drwx - Nov 03 2018 17:40:58 POST 9 -rw- 45 Jun 19 2019 20:20:24 SysResTemplate.ini 10 -rw- 97 Nov 03 2018 17:41:02 collect_diag_info.bat 11 -rw- 22,691 Sep 19 2019 13:07:28 device.sys 12 drwx - Sep 19 2019 20:33:17 logfile 13 -rw- 1,933 Sep 19 2019 13:07:28 vrpcfg.zip 14 -rw- 5,914 Jun 05 2019 01:54:02 ztp_20190304170526.log 15 -rw- 13,961 Jun 05 2019 01:35:46 ztp_20190304170526.log.1 16 -rw- 13,961 Jun 05 2019 01:28:16 ztp_20190304170526.log.2 17 -rw- 13,961 Jun 05 2019 01:20:47 ztp_20190304170526.log.3 18 -rw- 13,961 Jun 05 2019 01:13:16 ztp_20190304170526.log.4 269,168 KB total (141,040 KB free) <sw22>system-view Enter system view, return user view with return command. [~sw22]sftp -force-receive-pubkey 10.x.x.x -vpn-instance VRF-DAD-1 Trying 10.x.x.x ... Press CTRL+K to abort Connected to 10.x.x.x ... The server is not authenticated. Continue to access it? [Y/N]:y Save the server's public key? [Y/N]:n Please input the username: Enter password: sftp-client>cd /opt/soft/huawei/bup_sw22 Current directory is: /opt/soft/huawei/bup_sw22 sftp-client>put CE6810LI-V200R002C50SPC800.cc CE6810LI-V200R002C50SPC800.cc Local file: CE6810LI-V200R002C50SPC800.cc ---> Remote file: /opt/soft/huawei/bup_sw22/CE6810LI-V200R002C50SPC800.cc Uploading the file. Please wait...\ Uploading file successfully ended. File upload is completed in 109 seconds. sftp-client>put vrpcfg.zip vrpcfg.zip Local file: vrpcfg.zip ---> Remote file: /opt/soft/huawei/bup_sw22/vrpcfg.zip Uploading the file. Please wait...\ Uploading file successfully ended. File upload is completed in 0 seconds. sftp-client>ls -rw-r--r-- 1 xxx root 1933 Sep 19 13:21 vrpcfg.zip drwxr-xr-x 4 xxx root 4096 Sep 19 13:16 .. -rw-r--r-- 1 xxx root 128772300 Sep 19 13:20 CE6810LI-V200R002C50SPC800-bup.cc drwxr-xr-x 2 xxx root 4096 Sep 19 13:21 . sftp-client>bye Bye [~sw22]quit <sw22>
Процедура коротко.
Выполняем на обоих коммутаторах.
Ниже показан способ подключения к серверу из документации, но мне больше нравится из предыдущего пункта, когда говорим принимать ключ.
<sw22>system-view [~sw22]ssh client first-time enable [*sw22]commit [~sw22]sftp 10.x.x.x -vpn-instance VRF-DAD-1 sftp-client>cd /opt/soft/huawei sftp-client>get CE6810LI-V200R002SPH022.PAT sftp-client>bye [~sw22]quit <sw22>patch load flash:/CE6810LI-V200R002SPH022.PAT all run <sw22>dis patch-information
После применения патчка на первом коммутаторе состояние MLAG (dis dfs-group 1 m-lag) никак не поменялось, все продолжило работать.
Для порядку можно перегрузить коммутаторы.
Процедура подробнее.
Процедура подробнее.
1. Закачиваем на коммутатор патч.
<sw22>system-view [~sw22]ssh client first-time enable [*sw22]commit [~sw22]sftp 10.x.x.x -vpn-instance VRF-DAD-1 Trying 10.x.x.x ... Press CTRL+K to abort Connected to 10.x.x.x ... The server is not authenticated. Continue to access it? [Y/N]:y Save the server's public key? [Y/N]:n Please input the username: Enter password: sftp-client>? Current view commands: bye Quit sftp cd Change remote directory to path cdup Change remote directory to uplevel delete Delete remote files(no more than 10 in one time) dir Display remote directory listing."-a" will only list all files name of the directory."-l" will list not only files name, but also other information. If path argument is not given, list current directory exit Quit sftp get Download file. Default local file name is the same with remote file help "help all" will print all command names. "help [command name]" will print information for specified command ls Display remote directory listing."-a" will only list all files name of the directory."-l" will list not only files name, but also other information. If path argument is not given, list current directory mkdir Create remote directory put Upload file. Default remote file name is the same with local file pwd Display remote working directory quit Quit sftp remove Delete remote files(no more than 10 in one time) rename Rename remote file rmdir Delete remote directories(no more than 10 in one time) sftp-client> sftp-client>cd /opt/soft/huawei Current directory is: /opt/soft/huawei sftp-client>ls drwxr-xr-x 3 xxx root 4096 Sep 19 12:21 .. -rw-r--r-- 1 xxx root 4206539 Sep 19 12:21 CE6810LI-V200R002SPH022.PAT -rw-r--r-- 1 xxx root 135653156 Sep 19 12:22 CE6810LI-V200R005C10SPC800.cc -rw-r--r-- 1 xxx root 2826131 Sep 19 12:21 CE6810LI-V200R005SPH008.PAT drwxr-xr-x 2 xxx root 4096 Sep 19 12:22 . sftp-client> sftp-client>get CE6810LI-V200R002SPH022.PAT Remote file: /opt/soft/huawei/CE6810LI-V200R002SPH022.PAT ---> Local file: CE6810LI-V200R002SPH022.PAT Downloading the file. Please wait...| Downloading file successfully ended. File download is completed in 18 seconds. sftp-client> sftp-client>bye Bye [~sw22]quit <sw22>dir flash:/ Directory of flash:/ Idx Attr Size(Byte) Date Time FileName 0 dr-x - Sep 19 2019 17:30:47 $_checkpoint 1 dr-x - Aug 22 2017 09:12:27 $_install_mod 2 dr-x - Nov 03 2018 14:44:11 $_license 3 dr-x - Jun 18 2019 21:31:05 $_security_info 4 dr-x - Jun 18 2019 21:03:08 $_system 5 -rw- 128,772,300 Jan 04 2018 10:21:49 CE6810LI-V200R002C50SPC800.cc 6 -rw- 1,339,283 Nov 03 2018 14:45:40 CE6810LI-V200R002SPH016.PAT 7 -rw- 4,206,539 Sep 19 2019 17:36:55 CE6810LI-V200R002SPH022.PAT 8 drwx - Nov 03 2018 14:40:58 POST 9 -rw- 45 Jun 19 2019 17:20:24 SysResTemplate.ini 10 -rw- 97 Nov 03 2018 14:41:02 collect_diag_info.bat 11 -rw- 22,691 Sep 18 2019 18:24:53 device.sys 12 drwx - Sep 19 2019 17:33:17 logfile 13 -rw- 1,837 Sep 18 2019 18:24:53 vrpcfg.zip 14 -rw- 5,914 Jun 04 2019 22:54:02 ztp_20190304170526.log 15 -rw- 13,961 Jun 04 2019 22:35:46 ztp_20190304170526.log.1 16 -rw- 13,961 Jun 04 2019 22:28:16 ztp_20190304170526.log.2 17 -rw- 13,961 Jun 04 2019 22:20:47 ztp_20190304170526.log.3 18 -rw- 13,961 Jun 04 2019 22:13:16 ztp_20190304170526.log.4 269,168 KB total (140,568 KB free)
2. На sw21 смотрим состояние MLAG до применения патча на sw22.
<sw21>dis dfs-group 1 m-lag * : Local node Heart beat state : OK Node 1 * Dfs-Group ID : 1 Priority : 150 Address : ip address 10.20.35.141 vpn-instance VRF-DAD-1 State : Master Causation : - System ID : 4857-02cc-aef1 SysName : sw21 Version : V200R002C50SPC800 Device Type : CE6810LI Node 2 Dfs-Group ID : 1 Priority : 120 Address : ip address 10.20.35.142 vpn-instance VRF-DAD-1 State : Backup Causation : - System ID : 4857-02cc-aea1 SysName : sw22 Version : V200R002C50SPC800 Device Type : CE6810LI <sw21>dis dfs-group 1 no <sw21>dis dfs-group 1 node 1 m <sw21>dis dfs-group 1 node 1 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Up Status : active(*)-active <sw21>dis dfs-group 1 node 2 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Up Status : active-active(*)
3. Применяем патч на sw22.
<sw22>patch load ? STRING<5-127> [drive][path][file name] flash: Device name <sw22>patch load flash:/? flash:/../ flash:/./ flash:/CE6810LI-V200R002C50SPC800.cc flash:/CE6810LI-V200R002SPH016.PAT flash:/CE6810LI-V200R002SPH022.PAT flash:/SysResTemplate.ini flash:/collect_diag_info.bat flash:/device.sys flash:/vrpcfg.zip flash:/ztp_20190304170526.log flash:/ztp_20190304170526.log.1 flash:/ztp_20190304170526.log.2 flash:/ztp_20190304170526.log.3 flash:/ztp_20190304170526.log.4 <sw22>patch load flash:/CE6810LI-V200R002SPH022.PAT ? all All the slots <sw22>patch load flash:/CE6810LI-V200R002SPH022.PAT all ? active Activate the patch run Run the patch | Matching output > Redirect the output to a file >> Redirect the output to a file in append mode <cr> <sw22>patch load flash:/CE6810LI-V200R002SPH022.PAT all ru <sw22>patch load flash:/CE6810LI-V200R002SPH022.PAT all run ? | Matching output > Redirect the output to a file >> Redirect the output to a file in append mode <cr> <sw22>patch load flash:/CE6810LI-V200R002SPH022.PAT all run Info: Operating, please wait for a moment..............done. Info: Succeeded in running the patch. <sw22>dis patch-information Patch Package Name :flash:/CE6810LI-V200R002SPH022.PAT Patch Package Version :V200R002SPH022 Patch Package State :Running Patch Package Run Time:2019-09-19 17:42:47 <sw22>
4. Проверяем состонияние MLAG после применения патчка на sw22.
Состояние не изменилось, все продолжило работать.
Что бы сократить время, когда коммутаторы работают на разныйх версиях софта, лучше софт закачать сразу на оба коммутатора.
Процедура коротко.
Выполняем на обоих коммутаторах.
<sw22>system-view [~sw22]sftp -force-receive-pubkey 10.x.x.x -vpn-instance VRF-DAD-1 sftp-client>cd /opt/soft/huawei sftp-client>ls sftp-client>get CE6810LI-V200R005C10SPC800.cc sftp-client>get CE6810LI-V200R005SPH008.PAT sftp-client>bye Bye [~sw22]quit <sw22>dir flash:/
Процедура подробнее.
Процедура подробнее.
[~sw22]sftp -force-receive-pubkey 10.x.x.x -vpn-instance VRF-DAD-1 Trying 10.x.x.x ... Press CTRL+K to abort Connected to 10.x.x.x ... The server is not authenticated. Continue to access it? [Y/N]:y Save the server's public key? [Y/N]:n Please input the username: Enter password: sftp-client>cd /opt/soft/huawei Current directory is: /opt/soft/huawei sftp-client>ls drwxr-xr-x 2 xxx root 4096 Sep 19 13:21 bup_sw21 drwxr-xr-x 3 xxx root 4096 Sep 19 12:21 .. -rw-r--r-- 1 xxx root 4206539 Sep 19 12:21 CE6810LI-V200R002SPH022.PAT -rw-r--r-- 1 xxx root 135653156 Sep 19 12:22 CE6810LI-V200R005C10SPC800.cc -rw-r--r-- 1 xxx root 2826131 Sep 19 12:21 CE6810LI-V200R005SPH008.PAT drwxr-xr-x 2 xxx root 4096 Sep 19 13:16 bup_sw22 drwxr-xr-x 4 xxx root 4096 Sep 19 13:16 . sftp-client>get CE6810LI-V200R005C10SPC800.cc Remote file: /opt/soft/huawei/CE6810LI-V200R005C10SPC800.cc ---> Local file: CE6810LI-V200R005C10SPC800.cc Downloading the file. Please wait...\ Downloading file successfully ended. File download is completed in 130 seconds. sftp-client>get CE6810LI-V200R005SPH008.PAT Remote file: /opt/soft/huawei/CE6810LI-V200R005SPH008.PAT ---> Local file: CE6810LI-V200R005SPH008.PAT Downloading the file. Please wait...| Downloading file successfully ended. File download is completed in 13 seconds. sftp-client>bye Bye [~sw22]quit <sw22>dir fl <sw22>dir flash:/ Directory of flash:/ Idx Attr Size(Byte) Date Time FileName 0 dr-x - Sep 19 2019 13:11:35 $_checkpoint 1 dr-x - Aug 22 2017 12:12:27 $_install_mod 2 dr-x - Nov 03 2018 17:44:11 $_license 3 dr-x - Jun 19 2019 00:31:05 $_security_info 4 dr-x - Sep 19 2019 13:10:46 $_system 5 -rw- 128,772,300 Jan 04 2018 13:21:49 CE6810LI-V200R002C50SPC800.cc 6 -rw- 1,339,283 Nov 03 2018 17:45:40 CE6810LI-V200R002SPH016.PAT 7 -rw- 4,206,539 Sep 19 2019 20:36:55 CE6810LI-V200R002SPH022.PAT 8 -rw- 135,653,156 Sep 19 2019 13:24:34 CE6810LI-V200R005C10SPC800.cc 9 -rw- 2,826,131 Sep 19 2019 13:27:52 CE6810LI-V200R005SPH008.PAT 10 drwx - Nov 03 2018 17:40:58 POST 11 -rw- 45 Jun 19 2019 20:20:24 SysResTemplate.ini 12 -rw- 97 Nov 03 2018 17:41:02 collect_diag_info.bat 13 -rw- 22,691 Sep 19 2019 13:07:28 device.sys 14 drwx - Sep 19 2019 20:33:17 logfile 15 -rw- 1,933 Sep 19 2019 13:07:28 vrpcfg.zip 16 -rw- 5,914 Jun 05 2019 01:54:02 ztp_20190304170526.log 17 -rw- 13,961 Jun 05 2019 01:35:46 ztp_20190304170526.log.1 18 -rw- 13,961 Jun 05 2019 01:28:16 ztp_20190304170526.log.2 19 -rw- 13,961 Jun 05 2019 01:20:47 ztp_20190304170526.log.3 20 -rw- 13,961 Jun 05 2019 01:13:16 ztp_20190304170526.log.4 269,168 KB total (10,744 KB free) <sw22>
На флешке осталось всего 10 Мбайт.
У коммутаторов ce6810LI только одна загрузочная область.
<sw21>dis startup slot ? <1> The available slot <sw21>dis startup slot 1 MainBoard: Configured startup system software: flash:/CE6810LI-V200R005C10SPC800.cc Startup system software: flash:/CE6810LI-V200R005C10SPC800.cc Next startup system software: flash:/CE6810LI-V200R005C10SPC800.cc Startup saved-configuration file: flash:/vrpcfg.zip Next startup saved-configuration file: flash:/vrpcfg.zip Startup paf file: default Next startup paf file: default Startup patch package: flash:/CE6810LI-V200R005SPH008.PAT Next startup patch package: flash:/CE6810LI-V200R005SPH008.PAT <sw21>
Поэтому смысл обновления заключается в том, что бы указать коммутатору с какой версии софта и с каким патчем ему нужно грузиться (из единственной загрузочной обласли) в следующий раз и перезагрузить коммутатор.
Когда обновится слейв коммутатор, MLAG-пара продолжит нормально работать.
Единственное, что коммутатор будет в выводе “dis dfs-group 1 m-lag” явным образом сообщать о том, что версии софта на нодах разные.
Информации о том, сколько коммутаторы могут проработать в таком состоянии не нашел, но лучше не затягивать.
Mellanox в таком состоянии будет работать в течении часа, а потом положит MLAG порты.
Так же, в версии софта V200R005, увеличилось дефолтное значение MLAG таймера “up-delay” - c 120 до 240 секунд.
Т.е. после того как коммутатор загрузится, MLAG порты еще 4 минуты будут в дауне.
Процедура коротко.
<sw22>startup system-software flash:/CE6810LI-V200R005C10SPC800.cc <sw22>startup patch flash:/CE6810LI-V200R005SPH008.PAT all <sw22>dis startup <sw22>reboot
Процедура подробнее.
Процедура подробнее.
1. Задаем версию софта с которой надо загружаться.
<sw22>startup system-software ? STRING<5-127> [drive][path][file name] flash: Device name <sw22>startup system-software flash:/? flash:/../ flash:/./ flash:/CE6810LI-V200R002C50SPC800.cc flash:/CE6810LI-V200R002SPH022.PAT flash:/CE6810LI-V200R005C10SPC800.cc flash:/CE6810LI-V200R005SPH008.PAT flash:/SysResTemplate.ini flash:/collect_diag_info.bat flash:/device.sys flash:/vrpcfg.zip flash:/ztp_20190304170526.log flash:/ztp_20190304170526.log.1 flash:/ztp_20190304170526.log.2 flash:/ztp_20190304170526.log.3 flash:/ztp_20190304170526.log.4 <sw22>startup system-software flash:/CE6810LI-V200R005C10SPC800.cc Info: Operating, please wait for a moment............................................done. Info: Succeeded in setting the next startup software in slot 1. <sw22>startup patch flash:/CE6810LI-V200R005SPH008.PAT ? all All the slots <sw22>startup patch flash:/CE6810LI-V200R005SPH008.PAT all ? | Matching output > Redirect the output to a file >> Redirect the output to a file in append mode <cr> <sw22>startup patch flash:/CE6810LI-V200R005SPH008.PAT all Info: Operating, please wait for a moment...........done. Info: Succeeded in setting startup the patch. <sw22>dis startup MainBoard: Configured startup system software: flash:/CE6810LI-V200R002C50SPC800.cc Startup system software: flash:/CE6810LI-V200R002C50SPC800.cc Next startup system software: flash:/CE6810LI-V200R005C10SPC800.cc Startup saved-configuration file: flash:/vrpcfg.zip Next startup saved-configuration file: flash:/vrpcfg.zip Startup paf file: default Next startup paf file: default Startup patch package: flash:/CE6810LI-V200R002SPH022.PAT Next startup patch package: flash:/CE6810LI-V200R005SPH008.PAT
2. Перегружаем слейв коммутатор.
<sw22>reboot slot 1: Next startup system software: flash:/CE6810LI-V200R005C10SPC800.cc Next startup saved-configuration file: flash:/vrpcfg.zip Next startup paf file: default Next startup patch package: flash:/CE6810LI-V200R005SPH008.PAT Warning: The system will reboot. Continue? [Y/N]:y Sep 19 2019 13:46:00+03:00 sw22 %%01SYSTEM/4/SYS_RELOAD_ACTION(l):CID=0x80cc000d;The board was reboot. (ReloadIndex=257, ReloadImage=2, ReloadCfg=83, ReloadReason=, ReloadTime=0000-00-00 00:00:00, ReloadAction=3, ReloadPaf=0, ReloadLicense=0, ReloadPatch=2). #11/active/linkDown/Major/occurredTime:2019-09-19 13:46:02+03:00/-/-/alarmID:0x08520003/CID=0x807a0404:The interface status changes. (ifName=Eth-Trunk16, AdminStatus=UP, OperStatus=DOWN, Reason=The conditions for the activation of the interface are not met, mainIfname=Eth-Trunk16) #12/active/linkDown/Major/occurredTime:2019-09-19 13:46:02+03:00/-/-/alarmID:0x08520003/CID=0x807a0404:The interface status changes. (ifName=10GE1/0/16, AdminStatus=UP, OperStatus=DOWN, Reason=Interface physical link is down, mainIfname=Eth-Trunk16) #13/active/hwLacpNegotiateFailed/Major/occurredTime:2019-09-19 13:46:02+03:00/-/-/alarmID:0x09360000/CID=0x807a0404:The member of LAG negotiation failed. (TrunkIndex=2, PortIfIndex=20, TrunkId=16, TrunkName=Eth-Trunk16, PortName=10GE1/0/16, Reason=A link fault occurred or negotiation information synchronization failed.) Sep 19 2019 13:46:03+03:00 sw22 %%01LACP/4/LACP_STATE_DOWN(l):CID=0x80480432;The LACP state is down. (PortName=10GE1/0/16, TrunkName=Eth-Trunk16, LastReceivePacketTime=[2019-09-19 13:45:52:225+03:00], Reason=The interface went down physically or flapped to down. Please check the interface's status, duplex mode, bandwidth, and so on.) #14/active/hwLocalFaultAlarm/Minor/occurredTime:2019-09-19 13:46:03+03:00/-/-/alarmID:0x081320c6/CID=0x80fc0101:The local fault alarm has occurred. (IfIndex=55, IfName=40GE1/0/3) Sep 19 2019 13:46:02+03:00 sw22 %%01IFNET/2/linkDown_active(l):CID=0x807a0404-alarmID=0x08520003;The interface status changes. (ifName=Eth-Trunk16, AdminStatus=UP, OperStatus=DOWN, Reason=The conditions for the activation of the interface are not met, mainIfname=Eth-Trunk16) Sep 19 2019 13:46:02+03:00 sw22 %%01IFNET/2/linkDown_active(l):CID=0x807a0404-alarmID=0x08520003;The interface status changes. (ifName=10GE1/0/16, AdminStatus=UP, Op ************************************************************* * Copyright (C) <2012-2018> Huawei Technologies Co., Ltd. * ************************************************************* CloudEngine BIOS Version: 433 (Oct 11 2018 - 20:19:17) Board Name ..................................... CE6810-48S4Q-LI SDRAM Memory Initializing ...................... BEGIN SDRAM Memory Size .............................. 2 Gbytes SDRAM Memory Initializing ...................... DONE Press CTRL+T for full memory test .............. skip Starting ... Press CTRL+B to enter BIOS menu or CTRL+E to boot DFX: 0 Boot Disk: flash Startup File: CE6810LI-V200R005C10SPC800.cc Try Times: 1 Check system software ........................................ DONE Check APPDB .................................................. DONE Set next bootfile ............................................ DONE Load system software ......................................... DONE Create file system ........................................... DONE Now starting VRP ........ Configured switch mode ....................................... AUTO-NEGOTIATION Current switch mode .......................................... STACK Press CTRL+Y to modify the switch mode ....................... skip Initializing stack ........................................... DONE Stack member ID .............................................. 1 Stack domain ID .............................................. INVALID Stack priority ............................................... 100 Default MAC .................................................. 48-57-02-cc-ae-a0 Competing with other devices in the stack .................... DONE Stack role ................................................... MASTER Stack role ................................................... MASTER User interface con0 is available Please Press ENTER. Password: Info: The max number of VTY users is 5, the number of current VTY users online is 0, and total number of terminal users online is 1. The current login time is 2019-09-19 13:51:14+03:00. <sw22> <sw22>dis current-configuration Error: The system is busy in building configuration. Please wait for a moment...
3. Проверяем версию и состояние MLAG.
DAD все хорошо, PeerLink поднялись, роли коммутаторов распределились правильно, видим mismatch в версиях софта.
Трафик при этом ходит.
<sw22>dis version Huawei Versatile Routing Platform Software VRP (R) software, Version 8.180 (CE6810LI V200R005C10SPC800) Copyright (C) 2012-2018 Huawei Technologies Co., Ltd. HUAWEI CE6810-48S4Q-LI uptime is 0 day, 0 hour, 3 minutes Patch Version: V200R005SPH008 CE6810-48S4Q-LI(Master) 1 : uptime is 0 day, 0 hour, 1 minutes StartupTime 2019/09/19 13:49:39+03:00 Memory Size : 2048 M bytes Flash Size : 512 M bytes CE6810-48S4Q-LI version information 1. PCB Version : CEM48S4QP04 VER A 2. MAB Version : 1 3. Board Type : CE6810-48S4Q-LI 4. CPLD1 Version : 101 5. CPLD2 Version : 101 6. BIOS Version : 433 <sw22>dis patch-information Patch Package Name :flash:/CE6810LI-V200R005SPH008.PAT Patch Package Version :V200R005SPH008 Patch Package State :Running Patch Package Run Time:2019-09-19 13:48:00+03:00 <sw22> <sw22>dis dfs-group 1 Dfs-Group ID : 1 Priority : 120 UDP Port : - Address : ip address 10.20.35.142 vpn-instance VRF-DAD-1 System ID : 4857-02cc-aea1 SysName : sw22 Up-delay/Interval : 240/0 Switch-delay : 0 <sw22>dis dfs-group 1 m-lag * : Local node Heart beat state : OK Node 2 * Dfs-Group ID : 1 Priority : 120 Address : ip address 10.20.35.142 vpn-instance VRF-DAD-1 State : Backup Causation : - System ID : 4857-02cc-aea1 SysName : sw22 Version : V200R005C10SPC800(mismatch) Device Type : CE6810LI Node 1 Dfs-Group ID : 1 Priority : 150 Address : ip address 10.20.35.141 vpn-instance VRF-DAD-1 State : Master Causation : - System ID : 4857-02cc-aef1 SysName : sw21 Version : V200R002C50SPC800(mismatch) Device Type : CE6810LI <sw22> <sw22>dis dfs-group 1 node 1 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Up Status : active-inactive(*) Member Port Role : Master-Invalid(*) <sw22>dis dfs-group 1 node 2 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Down Status : inactive(*)-active Member Port Role : Invalid(*)-Master <sw22>dis dfs-group 1 Dfs-Group ID : 1 Priority : 120 UDP Port : - Address : ip address 10.20.35.142 vpn-instance VRF-DAD-1 System ID : 4857-02cc-aea1 SysName : sw22 Up-delay/Interval : 240/0 Switch-delay : 0 Configuration consistency check: --
Последовательность действий целом такая же.
Процедура коротко.
<sw21>startup system-software flash:/CE6810LI-V200R005C10SPC800.cc <sw21>startup patch flash:/CE6810LI-V200R005SPH008.PAT all <sw21>dis startup <sw21>reboot
Процедура подробнее.
Процедура подробнее.
1. Задаем версию софта с которой надо загружаться.
<sw21>dir flash:/ Directory of flash:/ Idx Attr Size(Byte) Date Time FileName 0 dr-x - Sep 19 2019 13:04:34 $_checkpoint 1 dr-x - Aug 22 2017 12:12:27 $_install_mod 2 dr-x - Nov 03 2018 17:55:23 $_license 3 dr-x - Jun 14 2019 23:30:32 $_security_info 4 dr-x - Sep 19 2019 13:03:45 $_system 5 -rw- 128,772,300 Jan 04 2018 13:21:49 CE6810LI-V200R002C50SPC800.cc 6 -rw- 1,339,283 Nov 03 2018 17:57:13 CE6810LI-V200R002SPH016.PAT 7 -rw- 4,206,539 Sep 19 2019 12:50:14 CE6810LI-V200R002SPH022.PAT 8 -rw- 135,653,156 Sep 19 2019 13:35:20 CE6810LI-V200R005C10SPC800.cc 9 -rw- 2,826,131 Sep 19 2019 13:36:08 CE6810LI-V200R005SPH008.PAT 10 drwx - Nov 03 2018 17:52:09 POST 11 -rw- 45 Sep 11 2019 23:29:44 SysResTemplate.ini 12 -rw- 97 Nov 03 2018 17:52:12 collect_diag_info.bat 13 -rw- 22,691 Sep 19 2019 13:00:35 device.sys 14 drwx - Sep 19 2019 09:30:09 logfile 15 -rw- 2,150 Sep 19 2019 13:00:35 vrpcfg.zip 16 -rw- 5,914 Mar 04 2019 20:00:29 ztp_20190304165135.log 17 -rw- 13,961 Mar 04 2019 19:58:59 ztp_20190304165135.log.1 18 -rw- 824 Mar 04 2019 19:54:19 ztp_20190304165135.log.2 269,168 KB total (11,080 KB free) <sw21>startup system-software flash:/CE6810LI-V200R005C10SPC800.cc ? all For all boards slave-board For the slave control board slot Specify the slot ID | Matching output > Redirect the output to a file >> Redirect the output to a file in append mode <cr> <sw21>startup system-software flash:/CE6810LI-V200R005C10SPC800.cc Info: Operating, please wait for a moment..............................................done. Info: Succeeded in setting the next startup software in slot 1. <sw21>startup patch flash:/CE6810LI-V200R005SPH008.PAT ? all All the slots <sw21>startup patch flash:/CE6810LI-V200R005SPH008.PAT all Info: Operating, please wait for a moment...........done. Info: Succeeded in setting startup the patch. <sw21>dis startup MainBoard: Configured startup system software: flash:/CE6810LI-V200R002C50SPC800.cc Startup system software: flash:/CE6810LI-V200R002C50SPC800.cc Next startup system software: flash:/CE6810LI-V200R005C10SPC800.cc Startup saved-configuration file: flash:/vrpcfg.zip Next startup saved-configuration file: flash:/vrpcfg.zip Startup paf file: default Next startup paf file: default Startup patch package: flash:/CE6810LI-V200R002SPH022.PAT Next startup patch package: flash:/CE6810LI-V200R005SPH008.PAT
2. Перегружаем мастер коммутатор.
Сразу после того, как sw21 начал перегружаться, ушел mismatch на sw22.
<sw22>dis dfs-group 1 m-lag * : Local node Heart beat state : Lost Node 1 * Dfs-Group ID : 1 Priority : 120 Address : ip address 10.20.35.142 vpn-instance VRF-DAD-1 State : Master Causation : PEERLINKDOWN System ID : 4857-02cc-aea1 SysName : sw22 Version : V200R005C10SPC800 Device Type : CE6810LI <sw22>
Процедура перезагрузки.
<sw21>reboot slot 1: Next startup system software: flash:/CE6810LI-V200R005C10SPC800.cc Next startup saved-configuration file: flash:/vrpcfg.zip Next startup paf file: default Next startup patch package: flash:/CE6810LI-V200R005SPH008.PAT Warning: The system will reboot. Continue? [Y/N]:y После ребута sw21 на sw22 мисматч ушел <sw22>dis dfs-group 1 m-lag * : Local node Heart beat state : Lost Node 1 * Dfs-Group ID : 1 Priority : 120 Address : ip address 10.20.35.142 vpn-instance VRF-DAD-1 State : Master Causation : PEERLINKDOWN System ID : 4857-02cc-aea1 SysName : sw22 Version : V200R005C10SPC800 Device Type : CE6810LI <sw22> Warning: The system will reboot. Continue? [Y/N]:y Sep 19 2019 14:01:52+03:00 sw21 %%01SYSTEM/4/SYS_RELOAD_ACTION(l):CID=0x80cc000d;The board was reboot. (ReloadIndex=257, ReloadImage=2, ReloadCfg=14, ReloadReason=, ReloadTime=0000-00-00 00:00:00, ReloadAction=3, ReloadPaf=0, ReloadLicense=0, ReloadPatch=3). #30/active/hwLocalFaultAlarm/Minor/occurredTime:2019-09-19 14:01:54+03:00/-/-/alarmID:0x081320c6/CID=0x80fc0101:The local fault alarm has occurred. (IfIndex=55, IfName=40GE1/0/3) #31/active/hwRemoteFaultAlarm/Minor/occurredTime:2019-09-19 14:01:54+03:00/-/-/alarmID:0x081320c7/CID=0x80fc0101:The remote fault alarm has occurred. (IfIndex=55, IfName=40GE1/0/3) #32/active/hwLocalFaultAlarm/Minor/occurredTime:2019-09-19 14:01:54+03:00/-/-/alarmID:0x081320c6/CID=0x80fc0101:The local fault alarm has occurred. (IfIndex=56, IfName=40GE1/0/4) #33/active/hwRemoteFaultAlarm/Minor/occurredTime:2019-09-19 14:01:54+03:00/-/-/alarmID:0x081320c7/CID=0x80fc0101:The remote fault alarm has occurred. (IfIndex=56, IfName=40GE1/0/4) #34/active/linkDown/Major/occurredTime:2019-09-19 14:01:54+03:00/-/-/alarmID:0x08520003/CID=0x807a0404:The interface status changes. (ifName=Eth-Trunk16, AdminStatus=UP, OperStatus=DOWN, Reason=The conditions for the activation of the interface are not met, mainIfname=Eth-Trunk16) #35/active/linkDown/Major/occurredTime:2019-09-19 14:01:54+03:00/-/-/alarmID:0x08520003/CID=0x807a0404:The interface status changes. (ifName=10GE1/0/16, AdminStatus=UP, OperStatus=DOWN, Reason=Interface physical link is down, mainIfname=Eth-Trunk16) #36/active/linkDown/Major/occurredTime:2019-09-19 14:01:54+03:00/-/-/alarmID:0x08520003/CID=0x807a0404:The interface status changes. (ifName=40GE1/0/3, AdminStatus=UP, OperStatus=DOWN, R ************************************************************* * Copyright (C) <2012-2018> Huawei Technologies Co., Ltd. * ************************************************************* CloudEngine BIOS Version: 433 (Oct 11 2018 - 20:19:17) Board Name ..................................... CE6810-48S4Q-LI SDRAM Memory Initializing ...................... BEGIN SDRAM Memory Size .............................. 2 Gbytes SDRAM Memory Initializing ...................... DONE Press CTRL+T for full memory test .............. skip Starting ... Press CTRL+B to enter BIOS menu or CTRL+E to boot DFX: 0 Boot Disk: flash Startup File: CE6810LI-V200R005C10SPC800.cc Try Times: 1 Check system software ........................................ DONE Check APPDB .................................................. DONE Set next bootfile ............................................ DONE Load system software ......................................... DONE Create file system ........................................... DONE Now starting VRP ........ Configured switch mode ....................................... AUTO-NEGOTIATION Current switch mode .......................................... STACK Press CTRL+Y to modify the switch mode ....................... skip Initializing stack ........................................... DONE Stack member ID .............................................. 1 Stack domain ID .............................................. INVALID Stack priority ............................................... 100 Default MAC .................................................. 48-57-02-cc-ae-f0 Competing with other devices in the stack .................... DONE Stack role ................................................... MASTER User interface con0 is available Please Press ENTER. Password: Info: The max number of VTY users is 5, the number of current VTY users online is 0, and total number of terminal users online is 1. The current login time is 2019-09-19 14:07:03+03:00.
3. Момент когда поднимаются PL порты.
После того, как подялись PL, все заработало.
Mismatch ушел.
<sw21>dis dfs-group 1 Dfs-Group ID : 1 Priority : 150 UDP Port : - Address : ip address 10.20.35.141 vpn-instance VRF-DAD-1 System ID : 4857-02cc-aef1 SysName : sw21 Up-delay/Interval : 240/0 Switch-delay : 0 Configuration consistency check: -- <sw21>dis dfs-group 1 m-lag * : Local node Heart beat state : Lost Node 1 * Dfs-Group ID : 1 Priority : 150 Address : ip address 10.20.35.141 vpn-instance VRF-DAD-1 State : Master Causation : PEERLINKDOWN System ID : 4857-02cc-aef1 SysName : sw21 Version : V200R005C10SPC800 Device Type : CE6810LI <sw21>dis dfs-group 1 m-lag * : Local node Heart beat state : Lost Node 1 * Dfs-Group ID : 1 Priority : 150 Address : ip address 10.20.35.141 vpn-instance VRF-DAD-1 State : Master Causation : PEERLINKDOWN System ID : 4857-02cc-aef1 SysName : sw21 Version : V200R005C10SPC800 Device Type : CE6810LI <sw21>dis dfs-group 1 m-lag * : Local node Heart beat state : Lost Node 1 * Dfs-Group ID : 1 Priority : 150 Address : ip address 10.20.35.141 vpn-instance VRF-DAD-1 State : Master Causation : TIMEOUT System ID : 4857-02cc-aef1 SysName : sw21 Version : V200R005C10SPC800 Device Type : CE6810LI <sw21>dis dfs-group 1 m-lag * : Local node Heart beat state : OK Node 1 * Dfs-Group ID : 1 Priority : 150 Address : ip address 10.20.35.141 vpn-instance VRF-DAD-1 State : Master Causation : - System ID : 4857-02cc-aef1 SysName : sw21 Version : V200R005C10SPC800 Device Type : CE6810LI Node 2 Dfs-Group ID : 1 Priority : 120 Address : ip address 10.20.35.142 vpn-instance VRF-DAD-1 State : Backup Causation : - System ID : 4857-02cc-aea1 SysName : sw22 Version : V200R005C10SPC800 Device Type : CE6810LI <sw21>
Версия софтра на sw21.
<sw21>dis version Huawei Versatile Routing Platform Software VRP (R) software, Version 8.180 (CE6810LI V200R005C10SPC800) Copyright (C) 2012-2018 Huawei Technologies Co., Ltd. HUAWEI CE6810-48S4Q-LI uptime is 0 day, 0 hour, 5 minutes Patch Version: V200R005SPH008 CE6810-48S4Q-LI(Master) 1 : uptime is 0 day, 0 hour, 4 minutes StartupTime 2019/09/19 14:05:34+03:00 Memory Size : 2048 M bytes Flash Size : 512 M bytes CE6810-48S4Q-LI version information 1. PCB Version : CEM48S4QP04 VER A 2. MAB Version : 1 3. Board Type : CE6810-48S4Q-LI 4. CPLD1 Version : 101 5. CPLD2 Version : 101 6. BIOS Version : 433 <sw21>dis patch-information Patch Package Name :flash:/CE6810LI-V200R005SPH008.PAT Patch Package Version :V200R005SPH008 Patch Package State :Running Patch Package Run Time:2019-09-19 14:03:42+03:00
Через 4 минуты поднялись MLAG порты на sw21 и все заработало в штатном режиме.
<sw21>dis dfs-group 1 node 1 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Down Status : inactive(*)-active Member Port Role : Invalid(*)-Master <sw21>dis dfs-group 1 node 2 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Up Status : active-inactive(*) Member Port Role : Master-Invalid(*) <sw21> <sw21>dis dfs-group 1 node 1 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Up Status : active(*)-active Member Port Role : Backup(*)-Master <sw21>dis dfs-group 1 node 2 m-lag * - Local node M-Lag ID : 16 Interface : Eth-Trunk 16 Port State : Up Status : active-active(*) Member Port Role : Master-Backup(*)
В выводе появилось новое поле “Member Port Role”.
Распределиение ролей в не соответствет заданным приоритетам, все наоборот. С этим надо будет еще разобраться.
Все, на этом обновление закончено.