On 5/4/22 21:56, Nehul Patel wrote:
Hi NANOG,
We are seeing some strange behavior on our Juniper MX240 Chassis it is randomly dropping the routes to the certain destination IP address getting the following errors on the MX240 Chassis
If Someone has seen these errors before please suggest how to resolve it
May 4 12:42:00 cr01 newsyslog[44735]: logfile turned over due to size>1024K May 4 12:42:01 /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5 (Invalid) May 4 12:42:01 /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, err 5 (Invalid) May 4 12:42:01 last message repeated 4 times May 4 12:42:01 fpc0 RT: IPv6:0 - 2600:40fc:1011::/48 (add rt entry into jtree failed) May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2028: rt_halp_vectors->rt_create failed May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv6,len 48 prefix 2600:40fc:1011::/48 nh 1048576 May 4 12:42:01 fpc0 RT-HAL,rt_msg_handler,540: route process failed May 4 12:42:01 fpc0 RT: Failed prefix add IPv6 - 2001:67c:20fc::/48 (No memory) on FE 0 May 4 12:42:01 fpc0 RT: IPv6:0 - 2001:67c:20fc::/48 (add rt entry into jtree failed) May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2028: rt_halp_vectors->rt_create failed May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv6,len 48 prefix 2001:67c:20fc::/48 nh 1048576 May 4 12:42:01 fpc0 RT-HAL,rt_msg_handler,540: route process failed May 4 12:42:01 fpc0 RT: Failed prefix add IPv6 - 2606:2800:e004::/48 (No memory) on FE 0 May 4 12:42:01 fpc0 RT: Failed prefix add IPv6 - 2a05:3181:ffff::/48 (No memory) on FE 0 May 4 12:42:01 /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, err 5 (Invalid) May 4 12:42:01 /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5 (Invalid) May 4 12:42:01 /kernel: RT_PFE: RT msg op 2 (PREFIX DELETE) failed, err 5 (Invalid) May 4 12:42:02 fpc0 RT: Failed prefix add IPv4 - 79.120.22/24 (No memory) on FE 0 May 4 12:42:02 fpc0 RT: IPv4:0 - 79.120.22/24 (add rt entry into jtree failed) May 4 12:42:02 fpc0 RT-HAL,rt_entry_add_msg_proc,2028: rt_halp_vectors->rt_create failed May 4 12:42:02 fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv4,len 24 prefix 79.120.22/24 nh 1048583 May 4 12:42:02 /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5 (Invalid) May 4 12:42:02 fpc0 RT-HAL,rt_msg_handler,540: route process failed
May 4 09:33:17 fpc0 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-pages Available:20 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:17 fpc0 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-dwords Available:1280 is less than LWM limit:104857, rsmon_syslog_limit() May 4 09:33:18 fpc0 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-pages Available:19 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:18 fpc0 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-dwords Available:1216 is less than LWM limit:104857, rsmon_syslog_limit() May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-pages Available:16 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-dwords Available:1024 is less than LWM limit:104857, rsmon_syslog_limit() May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM limit:104857, rsmon_syslog_limit() May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-pages Available:19 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:19 fpc1 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-dwords Available:1216 is less than LWM limit:104857, rsmon_syslog_limit() May 4 09:33:19 fpc1 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-pages Available:17 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:19 fpc1 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-dwords Available:1088 is less than LWM limit:104857, rsmon_syslog_limit() May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-pages Available:15 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-dwords Available:960 is less than LWM limit:104857, rsmon_syslog_limit() May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM limit:1638, rsmon_syslog_limit() May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM limit:104857, rsmon_syslog_limit()
Any suggestions will be helpful
Looks like you are out of FIB slots. Would recommend reducing the number of routes you need to send into FIB, or upgrading to newer hardware that has more space. Mark.