Environment
Linux RHEL = 6.2
SFHA/DR = 6.0.2
Query
I installed SFHA 6.0.2 and configured it. This is only one node in the cluster this time. I installed LLT and GAB as well so In future I can add the second node in that cluster. I noticed that the system gets unresponsive for few minutes and responsive back after few minutes. On one Linux Terminal I execute a command tail -f /var/log/messages and keep waiting to become the system unresponsive. I noticed that as the below messages printing in log, the cluster node become unresponsive.
Feb 27 16:11:46 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10541 llt_send_hb: timer not called for 5 secs (5274 ticks). Send out of context hbs to peers from llt_deliver. 174 secs more to go
Feb 27 16:11:46 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 5275 ticks
Feb 27 16:11:49 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1213 ticks
Feb 27 16:11:49 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1850 ticks
Feb 27 16:11:57 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 4246 ticks
Feb 27 16:11:57 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1676 ticks
Feb 27 16:12:02 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 2592 ticks
Feb 27 16:12:07 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10541 llt_send_hb: timer not called for 3 secs (3528 ticks). Send out of context hbs to peers from llt_deliver. 176 secs more to go
Feb 27 16:12:07 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 3529 ticks
Feb 27 16:12:17 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 9895 ticks
Feb 27 16:12:17 CLUSTER-NODE1 kernel: GAB INFO V-15-1-20124 timer not called for 10 seconds
Feb 27 16:12:19 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1833 ticks
Feb 27 16:12:21 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1151 ticks
Feb 27 16:12:25 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1318 ticks
Feb 27 16:12:32 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 2451 ticks
Feb 27 16:12:43 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10541 llt_send_hb: timer not called for 4 secs (4513 ticks). Send out of context hbs to peers from llt_deliver. 175 secs more to go
Feb 27 16:12:43 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 4514 ticks
Feb 27 16:12:45 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1357 ticks
Feb 27 16:12:48 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 2017 ticks
Feb 27 16:12:54 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10541 llt_send_hb: timer not called for 2 secs (2303 ticks). Send out of context hbs to peers from llt_deliver. 177 secs more to go
Feb 27 16:12:54 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 2304 ticks
Feb 27 16:12:55 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1711 ticks
Feb 27 16:13:13 CLUSTER-NODE1 rtkit-daemon[4047]: The canary thread is apparently starving. Taking action.
Feb 27 16:13:13 CLUSTER-NODE1 rtkit-daemon[4047]: Demoting known real-time threads.
Feb 27 16:13:13 CLUSTER-NODE1 rtkit-daemon[4047]: Demoted 0 threads.
Feb 27 16:13:13 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 17538 ticks
Feb 27 16:13:13 CLUSTER-NODE1 kernel: GAB INFO V-15-1-20124 timer not called for 18 seconds
Feb 27 16:13:16 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1499 ticks
Feb 27 16:13:19 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 1912 ticks
Feb 27 16:13:24 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10035 timer not called for 2438 ticks
Feb 27 16:13:35 CLUSTER-NODE1 kernel: LLT INFO V-14-1-10541 llt_send_hb: timer not called for 7 secs (7464 ticks). Send out of context hbs to peers from llt_deliver. 172 secs more to go