I can not solve the problem, falls off storage from ESXi. =(((
Infrastructure:
1. iSCSI Storage: "P2000G3 iSCSI"
2. The array is connected to the Switch "HP A5820X-24XG-SFP+" on SFP+ direct attach cable.
3. Server (Supermicro) witch "ESXi 5.0 update 1" witch ethernet adapter "QLogic QLE3242 10Gb" dual port is connected to the "HP A5820X-24XG-SFP+" on SFP+ direct attach cable.
The logs are messages ESXi:
2012-09-20T00:21:18.165Z cpu4:4100)ScsiDeviceIO: 1198: Device naa.600c0ff00011347d9b20f74f01000000 performance has improved. I/O latency reduced from 1222642 microseconds to 243409 microseconds.
2012-09-20T00:21:22.728Z cpu16:4112)ScsiDeviceIO: 1198: Device naa.600c0ff00011347d9b20f74f01000000 performance has improved. I/O latency reduced from 243409 microseconds to 55525 microseconds.
2012-09-20T00:22:10.827Z cpu0:4147)<6>qlcnic 0000:04:00.1: Freed vmnic5[1] irq.
2012-09-20T00:22:10.953Z cpu1:4147)<6>qlcnic 0000:04:00.0: Freed vmnic4[1] irq.
2012-09-20T00:40:06.493Z cpu19:6969)NetPort: 1427: disabled port 0x300000a
2012-09-20T00:40:15.756Z cpu3:4147)<6>qlcnic 0000:04:00.0: Feature 0x1 needed.
2012-09-20T00:40:16.152Z cpu11:4147)IDT: 991: 0x21 <vmnic4[1]> exclusive (entropy source), flags 0x10
2012-09-20T00:40:16.152Z cpu11:4147)VMK_VECTOR: 137: Added handler for vector 33, flags 0x10
2012-09-20T00:40:16.152Z cpu11:4147)<6>qlcnic 0000:04:00.1: Feature 0x1 needed.
2012-09-20T00:40:16.507Z cpu1:4147)IDT: 991: 0x61 <vmnic5[1]> exclusive (entropy source), flags 0x10
2012-09-20T00:40:16.507Z cpu1:4147)VMK_VECTOR: 137: Added handler for vector 97, flags 0x10
2012-09-20T00:40:31.827Z cpu20:7372)WARNING: ScsiDeviceIO: 1218: Device naa.600c0ff00011347d9b20f74f01000000 performance has deteriorated. I/O latency increased from average value of 28334 microseconds to 591034 microseconds.
2012-09-20T00:40:33.430Z cpu18:4114)ScsiDeviceIO: 1198: Device naa.600c0ff00011347d9b20f74f01000000 performance has improved. I/O latency reduced from 591034 microseconds to 114578 microseconds.
2012-09-20T00:40:34.508Z cpu7:4103)ScsiDeviceIO: 1198: Device naa.600c0ff00011347d9b20f74f01000000 performance has improved. I/O latency reduced from 114578 microseconds to 55214 microseconds.
2012-09-20T00:40:56.467Z cpu23:6970)NetPort: 2602: resuming traffic on DV port 404
2012-09-20T00:40:56.467Z cpu23:6970)NetPort: 1240: enabled port 0x300000a with mac 00:50:56:b2:29:73
2012-09-20T00:41:09.433Z cpu8:7162)WARNING: ScsiDeviceIO: 1218: Device naa.600c0ff00011347d9b20f74f01000000 performance has deteriorated. I/O latency increased from average value of 28343 microseconds to 582066 microseconds.
2012-09-20T00:41:10.912Z cpu21:4117)WARNING: ScsiDeviceIO: 1218: Device naa.600c0ff00011347d9b20f74f01000000 performance has deteriorated. I/O latency increased from average value of 28354 microseconds to 1179381 microseconds.
2012-09-20T00:41:16.469Z cpu9:4105)ScsiDeviceIO: 1198: Device naa.600c0ff00011347d9b20f74f01000000 performance has improved. I/O latency reduced from 1179381 microseconds to 231284 microseconds.
2012-09-20T00:41:17.827Z cpu23:4119)WARNING: ScsiDeviceIO: 1218: Device naa.600c0ff00011347d9b20f74f01000000 performance has deteriorated. I/O latency increased from average value of 28411 microseconds to 583574 microseconds.
2012-09-20T00:41:20.758Z cpu2:7161)ScsiDeviceIO: 1198: Device naa.600c0ff00011347d9b20f74f01000000 performance has improved. I/O latency reduced from 583574 microseconds to 115807 microseconds.
2012-09-20T00:42:15.818Z cpu0:4120)ScsiDeviceIO: 1198: Device naa.600c0ff00011347d9b20f74f01000000 performance has improved. I/O latency reduced from 115807 microseconds to 55092 microseconds.
2012-09-20T00:42:15.837Z cpu5:4147)<6>qlcnic 0000:04:00.0: Freed vmnic4[1] irq.
2012-09-20T00:42:16.190Z cpu3:4147)<6>qlcnic 0000:04:00.1: Freed vmnic5[1] irq.
2012-09-20T01:10:06.581Z cpu19:6967)NetPort: 1427: disabled port 0x300000a
2012-09-20T01:10:15.745Z cpu9:4147)<6>qlcnic 0000:04:00.0: Feature 0x1 needed.
2012-09-20T01:10:16.228Z cpu4:4147)IDT: 991: 0x21 <vmnic4[1]> exclusive (entropy source), flags 0x10
2012-09-20T01:10:16.228Z cpu4:4147)VMK_VECTOR: 137: Added handler for vector 33, flags 0x10
2012-09-20T01:10:16.228Z cpu4:4147)<6>qlcnic 0000:04:00.1: Feature 0x1 needed.
2012-09-20T01:10:16.562Z cpu1:4147)IDT: 991: 0x61 <vmnic5[1]> exclusive (entropy source), flags 0x10
2012-09-20T01:10:16.562Z cpu1:4147)VMK_VECTOR: 137: Added handler for vector 97, flags 0x10
2012-09-20T01:10:56.233Z cpu22:6969)NetPort: 2602: resuming traffic on DV port 404
2012-09-20T01:10:56.233Z cpu22:6969)NetPort: 1240: enabled port 0x300000a with mac 00:50:56:b2:29:73
2012-09-20T01:12:11.246Z cpu8:4147)<6>qlcnic 0000:04:00.1: Freed vmnic5[1] irq.
2012-09-20T01:12:11.410Z cpu6:4147)<6>qlcnic 0000:04:00.0: Freed vmnic4[1] irq.
2012-09-20T01:20:06.987Z cpu14:6970)NetPort: 1427: disabled port 0x300000a
2012-09-20T01:20:10.741Z cpu1:4147)<6>qlcnic 0000:04:00.0: Feature 0x1 needed.
2012-09-20T01:20:10.927Z cpu0:4147)IDT: 991: 0x21 <vmnic4[1]> exclusive (entropy source), flags 0x10
2012-09-20T01:20:10.927Z cpu0:4147)VMK_VECTOR: 137: Added handler for vector 33, flags 0x10
2012-09-20T01:20:15.993Z cpu0:4147)<6>qlcnic 0000:04:00.0: Freed vmnic4[1] irq.
2012-09-20T01:20:23.884Z cpu21:5977)HBX: 2313: Waiting for timed out [HB state abcdef02 offset 3153920 gen 51 stampUS 65825493865 uuid 50596f1c-40094576-2894-000e1e059be0 jrnl <FB 5361600> drv 14.54] on vol 'P2000g3_R10'
2012-09-20T01:20:24.884Z cpu7:5970)HBX: 2313: Waiting for timed out [HB state abcdef02 offset 3153920 gen 51 stampUS 65825493865 uuid 50596f1c-40094576-2894-000e1e059be0 jrnl <FB 5361600> drv 14.54] on vol 'P2000g3_R10'
2012-09-20T01:20:26.885Z cpu12:5978)HBX: 2313: Waiting for timed out [HB state abcdef02 offset 3153920 gen 149 stampUS 65828494092 uuid 50596f1c-40094576-2894-000e1e059be0 jrnl <FB 1550000> drv 14.54] on vol 'P2000g3_R6'
2012-09-20T01:20:27.371Z cpu6:52361)ScsiCore: 63: Starting taskmgmt handler world 52361/2
2012-09-20T01:20:29.373Z cpu12:52363)ScsiCore: 63: Starting taskmgmt handler world 52363/3
2012-09-20T01:20:29.888Z cpu0:5979)HBX: 2313: Waiting for timed out [HB state abcdef02 offset 3153920 gen 149 stampUS 65828494092 uuid 50596f1c-40094576-2894-000e1e059be0 jrnl <FB 1550000> drv 14.54] on vol 'P2000g3_R6'
2012-09-20T01:20:31.395Z cpu10:4803)WARNING: iscsi_vmk: iscsivmk_TaskMgmtIssue: vmhba35:CH:1 T:3 L:0 : Task mgmt "Abort Task" with itt=0xc2490 (refITT=0xc2484) timed out.
2012-09-20T01:20:31.395Z cpu2:4290)ScsiCore: 97: Stopping taskMgmt handler world 42902
2012-09-20T01:20:35.986Z cpu0:4200)HBX: 2313: Waiting for timed out [HB state abcdef02 offset 3153920 gen 51 stampUS 65825493865 uuid 50596f1c-40094576-2894-000e1e059be0 jrnl <FB 5361600> drv 14.54] on vol 'P2000g3_R10'
2012-09-20T01:20:36.695Z cpu21:4803)WARNING: iscsi_vmk: iscsivmk_StopConnection: vmhba35:CH:1 T:3 CN:0: iSCSI connection is being marked "OFFLINE" (Event:4)
2012-09-20T01:20:36.695Z cpu21:4803)WARNING: iscsi_vmk: iscsivmk_StopConnection: Sess [ISID: 00023d00000a TARGET: iqn.1986-03.com.hp:storage.p2000g3.112512ec9b TPGT: 1 TSIH: 0]
2012-09-20T01:20:36.695Z cpu21:4803)WARNING: iscsi_vmk: iscsivmk_StopConnection: Conn [CID: 0 L: 10.0.10.16:56488 R: 10.0.10.10:3260]
2012-09-20T01:20:36.695Z cpu21:52361)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x412440383f00, 4120) to dev "naa.600c0ff00011347d9b20f74f01000000" on path "vmhba35:C1:T3:L0" Failed: H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-09-20T01:20:36.695Z cpu0:4805)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x88 (0x41244129d8c0, 6578) to dev "naa.600c0ff00011347d9b20f74f01000000" on path "vmhba35:C1:T3:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-09-20T01:20:36.695Z cpu21:52361)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600c0ff00011347d9b20f74f01000000" state in doubt; requested fast path state update...
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x41244129d8c0) 0x88, CmdSN 0x100 from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu21:52361)ScsiDeviceIO: 2322: Cmd(0x412440383f00) 0x2a, CmdSN 0x93d21 from world 4120 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x412441091b00) 0x88, CmdSN 0xd1 from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu0:4805)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x88 (0x4124403dc7c0, 6578) to dev "naa.600c0ff00011347d9b20f74f01000000" on path "vmhba35:C1:T3:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x4124403dc7c0) 0x88, CmdSN 0xd2 from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu21:4803)WARNING: iscsi_vmk: iscsivmk_TaskMgmtIssue: vmhba35:CH:1 T:3 L:0 : Task mgmt "Abort Task" with itt=0xc2492 (refITT=0xc2482) timed out.
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x4124411e31c0) 0x88, CmdSN 0xfb from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu0:4805)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x88 (0x4124403a9900, 6578) to dev "naa.600c0ff00011347d9b20f74f01000000" on path "vmhba35:C1:T3:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x4124403a9900) 0x88, CmdSN 0xc4 from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x4124403c86c0) 0x88, CmdSN 0xcb from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu0:4805)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x88 (0x412441918440, 6578) to dev "naa.600c0ff00011347d9b20f74f01000000" on path "vmhba35:C1:T3:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x412441918440) 0x88, CmdSN 0xfc from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x4124415263c0) 0x8a, CmdSN 0xed from world 6771 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu0:4805)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x8a (0x41244191ed40, 6771) to dev "naa.600c0ff00011347d9b20f74f01000000" on path "vmhba35:C1:T3:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-09-20T01:20:36.695Z cpu0:4805)ScsiDeviceIO: 2309: Cmd(0x41244191ed40) 0x8a, CmdSN 0xcf from world 6771 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu23:4803)WARNING: iscsi_vmk: iscsivmk_StopConnection: vmhba35:CH:5 T:3 CN:0: iSCSI connection is being marked "OFFLINE" (Event:4)
2012-09-20T01:20:36.695Z cpu23:4803)WARNING: iscsi_vmk: iscsivmk_StopConnection: Sess [ISID: 00023d00000e TARGET: iqn.1986-03.com.hp:storage.p2000g3.112512ec9b TPGT: 3 TSIH: 0]
2012-09-20T01:20:36.695Z cpu23:4803)WARNING: iscsi_vmk: iscsivmk_StopConnection: Conn [CID: 0 L: 10.0.10.16:56543 R: 10.0.10.11:3260]
2012-09-20T01:20:36.695Z cpu11:4805)ScsiDeviceIO: 2309: Cmd(0x4124403be5c0) 0x88, CmdSN 0xf7 from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-09-20T01:20:36.695Z cpu11:4805)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x88 (0x412441095b00, 6578) to dev "naa.600c0ff00011347d9b20f74f01000000" on path "vmhba35:C5:T3:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-09-20T01:20:36.695Z cpu11:4805)ScsiDeviceIO: 2309: Cmd(0x412441095b00) 0x88, CmdSN 0xd4 from world 6578 to dev "naa.600c0ff00011347d9b20f74f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
and on all the time:
2012-09-20T02:54:27.262Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.275Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.287Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.300Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.312Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.325Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.338Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.350Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.363Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.376Z cpu4:7367)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba35:C7:T3:L1, reservation state on device naa.600c0ff000113366f57c095001000000 is unknown.
2012-09-20T02:54:27.376Z cpu4:7367)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x16 (0x4124003d9540, 0) to dev "naa.600c0ff000113366f57c095001000000" on path "vmhba35:C7:T3:L1" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x2 0x3a 0x1. Act:EVAL
After a while he hangs on ESXi is dead, can only go to SSH.
If anyone faced a similar problem, prompt decision.
Sorry for my bad English.
Thank you!