Hello all you good people.
I am facing an issue powering multiple VMs on simultaneously beyond some limit.
I have script which I use to power on my VMs. The script is supposed to take a list & power on given number of VMs.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here is how the script looks like -
enable_luns=$1
vmlist_file=$2
i=0;
while [ $i -lt $enable_luns ];
do
i=`expr $i + 1`
vmid=$( head -n $i $vmlist_file | tail -n 1 | awk '{print $1}')
vm_name=$( head -n $i $vmlist_file | tail -n 1 | awk '{print $2}')
echo "$vmid $vm_name"
vim-cmd vmsvc/power.on $vmid &
done ## <= This is line 24 .
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
This works absolutely fine with 48 VMs.
However, if I try to power on more VMs, I get following errors -
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
# ./powerON_vms.sh 96 vmlist.txt
1 pegvm1
11 pegvm2
18 pegvm3
19 pegvm5
20 pegvm6
21 pegvm7
22 pegvm8
23 pegvm9
24 pegvm10
25 pegvm11
26 pegvm12
27 pegvm13
.
.
.
.
36 pegvm23
Powering on VM:
Powering on VM:
37 pegvm24
38 pegvm25
Powering on VM:
.
.
.
.
67 pegvm53
Powering on VM:
68 pegvm54
69 pegvm55
Powering on VM:
70 pegvm57
Failed to get current user.
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
71 pegvm58
Powering on VM:
72 pegvm59
73 pegvm60
Powering on VM:
Powering on VM:
74 pegvm6175 pegvm62
vim-cmd: error while loading shared libraries: libcrypt.so.1: cannot map zero-fill pages: Error 28
./powerON_vms.sh: line 24: can't fork
./powerON_vms.sh: line 24: can't fork
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
77 pegvm64
vim-cmd: error while loading shared libraries: libvmkctl.so: failed to map segment from shared object: Error 28
./powerON_vms.sh: line 24: can't fork
78
vim-cmd: error while loading shared libraries: libvmacore.so: failed to map segment from shared object: Error 28
./powerON_vms.sh: line 24: can't fork
./powerON_vms.sh: line 24: can't fork
80 pegvm66
./powerON_vms.sh: line 24: can't fork
pegvm67
82 pegvm69
83 pegvm70
vim-cmd: error while loading shared libraries: libvmkctl.so: failed to map segment from shared object: Error 28
84 pegvm71
85 pegvm72
vim-cmd: error while loading shared libraries: libvmkctl.so: failed to map segment from shared object: Error 28
Failed to get current user.
Powering on VM:
86 pegvm73
Powering on VM:
87 pegvm74
Powering on VM:
vim-cmd: error while loading shared libraries: libvmkctl.so: failed to map segment from shared object: Error 28
88
Powering on VM:
./powerON_vms.sh: line 24: can't fork
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
Is this because of the shell's ulimits? Am I sending too many processes to background? I doubled shell stack size (512 KB to 1024) & the number of processes (64 to 128); but it still fails.
It is 32 core machine. All VMs are assigned 2GB memory (ESX has sufficient memory). It has ESX5.1.
What can be the issue? something wrong with vim-cmd? Appreciate your help.
